Integration Guide

This guide provides detailed instructions for setting up and configuring data source integrations.

Overview

The platform supports multiple integration types for ingesting lead data:

• API — REST API integrations
• SFTP — File-based integrations via SFTP
• Email — Email-based data delivery (IMAP)
• Webhook — Real-time webhook endpoints
• CSV — Manual CSV file uploads

API integrations

Supported providers

• LeadPost API — Website visitor identification
• Buyer Intent Providers — Bombora, 6sense, etc.
• Custom APIs — Any REST API with JSON responses

Configuration

1. Navigate to Settings > Integrations > Add Integration

2. Select API as the integration type

3. Enter configuration:

   • Base URL — API endpoint base URL (e.g. https://api.leadpost.com/v1)

   • API Key — Authentication key (found in provider dashboard; keep secure, never commit to code)

   • Authentication Type:

     • Bearer Token — Authorization: Bearer {token}
     • Basic Auth — Authorization: Basic {base64}
     • Header — Custom header (e.g. X-API-Key)

   • Headers (optional) — Additional headers as JSON:

     { "Content-Type": "application/json", "Accept": "application/json" }

4. Set Sync Frequency:

   • Hourly — For real-time data (LeadPost)
   • Daily — For daily updates (Buyer Intent)
   • Weekly — For weekly batches
   • Manual — Trigger manually

5. Click Test Connection to verify

6. Click Create Integration

LeadPost API setup

Prerequisites: LeadPost account, API key from LeadPost dashboard

Type: API
Name: LeadPost API
Base URL: https://api.leadpost.com/v1
API Key: [Your LeadPost API Key]
Auth Type: Bearer Token
Sync Frequency: Hourly

Data mapping — the LeadPost worker automatically maps fields:

• visitor_email → email
• company_name → companyName
• phone → phone
• location → city, state, zipCode

Buyer Intent provider setup

Prerequisites: Buyer Intent Provider account (Bombora, 6sense, etc.), API credentials

Type: API
Name: Buyer Intent Provider
Base URL: https://api.provider.com/v1
API Key: [Your API Key]
Auth Type: Bearer Token
Sync Frequency: Daily

Data mapping:

• Intent topics mapped to intentTopics array
• Intent scores mapped to intentScore
• Company data mapped to firmographic fields

Custom API integration

For custom APIs, you may need to create a custom worker:

1. Create worker in workers/ingestion/custom-api.ts
2. Implement data fetching and mapping
3. Create sync endpoint in src/pages/api/integrations/custom/sync.ts
4. Add sync button in integrations UI

SFTP integrations

Configuration

1. Navigate to Settings > Integrations > Add Integration
2. Select SFTP as the integration type
3. Enter configuration:
   • Host — SFTP server hostname (e.g. sftp.example.com)
   • Port — SFTP port (default: 22)
   • Username — SFTP username
   • Password — SFTP password (consider using SSH keys for better security)
   • Remote Path — Path to files on SFTP server (e.g. /data/leads/)
4. Set Sync Frequency — Daily, Weekly, or Manual
5. Click Test Connection to verify
6. Click Create Integration

File format

SFTP integrations expect CSV or JSON files.

CSV format:

email,first_name,last_name,company_name,phone,city,state,zip_code
john@example.com,John,Doe,Acme Corp,555-1234,San Francisco,CA,94102

JSON format:

[{ "email": "john@example.com", "firstName": "John", "lastName": "Doe", "companyName": "Acme Corp", "phone": "555-1234", "city": "San Francisco", "state": "CA", "zipCode": "94102" }]

Security best practices

• Use SSH keys instead of passwords when possible
• Restrict SFTP user to read-only access
• Use separate credentials for each integration
• Rotate credentials regularly

Email integrations

Configuration

1. Navigate to Settings > Integrations > Add Integration
2. Select Email as the integration type
3. Enter configuration:
   • Email Address — Email account to monitor (e.g. leads@example.com)
   • Password — Email account password (use app-specific password for Gmail)
   • IMAP Host — IMAP server hostname (Gmail: imap.gmail.com, Outlook: outlook.office365.com)
   • IMAP Port — IMAP port (default: 993 for SSL)
4. Set Sync Frequency — Hourly, Daily, or Manual
5. Click Test Connection to verify
6. Click Create Integration

Email format

Emails should contain CSV or JSON attachments:

• Subject Line — Used for metadata (optional)
• Attachment — CSV or JSON file with lead data
• Body — Can contain metadata or be ignored

Gmail setup

For Gmail, use an app-specific password:

1. Go to Google Account settings
2. Enable 2-factor authentication
3. Generate app-specific password
4. Use app password in integration config

Webhook integrations

Configuration

1. Navigate to Settings > Integrations > Add Integration
2. Select Webhook as the integration type
3. Configuration is auto-generated:
   • Webhook URL — Auto-generated unique URL
   • Webhook Secret — Auto-generated secret for verification
4. Copy the webhook URL and secret
5. Configure the external system to send data to the webhook URL
6. Include the secret in webhook requests (header: X-Webhook-Secret)

Webhook endpoint

Webhook URL format:

https://yourdomain.com/api/integrations/webhook/{webhookId}

Webhook payload

Expected JSON payload:

{ "leads": [{ "email": "john@example.com", "firstName": "John", "lastName": "Doe", "companyName": "Acme Corp", "phone": "555-1234", "city": "San Francisco", "state": "CA", "zipCode": "94102" }] }

Security

• Webhook secret is required for verification
• Include secret in X-Webhook-Secret header
• Use HTTPS for webhook URLs
• Validate payload structure

CSV import

Manual import

1. Navigate to Leads > Import CSV
2. Upload CSV file (max 10 MB)
3. Review column mapping (auto-detected)
4. Choose auto-process option
5. Click Import

CSV format

Required columns: email (or company_name if B2B)

Optional columns:

• first_name, last_name
• phone
• company_name, domain
• city, state, zip_code
• industry, company_size
• intent_topics (comma-separated)
• intent_score

Column mapping

The system auto-detects common column names:

• email, e-mail, Email Address
• first_name, firstName, First Name
• company_name, companyName, Company

You can manually adjust mappings if needed.

Testing integrations

Test connection

Use the Test Connection button to verify:

• API credentials are valid
• SFTP connection works
• Email credentials are correct
• Webhook endpoint is accessible

Manual sync

For API integrations, use the Sync button to:

• Trigger immediate data sync
• Test integration without waiting for schedule
• Verify data is being imported correctly

Monitoring

Check integration status:

• Last Sync — When last sync occurred
• Status — Success, In Progress, or Failed
• Error Count — Number of failed syncs
• Records Imported — Number of leads imported

Troubleshooting

API integration issues

Problem: Connection test fails

• Verify API key is correct
• Check base URL is correct
• Verify authentication type matches provider
• Check network connectivity

Problem: No data imported

• Check API endpoint returns data
• Verify data format matches expected schema
• Check integration logs for errors
• Verify sync frequency is set correctly

SFTP integration issues

Problem: Connection timeout

• Verify host and port are correct
• Check firewall allows SFTP connections
• Verify credentials are correct
• Test connection with an SFTP client

Problem: Files not found

• Verify remote path is correct
• Check file permissions on SFTP server
• Ensure files match expected format
• Check sync frequency

Email integration issues

Problem: Cannot connect to IMAP

• Verify IMAP host and port are correct
• Check email credentials
• For Gmail, use app-specific password
• Verify IMAP is enabled in email account

Problem: No emails processed

• Check email account has new emails
• Verify email format matches expected
• Check attachment format (CSV/JSON)
• Review integration logs

Webhook integration issues

Problem: Webhook not receiving data

• Verify webhook URL is correct
• Check external system is sending to correct URL
• Verify webhook secret is included
• Check webhook endpoint logs

Problem: Data not processing

• Verify payload format matches expected
• Check webhook secret is correct
• Review webhook endpoint logs
• Verify data validation passes

Best practices

Security

• Use strong credentials — Strong passwords for all integrations
• Rotate credentials — Regularly rotate API keys and passwords
• Limit access — Use read-only credentials when possible
• Monitor access — Review integration logs regularly

Performance

• Sync frequency — Set appropriate sync frequency (not too frequent)
• Batch size — Process data in batches for large imports
• Error handling — Implement retry logic for transient failures
• Monitoring — Monitor sync performance and errors

Data quality

• Validation — Validate data before import
• Normalization — Normalize data format (email, phone, addresses)
• Deduplication — Let the system handle duplicate detection
• Error handling — Review and fix data errors promptly

Integration examples

LeadPost API

{ "type": "api", "name": "LeadPost API", "config": { "baseUrl": "https://api.leadpost.com/v1", "apiKey": "your-api-key", "authType": "bearer" }, "syncFrequency": "hourly" }

SFTP integration

{ "type": "sftp", "name": "Partner SFTP", "config": { "host": "sftp.partner.com", "port": 22, "username": "leads_user", "password": "secure-password", "remotePath": "/data/leads/" }, "syncFrequency": "daily" }

Webhook integration

{ "type": "webhook", "name": "Partner Webhook", "config": { "webhookUrl": "https://yourdomain.com/api/integrations/webhook/abc123", "webhookSecret": "secret-key-here" }, "syncFrequency": "manual" }

Support

For integration issues:

1. Check integration logs
2. Review error messages
3. Test connection manually
4. Contact integration provider support
5. Review platform documentation