Email Event Ingestion Platform

A lightweight internal service that captures Customer.io email events in real time, stores structured metadata in Postgres, and exposes a fast lookup API for iCanSee.

Real-time webhooks Async queue processing Idempotent event storage iCanSee-ready API S3 attachment storage

Architecture

The platform sits between Customer.io and iCanSee. Webhooks are received, validated, queued, and processed asynchronously.

Customer.ioSends webhook

→

Webhook APIValidate & enqueue

→

Redis QueueBullMQ

→

WorkerProcess & store

→

PostgresSupabase

S3 StorageAttachments

←

WorkerUpload files

iCanSeeLookup API

←

PostgresQuery

ⓘ

Key principle: The webhook returns 202 Accepted immediately. All heavy processing happens in the background worker. This means Customer.io never times out waiting for a response.

Webhook Setup

Instructions for the Customer.io team.

Customer.io Team

Configure the reporting webhook

In Customer.io, set up a reporting webhook that sends email events to our ingestion endpoint.

Setting	Value
Endpoint URL	`https://email-insights.hipp.co/webhooks/customerio/email-events`
Method	`POST`
Content-Type	`application/json`
Events to enable	sent, delivered, opened, clicked, bounced, dropped, spammed, unsubscribed
Signing secret	Optional but recommended. If configured, the platform verifies the `x-cio-signature` HMAC header.

⚠

Important: The endpoint returns 202 Accepted on success. If Customer.io receives anything other than a 2xx response, it will retry the webhook. Our platform handles deduplication, so retries are safe.

Customer.io Team

Webhook payload format

Each webhook POST should include the following JSON body. All fields are validated on receipt.

Field	Type	Notes
`event_type`	string	Required — One of the supported event types
`timestamp`	integer	Required — Unix epoch seconds
`message_id`	string	Required — Customer.io message identifier
`customer_id`	string	Required — Your internal customer/contact ID
`email_address`	string	Required — Recipient email address
`from_address`	string	Optional — Sender address
`subject`	string	Optional — Email subject line
`transactional_message_id`	string	Optional — If this was a transactional send
`cc`	string[]	Optional — CC recipients
`bcc`	string[]	Optional — BCC recipients
`crm_entity_type`	string	Optional — e.g. "case", "order", "finance_agreement"
`crm_entity_id`	string	Optional — The CRM record this email relates to
`attachments`	array	Optional — Attachment metadata (see below)

Attachment object

Field	Type	Notes
`filename`	string	Required
`mime_type`	string	Optional — defaults to application/octet-stream
`size_bytes`	integer	Optional
`content_base64`	string	Optional — Base64-encoded file content (will be uploaded to S3)

Example payload

// POST /webhooks/customerio/email-events { "event_type": "sent", "timestamp": 1711209600, "message_id": "msg_abc123", "customer_id": "cust_456", "email_address": "john.smith@example.com", "from_address": "hello@hippo.co.uk", "subject": "Your order confirmation", "crm_entity_type": "order", "crm_entity_id": "ORD-78901" }

Success response

// 202 Accepted { "status": "accepted", "dedupeHash": "a1b2c3d4e5..." }

Customer.io Team

Supported event types

The platform accepts and stores the following Customer.io email event types:

Event	Description	When it fires
`sent`	Email accepted for delivery	Customer.io hands off to the mail provider
`delivered`	Email delivered to inbox	Mail provider confirms delivery
`opened`	Recipient opened the email	Tracking pixel loaded
`clicked`	Recipient clicked a link	Tracked link followed
`bounced`	Email bounced	Hard or soft bounce from mail provider
`dropped`	Email dropped before send	Suppression, invalid address, etc.
`spammed`	Marked as spam	Recipient reported spam
`unsubscribed`	Recipient unsubscribed	Unsubscribe link clicked
`converted`	Conversion attributed	Goal/conversion tracked
`failed`	Send failed	Delivery could not be attempted

Customer.io Team

Idempotency & retries

Every incoming event is deduplicated using a hash of message_id + event_type + timestamp. If the same event is sent twice (e.g. due to a retry), the duplicate is silently ignored.

✓

Safe to retry. Customer.io can retry failed webhook deliveries as many times as needed. The platform will never create duplicate event records.

The first event for a given message_id creates the email record. Subsequent events (delivered, opened, clicked, etc.) update the latest_status field and add to the event timeline.

iCanSee Lookup Endpoint

The primary endpoint for iCanSee integration.

iCanSee

GET /api/crm/lookup

Look up all email history for a contact by their email address or customer ID. This is the main endpoint iCanSee should call when viewing a contact's communication history.

Query parameters

Param	Type	Notes
`email`	string	Recipient email address to search for
`customerId`	string	Internal customer ID to search for
`page`	integer	Optional — default 1
`limit`	integer	Optional — default 20, max 100

⚠

At least one of email or customerId must be provided. You can provide both to narrow results.

Example requests

# Lookup by email address GET /api/crm/lookup?email=john.smith@example.com # Lookup by customer ID GET /api/crm/lookup?customerId=cust_456 # Both, with pagination GET /api/crm/lookup?email=john.smith@example.com&customerId=cust_456&page=1&limit=10

Example response

{ "data": [ { "id": "a1b2c3d4-...", "customerId": "cust_456", "provider": "customerio", "providerMessageId": "msg_abc123", "subject": "Your order confirmation", "from": "hello@hippo.co.uk", "to": "john.smith@example.com", "sentAt": "2024-03-23T12:00:00.000Z", "latestStatus": "delivered", "crmEntityType": "order", "crmEntityId": "ORD-78901", "totalEvents": 3, "totalAttachments": 1, "recentEvents": [ { "type": "opened", "timestamp": "2024-03-23T14:30:00.000Z" }, { "type": "delivered", "timestamp": "2024-03-23T12:01:00.000Z" }, { "type": "sent", "timestamp": "2024-03-23T12:00:00.000Z" } ], "attachments": [ { "id": "x1y2z3...", "filename": "invoice.pdf", "mimeType": "application/pdf", "sizeBytes": 45230 } ] } ], "pagination": { "page": 1, "limit": 20, "total": 1, "totalPages": 1 } }

iCanSee

GET /api/crm/lookup/:emailId

Get the full detail for a single email, including the complete event timeline and all attachment metadata.

Example request

GET /api/crm/lookup/a1b2c3d4-e5f6-7890-abcd-ef1234567890

Example response

{ "data": { "id": "a1b2c3d4-...", "customerId": "cust_456", "subject": "Your order confirmation", "from": "hello@hippo.co.uk", "to": "john.smith@example.com", "sentAt": "2024-03-23T12:00:00.000Z", "latestStatus": "opened", "crmEntityType": "order", "crmEntityId": "ORD-78901", "timeline": [ { "id": "...", "type": "sent", "timestamp": "2024-03-23T12:00:00.000Z" }, { "id": "...", "type": "delivered", "timestamp": "2024-03-23T12:01:00.000Z" }, { "id": "...", "type": "opened", "timestamp": "2024-03-23T14:30:00.000Z" } ], "attachments": [ { "id": "x1y2z3...", "filename": "invoice.pdf", "mimeType": "application/pdf", "sizeBytes": 45230, "sha256": "e3b0c44298..." } ] } }

iCanSee

GET /api/emails/search

General-purpose search across all stored emails. Useful for building search UI or running reports.

Param	Type	Description
`to`	string	Filter by recipient email (partial match)
`subject`	string	Filter by subject (partial match, case-insensitive)
`status`	string	Filter by latest status (e.g. delivered, bounced)
`customerId`	string	Filter by customer ID
`crmEntityType`	string	Filter by CRM entity type (e.g. case, order)
`crmEntityId`	string	Filter by CRM entity ID
`fromDate`	ISO date	Emails sent on or after this date
`toDate`	ISO date	Emails sent on or before this date
`page`	integer	Page number (default 1)
`limit`	integer	Results per page (default 20, max 100)

Example

GET /api/emails/search?to=john@example.com&status=delivered&fromDate=2024-01-01

iCanSee

Working with attachments

Attachments are stored in S3-compatible object storage. The database stores metadata only. To download a file, request a time-limited signed URL.

List attachments

GET /api/emails/:emailId/attachments

Get a signed download URL

POST /api/emails/:emailId/attachments/:attachmentId/sign-download

Response

{ "data": { "attachmentId": "x1y2z3...", "filename": "invoice.pdf", "mimeType": "application/pdf", "sizeBytes": 45230, "downloadUrl": "https://s3.../invoice.pdf?X-Amz-Signature=...", "expiresInSeconds": 3600 } }

ⓘ

Signed URLs expire after 1 hour. Request a new one each time a user wants to download. Do not cache these URLs.

iCanSee

Authentication

All /api/* endpoints (except health check) require an API key when API_KEY is configured on the server.

How to authenticate

Include the key in one of two ways:

# Option 1: x-api-key header GET /api/crm/lookup?email=john@example.com x-api-key: your-api-key-here # Option 2: Bearer token GET /api/crm/lookup?email=john@example.com Authorization: Bearer your-api-key-here

Error response (401)

{ "error": "Unauthorized: invalid or missing API key" }

⚠

The webhook endpoint (/webhooks/...) does not use API key auth. It uses Customer.io's own HMAC signature verification instead.

All Endpoints

Complete endpoint reference.

Both Teams

Method	Path	Description
POST	`/webhooks/customerio/email-events`	Ingest a Customer.io email event
GET	`/api/crm/lookup`	CRM lookup by email address or customer ID
GET	`/api/crm/lookup/:emailId`	Full email detail with timeline + attachments
GET	`/api/crm/cases/:caseId/emails`	Emails linked to a CRM case
GET	`/api/customers/:customerId/emails`	List all emails for a customer
GET	`/api/customers/:customerId/emails/:emailId`	Single email detail (scoped to customer)
GET	`/api/emails/:emailId/events`	Event timeline for an email
GET	`/api/emails/:emailId/attachments`	List attachments for an email
POST	`/api/emails/:emailId/attachments/:attachmentId/sign-download`	Generate signed download URL
GET	`/api/emails/search`	Search emails with filters
GET	`/api/health/customerio-ingestion`	Health check (Postgres + Redis)

Database Schema

Three tables in Supabase Postgres.

emails

id uuid PK
customer_id string
crm_entity_type string?
crm_entity_id string?
provider string
provider_message_id string
transactional_message_id string?
subject string?
from_email string
to_email string
cc_json json?
bcc_json json?
sent_at datetime?
latest_status string
html_snapshot_url string?
created_at datetime
updated_at datetime

email_events

id uuid PK
email_id uuid FK
dedupe_hash string
event_type string
event_timestamp datetime
raw_payload_json json?
created_at datetime

email_attachments

id uuid PK
email_id uuid FK
filename string
mime_type string
size_bytes integer
object_storage_key string
download_url_cached string?
sha256 string?
created_at datetime

ⓘ

Indexes: The emails table is indexed on customer_id, to_email, crm_entity_type + crm_entity_id, latest_status, and sent_at for fast CRM queries. The dedupe_hash column on email_events has a unique constraint for idempotent processing.

Both Teams

GET /api/health/customerio-ingestion

Returns the health status of the ingestion platform, including Postgres and Redis connectivity.

{ "status": "healthy", "timestamp": "2024-03-23T16:00:00.000Z", "checks": { "postgres": { "status": "ok", "latencyMs": 12 }, "redis": { "status": "ok", "latencyMs": 8 } } }

Returns 200 when healthy, 503 when degraded.

Email Event Ingestion Platform — Internal Developer Documentation