Skip to main content

How Delivery Works

Push extracted, resolved, and reviewed data to any downstream system. Delivery is a typed, at-least-once pipeline with idempotency keys on the wire, append-only history, and a dead-letter queue for terminal failures. The system is fully configurable without code changes — signals, deliverable resolvers, serializers, and connectors are four orthogonal registries that compose independently, so adding a new destination type never requires changes to the signal or serializer code.

Every delivery flows through a five-stage pipeline. Producers are stateless — they only publish typed events into an outbox and never interact with destinations or bindings directly. A background poller drains the outbox every 5 seconds (configurable via delivery.poll_interval_ms), claiming up to 50 rows per tick using FOR UPDATE SKIP LOCKED for safe multi-instance operation. When the BullMQ queue depth exceeds the backpressure threshold (default 10,000), the poller pauses until the queue drains, preventing memory exhaustion under burst load. Matched events are enqueued as delivery jobs processed by workers (default concurrency: 10):

The delivery pipeline

ParameterTypeDescription
1. SignaleventA producer emits a typed event (e.g. document.extracted, result.approved) into the outbox. Producers are stateless — they only publish.
2. BindingmatchA poller drains the outbox and matches each event against active bindings. A binding joins a signal filter to a deliverable + destination + serializer.
3. ResolverloadThe deliverable resolver loads the payload (document metadata, a record snapshot, an extraction run, ...) at delivery time using only entity IDs from the signal.
4. SerializerencodeThe serializer encodes the payload into the wire format — json, ndjson, csv, csv_file, xlsx, rows, graph, raw, md, or txt — after an optional field_map projection.
5. ConnectortransportThe connector ships the encoded bytes through the TransportWrapper (SSRF guard, payload cap, rate limit, retry ladder). Slice-1 connector is webhook.

Every attempt is logged in delivery_items. Terminal failures (retry exhausted or permanent 4xx) write a delivery_dead_letter row, which is replayable. The outbox, history, DLQ, and catalog are all accessible via the `/v1/delivery/*` API.

The four registries — signals, deliverables, serializers, and connectors — are fully orthogonal. Adding a new destination type does not require changes to the signal or serializer code. This composable design means you can mix any supported signal with any compatible serializer and connector without custom integration work.

For best results, start with a webhook destination to verify your binding configuration end-to-end. Once the payload shape and delivery cadence match your expectations, expand to file-based destinations (S3, SFTP) or spreadsheet destinations (Google Sheets). Most teams create separate bindings for different downstream consumers rather than routing all events to a single destination.

Create a webhook destination and binding end-to-end
# 1) Create the destination
curl -X POST https://api.talonic.com/v1/delivery/destinations \
  -H "Authorization: Bearer $TALONIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Ops Webhook",
    "type": "webhook",
    "config": { "url": "https://ops.example.com/talonic", "method": "POST" },
    "signing_secret": "whsec_rotate_monthly"
  }'
# -> { "id": "dest_001", "is_active": true }

# 2) Create a binding
curl -X POST https://api.talonic.com/v1/delivery/bindings \
  -H "Authorization: Bearer $TALONIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Notify on extraction",
    "signal_filter": { "event_type": "document.extracted" },
    "deliverable_type": "notification",
    "destination_id": "dest_001",
    "serializer_format": "json"
  }'

# 3) Test the destination
curl -X POST https://api.talonic.com/v1/delivery/destinations/dest_001/test \
  -H "Authorization: Bearer $TALONIC_API_KEY"
# -> { "success": true, "duration_ms": 142 }

The delivery pipeline is designed for zero-code configuration. Adding a new destination, creating a binding, and testing the connection can all be done via the API or dashboard without writing any integration code. The compatibility triangle validation ensures that your binding configuration is valid before any events are routed — you never end up with a misconfigured binding that silently drops deliveries. Every attempt is logged with full request/response details, and terminal failures are captured in the dead-letter queue for replay, giving you complete visibility into your delivery pipeline.

Delivery is at-least-once with deterministic idempotency keys. Receivers should use the X-Talonic-Idempotency-Key header (or equivalent metadata for file-based connectors) to deduplicate on their end.

Frequently asked questions

How does the delivery pipeline work?+
Five stages: Signal (emit event), Binding (match to destination), Resolver (load payload), Serializer (encode to format), Connector (transport via webhook, S3, etc.). Delivery is at-least-once with idempotency keys.
What happens when a delivery fails?+
Failed deliveries retry with a backoff ladder. Terminal failures (retry exhausted or permanent 4xx) are written to the dead-letter queue (DLQ), which is fully replayable.
What serialization formats are supported?+
Ten formats: json, ndjson, csv, csv_file, xlsx, rows, graph, raw, md, and txt. Each serializer declares which deliverable shapes it supports, and the compatibility triangle validates the combination at binding creation time.
What is the default retry ladder for failed deliveries?+
The default retry ladder uses 7 attempts over approximately 10 hours with exponential backoff: 0s (immediate), 30s, 2min, 8min, 30min, 2h, 8h. After all attempts are exhausted, the delivery moves to the dead-letter queue. You can override the retry schedule per binding via the delivery_policy field.

Related

Destinations
Bindings
Signals & Catalog