Every pipeline in Talonic starts with a Spec: a reusable configuration that composes a rail of stages over one schema. Running the Spec compiles that rail into a pipeline that processes each document through its stages in order, filling a structured table with per-cell confidence scores and provenance traces.
The rail
Stage
Purpose
Source
Select the documents the run reads — uploads, connectors, or API ingestion
Field Registry
Map schema fields to registry fields discovered across your corpus
Extraction
Locate and extract values from each document, with confidence and source spans
Decide matches against reference data: auto_match, shortlist, or no_match
Agent (optional)
Park document-scoped tasks for an external agent over MCP
Validation
Type, format, and coherence checks; failures route to the review queue
Assembly
Compose per-document records into the final table
Data Product
The terminal, governed output other systems consume
Only Source, Extraction, and the terminal Data Product are mandatory — the rest compose per Spec. The full stage-by-stage guide lives in the Platform Guide.
Confidence and review
Every cell carries a confidence score and a reasoning trace. Validation checkpoints flag cells and records that fail checks into a review queue, where a reviewer sees the value beside its highlighted source region and accepts or corrects it. Corrections are recorded with full audit history.
What is the difference between a Spec, a Pipeline, and a Job?+−
A Spec is a reusable configuration over one schema: the composed rail, attached Data Policies, validation checkpoints, and assembly rule. A Pipeline is one compiled run of that Spec over a set of documents. A Job is the quick one-time path for ad-hoc extraction that does not justify a Spec. Specs and Pipelines are the production path; Jobs are the fast path.
What engine runs a compiled Spec?+−
A compiled Spec runs on the engine: a per-document phase runner over seven phase types. Each document passes through the phases the rail composes — Transfer (registry fill), Extraction (AI for gaps), Matching (decisive reference linkage), Agent (external-agent tasks via MCP), Resolution (Data Policies), and Validation (your gates) — in strict sequence, with documents processed in parallel, and Assembly composing groups once all documents are terminal. It is distinct from the 4-phase engine behind quick Jobs and writes every value as a versioned cell in the value plane.
Do I need to call a start endpoint after creating a pipeline?+−
No. The public POST /v1/pipelines endpoint compiles the Spec rail and enqueues the documents in a single call. The response status is active and enqueued_documents reflects how many were queued. Poll GET /v1/pipelines/{id}/progress to watch each phase advance.
How do I change the column order of a data product?+−
Reorder the fields in the Spec composer by dragging each row by its handle, then publish. A publish writes the field order to the schema, and the data product grid orders its columns by that published position. Unpublished draft order has no effect downstream.
Why does my pipeline run fail with "no composed rail"?+−
A pipeline needs the Spec to have a rail to compile. Set one with PUT /v1/schemas/{id}/rail. A minimal runnable rail is a single extraction stage. If you only need a one-off Job without configuring a Spec, use POST /v1/jobs instead.