Run a document processing pipeline via API: POST /v1/pipelines compiles a configured Spec's rail and runs extraction, resolution, and validation over documents.
POST /v1/pipelines is Talonic's document processing pipeline API: it runs a configured Spec over a set of documents in one call. A Spec is a Schema plus a composed rail — Source → Field Registry → Extraction → Resolution → Validation → Data Product — together with its Data Policies and validation gates. A pipeline is one run of that Spec on the engine. You name the Spec by its schema_id and pass the documents to run. The server does the rest: it compiles the Spec's saved rail into the pipeline's phase config, so you never hand-build phases. Every Spec compiles to the same engine.
Compilation is server-side and governed. The rail expands into one resolution phase per active policy, one validation phase per member gate at its checkpoint position, an enforced extraction phase, and a pipeline-scoped assembly step appended after all documents finish. The run is created, the documents are attached, and processing starts immediately. The response status is active.
This is the governed tier. For a quick one-off job that does not justify configuring a Spec, use POST /v1/jobs instead: jobs run the standard 4-phase pipeline against a schema with no saved rail, no policy-driven resolution phases, and no validation checkpoints. Pipelines are the right choice when you have curated a Spec with resolution policies and positional validation gates and want review holdback enforced.
A pipeline's row results are read through the data product it produces. After the run finishes, call POST /v1/pipelines/{id}/data-product and then read rows through the data-products endpoints. The data product is also where per-cell review holdback is enforced: fields a blocking gate parked for review surface with status only until a reviewer resolves them. This endpoint requires an API key with the write scope.
The pipeline_mode parameter controls how the call targets pipelines. The default, new, always creates a fresh pipeline. append adds the documents to the newest eligible existing pipeline on the Spec instead, so repeated calls accumulate into one growing table — the response then reports appended: true and reuses that pipeline's id. Omitted, the Spec's own Source-stage setting applies. Every call also returns a run_id: the submission-group id stamped on this call's documents, usable as the run_id filter on [GET /v1/pipelines/{id}/results](get-pipeline-results) to read back exactly what this call submitted.
If the named Spec has no composed rail, the request fails with 400 bad_request. Compose the rail via PUT /v1/schemas/{id}/rail or in the app first, or use POST /v1/jobs for an ad-hoc run that needs no saved rail.
POST/v1/pipelines
Body parameters
schema_id*stringThe Spec to run (a user_schema UUID). Must have a composed Spec rail.
document_ids*string[]Document UUIDs to run through the pipeline. At least 1; more than 10,000 fails validation with 400, and a run above the tested per-run maximum (configurable, default 3,000) is rejected with 413 — split it and retry.
namestringOptional run name (1-200 characters). Defaults to the Spec name plus the current date.
pipeline_modestringOptional: "new" or "append". append adds the documents to the newest eligible existing pipeline on the Spec instead of creating one; new always creates. Omitted → the Spec's own Source-stage setting (default new).
idstringPipeline run UUID. Under append mode this is the existing pipeline the documents were added to.
statusstringAlways "active" immediately after creation.
schemaobjectThe Spec used for this run: { id, name }.
document_countintegerNumber of documents requested in document_ids.
enqueued_documentsintegerNumber of documents actually enqueued for processing.
appendedbooleantrue when the documents were appended to an existing pipeline instead of a new one being created.
run_idstringThe submission-group id stamped on this call's documents — pass it as the run_id filter on GET /v1/pipelines/{id}/results to read back this call's own rows.
messagestringHuman-readable confirmation message (created, or appended to an existing pipeline).
links.selfstringURL to fetch the pipeline run.
links.progressstringURL to poll phase-by-phase progress.
links.data_productstringURL to produce a data product once the run finishes.
400bad_requestThe Spec has no composed pipeline rail, the rail compiled to zero executable phases, or the request body failed validation (including more than 10,000 document_ids).
401unauthorizedMissing or invalid API key.
404not_foundNo Spec (schema) with this schema_id exists for your organization.
413PIPELINE_RUN_TOO_LARGEThe run exceeds the tested per-run document maximum (default 3,000). Split the document set across multiple runs and retry.
429rate_limited / PIPELINE_QUEUE_OVERLOADEDDaily quota exhausted, or the processing queue is at its tested outstanding-document envelope. Both are retryable — wait and resubmit (the queue error names a retry-after period in its message).
503PIPELINE_QUEUE_UNAVAILABLEQueue depth could not be verified safely (transient infrastructure state). Retryable — resubmit shortly.
Frequently asked questions
How is this different from POST /v1/jobs?+−
A pipeline compiles a configured Spec's saved rail server-side: resolution policies become resolution phases and validation gates become positional checkpoints with review holdback. A job is the quick one-off path that runs the standard 4-phase pipeline against a schema with no saved rail, no policy phases, and no checkpoints.
Why does my run fail with 400 even though the schema exists?+−
The schema exists but has no composed Spec rail. Compose its stages via `PUT /v1/schemas/{id}/rail` or in the app, or run the documents through `POST /v1/jobs` instead, which needs no rail.
Where are the extracted rows?+−
Pipeline rows are read through the data product the run produces. Call `POST /v1/pipelines/{id}/data-product` after the run finishes, then read rows through the data-products endpoints. That path is also where per-cell review holdback is enforced.
How many documents can one pipeline run process?+−
Two limits apply: more than 10,000 document_ids fails request validation with 400, and a run above the tested per-run maximum (configurable, default 3,000) is rejected with 413 PIPELINE_RUN_TOO_LARGE before any work starts. For larger corpora, split the document set across multiple runs of the same Spec — or use append mode so the splits still land in one table.
What does pipeline_mode=append do?+−
It adds the call's documents to the newest eligible existing pipeline on the Spec instead of creating a new one, so bulk uploads accumulate into a single growing table. The response then carries appended: true and the existing pipeline's id, and the returned run_id lets you read back just this call's rows via the run_id filter on GET /v1/pipelines/{id}/results.