Skip to main content

Reference Primitives

Reference primitives (reference data) are versioned lookup tables shared across your Talonic workspace. Upload key-value reference data once and reuse it across multiple schemas for consistent code mapping and value resolution.

Manage reference data from the Reference Data tab on the Documents page; the manager lives at /sources/reference-data. Datasets can be uploaded as CSV or XLSX files, or created from JSON rows via POST /v1/reference-data. Each reference dataset is versioned independently, and schemas reference a specific version to ensure stable resolution.

The lookup convention follows a key / value structure where the key is the output code and the value is the human-readable label. During extraction, the platform maps FROM labels found in documents TO the canonical codes defined in the reference dataset. This ensures consistent, machine-readable output regardless of how values appear in source documents.

Because reference data now sits on its own tab beside the document list, the tables your documents are checked against live next to the documents themselves. A typical setup step is to open Documents, switch to Reference Data, upload a CSV or XLSX of the codes your downstream system accepts, and then bind that dataset to the matching schema fields in the Spec composer. Keeping the code list in one versioned dataset means every Spec that binds it maps labels to the same codes, and an update is a new version you adopt deliberately rather than a silent change.

For best results, keep reference datasets focused on a single domain: for example, one dataset for country codes, another for currency codes, and another for product categories. This makes each dataset reusable across multiple schemas and simplifies maintenance. When updating a dataset, test the new version against a few sample documents before updating the version reference in production schemas.

Create and version a reference dataset
# Create a reference dataset from JSON rows (requires write scope).
# Each row is an object; "columns" optionally fixes the column order.
curl -X POST https://api.talonic.com/v1/reference-data \
  -H "Authorization: Bearer $TALONIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Country Codes",
    "columns": ["code", "label"],
    "data": [
      { "code": "US", "label": "United States" },
      { "code": "US", "label": "USA" },
      { "code": "DE", "label": "Germany" },
      { "code": "DE", "label": "Deutschland" },
      { "code": "GB", "label": "United Kingdom" },
      { "code": "GB", "label": "UK" }
    ]
  }'

# List reference datasets:
curl -s https://api.talonic.com/v1/reference-data \
  -H "Authorization: Bearer $TALONIC_API_KEY"

# Browse rows in a dataset (paginated with page/limit, limit capped at 500):
curl -s "https://api.talonic.com/v1/reference-data/rd_001/rows?page=1&limit=100" \
  -H "Authorization: Bearer $TALONIC_API_KEY"
Versioning protects production stability. When you update a reference dataset, existing schemas continue using their pinned version until you explicitly update the version reference. This prevents unexpected changes to live extraction pipelines.

Reference data is the centralized source of truth for code mapping across your workspace. Unlike schema-level reference tables that are defined inline and scoped to a single schema, workspace-level reference datasets are shared resources that any schema can reference. This eliminates duplication: instead of maintaining identical country code tables across five schemas, you maintain one dataset and reference it from all five. When a new country or code variation needs to be added, you update the dataset once and bump the version reference in each schema. The three-stage lookup cascade (normalization, fuzzy, AI fallback) runs identically for both shared reference data and inline tables, so there is no difference in resolution quality.

  • Key-value structure — key is the canonical code, value is the label the platform matches against.
  • Independent versioning — each update creates a new version; schemas pin to a specific version.
  • Cross-schema sharing — one reference dataset can be referenced by any number of schemas.
  • Three-stage lookup cascade — string normalization, token fuzzy matching, and Haiku LLM fallback for ambiguous values.

Frequently asked questions

What is reference data?+
Reference data are versioned lookup tables shared across the workspace, managed from the Reference Data tab on the Documents page (/sources/reference-data). They let you upload key-value reference data once and reuse it across multiple schemas for consistent code mapping.
How does shared reference data differ from schema reference tables?+
Shared reference data is workspace-level and shared across schemas with independent versioning. Schema reference tables are defined inline within a specific schema.
How does the lookup cascade work?+
The platform tries three stages in sequence. First, exact string normalization strips whitespace and normalizes casing to find a direct match. If no exact match is found, token-based fuzzy matching compares individual tokens against all reference values and scores similarity. If the best fuzzy match falls below the confidence threshold, a Haiku LLM call evaluates the ambiguous value in context against the top candidates and selects the most likely match. This three-stage approach balances speed and accuracy — most lookups resolve in the first two stages without any LLM cost.
What happens when I update a reference dataset?+
A new version is created. Existing schemas continue using their pinned version. You must explicitly update the version reference in each schema to use the new data, which protects production pipelines from unexpected changes.
How do I pin a schema to a specific reference data version?+
When configuring a reference table on a schema field, specify both the reference dataset ID and the version number. The schema uses that exact version for all lookups until you explicitly update it. This pin-based approach means you can test a new dataset version against a draft schema without affecting production schemas that are pinned to the previous version.