Skip to main content

Generated Schemas

Generated schemas are AI-created output definitions that Talonic builds automatically for each document type, containing every Core and Proven field (the maturity levels carried as tier 1 and 2 in API payloads, and as maturity: core / maturity: proven on the /v1/fields API) discovered in that type. They are versioned and diffable, giving you an always-up-to-date view of what the platform has learned about each document type. Generated schemas complement user templates, the schemas you define yourself for a specific downstream use case.

For each document type, Talonic generates a schema containing all Core and Proven fields with occurrences in that type. Generated schemas are versioned — new versions are created when the registry changes. You can diff any two versions to see what changed. The diff view highlights added fields, removed fields, type changes, and updated instructions so you can track how your field landscape evolves over time.

Behind the scenes, the generation engine scans the Field Registry for every field that has been promoted to Core or Proven within a given document type. It assembles these fields into a schema definition, assigns data types based on observed extraction patterns, and attaches the AI-synthesized master instruction for each field. The entire process is automatic — no manual curation is required.

Generated schemas are most useful as a starting point for understanding what Talonic has discovered about your documents. Review the generated schema for a document type to see which fields the system has identified, then use that knowledge to build a User Template containing only the fields you actually need. You can also use the diff view to monitor how your field landscape evolves over time as new documents are processed and new fields are promoted.

Field maturity determines which fields appear in generated schemas. Core fields are the most frequently occurring and reliably extracted data points — they appear in nearly every document of the type. Proven fields occur in a significant portion of documents and have been validated through repeated extraction. Candidate fields are too new or infrequent to be included in generated schemas, but they may be promoted as more documents are processed and their occurrence rate crosses the promotion threshold.

Reviewing a Generated Schema

  1. Open the generated schema for a document type to see every Core and Proven field the platform has identified in that type.
  2. Inspect each field's data type, maturity badge, and AI-synthesized master instruction to judge how well the platform understands the type.
  3. Open the diff view between two versions to see added fields, removed fields, type changes, and updated instructions.
  4. Use what you learn to build a focused User Template containing only the fields your downstream use case needs.

The diff between any two versions is computed by comparing field lists: added fields appear only in the newer version, removed fields only in the older, and modified fields show differences in data type, maturity level, or instruction text. Reviewing the diff after a large ingestion is the quickest way to see what the new documents taught the platform about a document type.

Generated schemas are read-only and cannot be used directly for job execution. To run a Job, create a User Template and map its fields to the registry. Generated schemas serve as a discovery tool to understand what the platform has found in your documents.

Maturity is the lens to read a generated schema through. Every field carries one of three levels, shown as its Status in the Field Registry: Core, Proven, or Candidate. On the API, the same level appears as maturity (core, proven, candidate) next to the legacy numeric tier field, and GET /v1/fields?maturity=core lists only Core fields. A Candidate field that keeps recurring is promoted automatically, and the next regeneration picks it up.

Generated schemas are a discovery view, not a production configuration. Production work runs on a user-authored Schema inside a Spec, which adds Data Policies, validation checkpoints, review gates, and assembly. A common path is to read the generated schema for a new document type, decide which of its Core and Proven fields your downstream system actually needs, and build those into the Schema of a Spec in the Spec composer.

Frequently asked questions

What are generated schemas?+
Generated schemas are AI-created output definitions for each document type, containing all Core and Proven fields found in that type. They are versioned and support diffing between versions. Each field in the schema includes data type information, the AI-synthesized master instruction, and occurrence statistics.
How are generated schemas updated?+
New versions are created automatically when the Field Registry changes (new fields promoted, clusters merged). You can diff any two versions to see what changed. The versioning system is append-only, so every previous version is preserved in the timeline for reference.
Can I run a Job using a generated schema?+
No. Generated schemas are read-only references. To run a job, create a User Template, select the fields you need, map them to the registry, and publish a version. Generated schemas are designed as a discovery tool — use them to understand what the platform has found, then build a focused template for your specific output needs.
How do I trigger schema regeneration for a specific document type?+
You do not need to. Regeneration runs automatically as part of the fixed resolve-promote-regenerate chain after each batch resolution run: the platform scans the Field Registry for Core and Proven fields, assembles the schema, and creates a new version whenever the field set changed. If nothing changed in the registry, no new version is produced.
Why does a field I can see in the Field Registry not appear in the generated schema?+
Generated schemas only include Core and Proven fields. A Candidate field is either new or still rare in that document type, so it stays out until it recurs often enough to be promoted. Open Registry in the sidebar to see each field's Status and how many documents it occurs in.

Related

User Templates
Field Registry
Document Types