Test Extraction

Before publishing a draft, run a test extraction to compare draft vs. live results side-by-side. Select a few documents, run the test, and see exactly how your changes affect output. This is the safest way to validate schema changes — you can iterate on field instructions, reference tables, and format constraints without affecting production jobs or published data.

After running a test, you will see a comparison grid highlighting cells that changed between the draft and live versions. Focus on fields you modified — new fields, updated instructions, or changed reference tables — to verify they produce the expected values. Cells that improved show in green, cells that regressed show in red, and unchanged cells are neutral. This workflow catches regressions before they reach production, so you can iterate on your schema with confidence.

Test extractions run through the same extraction engine as production jobs, so the results you see are representative of what a full job would produce. The test uses a simplified single-call extraction mode under the hood, which is faster but still applies all schema features including reference table lookups, format constraints, modifiers, and bypass strategies. This gives you a reliable preview without the cost and time of a full 4-phase Job.

For best results, select 3-5 representative documents that cover the variety in your corpus — include at least one "clean" document and one with unusual formatting or missing fields. This gives you confidence that your schema handles both typical and edge-case documents correctly. Run the test after every significant change to a field instruction, reference table, or format constraint.

A typical iteration workflow looks like this: add or modify a field in the Workshop draft, run a test extraction on your sample documents, review the comparison grid to check that the new field produces correct values, adjust the instruction if needed, re-test, and publish when satisfied. This tight feedback loop is the fastest way to refine extraction accuracy without impacting production jobs or consuming unnecessary credits.

Running a Test Extraction

1. Open your template's Workshop draft and make your field changes (instructions, reference tables, constraints).
2. Start a Test Extraction and select 3-5 representative documents, including at least one clean document and one edge case.
3. Review the comparison grid: green cells improved against the live version, red cells regressed, neutral cells are unchanged.
4. Adjust the instruction or configuration for any regressed field, re-run the test, and publish once the grid looks right.

Test extraction results include a cell-by-cell comparison between the draft and live schema outputs. Each cell is classified as improved (the draft produces a better value), regressed (the draft produces a worse value), or unchanged. The comparison accounts for both the extracted value and its confidence score, so you can detect cases where the value is correct but the confidence dropped — which might indicate a fragile extraction instruction that should be refined before publishing.

In the Spec composer, the same idea is called Preview. A Spec keeps a pinned sample set of up to five documents, and Preview runs the compiled pipeline over exactly that set, so you can see what a change to a field, a Data Policy, or a validation checkpoint does before you publish the Spec. Because the sample set is pinned, two previews made before and after a change are directly comparable. Choose the samples the same way as for a test extraction: one typical document, one with unusual formatting, and one with missing fields.

Test extraction and Preview answer slightly different questions. A test extraction compares a draft template against its live version on a quick, single-call extraction, which is the fastest way to tune an instruction or a reference table. Preview exercises the whole Spec rail, including resolution and validation, so it is the right check before a change reaches production Pipeline runs and the data products that reviewers work on in Products.