Skip to main content

Phase 4: Re-read

Context-aware gap filling and deterministic transforms. Phase 4 serves two purposes: for each empty cell or low-confidence value, AI re-reads the original document with the field instruction and full grid context to find values missed in earlier phases. It also applies deterministic transforms to all cell values — ISO code normalization, date format standardization, unit conversion — and evaluates format constraints (regex patterns) with configurable mismatch behaviors. The modifier chain runs in a fixed order: format transforms first, then alias mapping, then max_length truncation.

Because Phase 4 has access to the full grid context — all values already resolved in earlier phases — it can use surrounding data as clues. For example, if a contract start date was resolved in Phase 1 but the end date is still empty, Phase 4 re-reads the document knowing the start date, which helps the AI locate the corresponding end date more accurately.

Phase 4 also applies deterministic transforms to all cell values: ISO code normalization, date format standardization, and unit conversion. Format constraints (regex patterns defined on schema fields) are evaluated at this stage. If a value fails its format constraint, the configured mismatch behavior kicks in — the cell is either cleared, flagged with an amber dot, or replaced with a constant. Original values are always preserved in the original_extractions table for audit purposes.

Expect Phase 4 to fill 5-15% of remaining empty cells, depending on document complexity and schema coverage. The phase is most effective for fields that require cross-referencing multiple sections of a document or interpreting values in the context of other extracted data. It is less effective for fields that are genuinely absent from the source document — those will remain empty with an unresolved provenance type.

If a job fails mid-run, you do not have to start over. Open the run detail page and click Resume: the Job detects the last completed phase and resumes from the next one, and the prior grid state is preserved, so every cell filled by earlier phases carries over into the resumed run. This makes long runs safe to retry after a transient failure such as a timeout or a temporary model outage.

Phase 4 is where the modifier chain and format constraint evaluation happen. The modifier chain runs in a strict order — format transforms first (converting dates and numbers to your target format), then alias mapping (replacing values using a lookup table), and finally max_length truncation. After all modifiers have been applied, format constraints evaluate the final transformed value against the regex pattern. If the value fails, the configured mismatch behavior kicks in. This ordering is important because it means constraints validate the output your downstream system will actually receive, not the raw AI extraction. Original values are always preserved in the original_extractions table, giving you a complete audit trail from raw extraction through every transformation step.

Phase 4 respects the confidence gate: it can only fill empty cells or upgrade cells below the confidence threshold. High-confidence values from Phase 1 are permanently protected. Original values are always preserved in the original_extractions table for audit, regardless of whether format constraints clear, flag, or replace them.

Phase 4 is the last chance a Job has to fill a cell, so what remains empty afterwards is informative. A field that stays unresolved across many documents is usually genuinely absent from the source, ambiguously described in the schema, or a candidate for a bypass strategy. Read the Phase 4 outcome together with the strategy yield before you spend effort on corrections. Improving the instruction or adding a reference table entry usually fixes a whole column, while a correction fixes one cell in one run.

Frequently asked questions

What does Phase 4 Re-read do?+
Phase 4 performs context-aware gap filling by re-reading the original document with field instructions and full grid context for each empty or low-confidence cell. Because it has access to all values resolved in earlier phases, it can use surrounding data as clues — for example, using a resolved start date to locate the corresponding end date more accurately.
Can Phase 4 overwrite high-confidence values?+
No. Phase 4 respects the confidence gate — it can only fill empty cells or upgrade cells below the confidence threshold. High-confidence values from earlier phases are permanently protected. This is the single most important Job rule, ensuring that reliable lookup results are never replaced by lower-confidence AI extractions.
What else happens in Phase 4 besides gap filling?+
Phase 4 also applies deterministic transforms (ISO codes, dates, units), evaluates format constraints (regex validation), and runs the modifier chain in a fixed order: format transforms first, then alias mapping, then max_length truncation. Constraint evaluation happens after all modifiers. Original values are always preserved in the original_extractions table for audit, regardless of whether constraints clear, flag, or replace them.
Can I resume a job that failed during Phase 4?+
Yes. Open the run detail page and click Resume. The pipeline detects which phase completed last and resumes from the next phase, preserving the prior grid state, so you never lose work from earlier phases when resuming a failed job.
Does Phase 4 call the Planner again?+
No. Phase 4 is a targeted re-read, not a second planning pass. It sends the field instruction and the current grid to the model for each remaining empty or low-confidence cell, then applies the modifier chain and format constraints deterministically. The Planner runs once, in Phase 2.
Can I resume a Job that failed in Phase 4?+
Yes. Open the run detail page and click Resume. The Job continues from the phase after the last one that completed, and every cell filled earlier carries over, so a timeout late in the run does not cost you the work already done.

Related

Phase 3: Validation
Confidence & Provenance
Corrections