Schema-First Document Extraction Needs More Than Valid JSON

Turning a document into text and turning it into a usable database record are different jobs. OCR preserves words and layout, but an application still has to decide which number is an invoice total, which date is a deadline, and which rows belong to a line-item table. Dharma-AI's new Field Extractor targets that second step directly: it accepts a document image and returns structured JSON, either by discovering fields or by filling a supplied schema.

The release is a useful example of task specialization. The announced model has 2 billion parameters and builds on DharmaOCR-LITE v1.1 with a LoRA adapter. Its training set contains 43,384 examples spanning English multi-domain material, Brazilian documents in Portuguese, and a smaller multilingual component. Those facts explain the intended direction, but they do not make extraction a solved integration problem. A production pipeline still needs to define contracts, inspect failures, and distinguish syntactic success from factual correctness.

Choose discovery or a contract deliberately

The model exposes two modes. Open extraction asks it to identify the structured fields present on a page. Schema-directed extraction provides exact keys and types, with absent requested values represented as null. The first mode is useful for exploration, document inventory, and bootstrapping a candidate schema. The second is better when output feeds a known API, table, or workflow.

That distinction should remain visible in system design. Open output is an observation about what the model noticed; it should not silently become an application contract. A practical discovery workflow can collect proposed keys across a representative sample, normalize synonyms, identify optional and repeated fields, and then promote a reviewed subset into a versioned schema. This prevents field names inferred from one supplier or form template from spreading unpredictably through downstream systems.

Schema-directed output also needs semantic rules beyond primitive types. A value can be a valid string yet still be the wrong supplier name. A date can parse correctly but use an ambiguous day-month order. Currency totals may require a currency code, decimal precision, and reconciliation against subtotals and tax. JSON validity is therefore the first validation layer, not the acceptance criterion.

Build validation around business invariants

A robust extractor should sit between two explicit boundaries: an input-quality check and an output validator. Before inference, reject or route images that are truncated, unreadable, password-protected, or outside the supported page policy. After inference, validate the schema and apply domain invariants. An invoice might require a nonnegative total, a recognized currency, and a line-item sum consistent with the stated subtotal within an approved rounding tolerance. An identity workflow might require expiry to follow issue date while forbidding automatic acceptance when a critical field is absent.

Treat null as an abstention signal, not automatically as an empty database value. The correct action depends on the field: a missing optional purchase-order number may be acceptable, while a missing patient identifier should stop scheduling. Preserve the original document, extracted result, schema version, model version, and validation outcome so a reviewer can reconstruct why a record advanced or was blocked.

Cross-field checks often catch errors that per-field confidence cannot. If a document contains both a printed total and a table, arithmetic reconciliation offers independent evidence. Reference data can verify supplier identifiers or permitted exam types. These checks should flag discrepancies rather than overwrite model output, keeping inference and correction auditable.

Read the benchmark within its boundaries

The release reports a benchmark of 546 documents evaluated in three scenarios, producing 1,638 cases. Its score combines field-level matching, value similarity, and valid-JSON rate. Dharma-AI reports 0.834 for the specialized model, compared with 0.816 for the closest tested frontier configuration and 0.741 for the lowest of the four configurations. The company also estimates lower per-extraction compute cost on a single L40S than the compared APIs.

These results support further evaluation, not universal superiority. The benchmark reflects its document mix, schema conventions, matching rules, prompting, model settings, and self-hosting assumptions. GPU-hour cost is not a complete production cost: utilization, batching, orchestration, monitoring, redundancy, engineering time, and latency targets all matter. Conversely, API list price may omit discounts and the operational value of a managed service.

Teams should reproduce the comparison on their own document distribution. Separate results by document family, language, scan quality, page count, table complexity, and frequency of missing fields. Report exact-match accuracy for identifiers and amounts, not only edit-distance similarity. Measure false fills for absent fields because a plausible invented value can be more damaging than null.

Test the workflow, not a showcase page

A useful pilot starts with a frozen, access-controlled set of documents representing routine cases and costly edge cases. Define the target schema and human-reviewed ground truth before tuning prompts or thresholds. Compare the specialized model with the current pipeline under identical acceptance rules, then measure straight-through processing rate, critical-field error rate, review volume, latency, and total cost per accepted record.

Include perturbations that occur in practice: rotation, compression, faint scans, handwriting, stamps over text, multiple currencies, changed templates, and tables split across pages. Hold out entire layouts or issuers to test generalization rather than memorization of familiar forms. Privacy and retention requirements also apply to evaluation artifacts, especially for identity, financial, and medical documents.

The central opportunity is not merely replacing OCR with a model that emits JSON. It is moving the extraction contract closer to the model while surrounding it with deterministic controls. Specialized models may make that pipeline faster and cheaper, but dependable automation comes from versioned schemas, explicit abstention behavior, business-rule validation, and evidence collected on the documents that actually matter.