Every figure, with the command that produced it.
Harald was measured on the public CUAD set: 510 commercial contracts with the clauses marked by lawyers. No firm’s own contracts were measured, and nothing here was checked by anyone outside.
Ten questions on 510 contracts
One live run on 11 November 2026: 5,100 cells, one seed, one model, 4.25 USD of model use.
2,661
Verified quotes
Each one found in its contract by code. No cell failed the audit of the run.
95.5%
Recall
Of the clauses CUAD marks, the share Harald found.
78.6%
Precision
Of Harald’s answers, the share CUAD also marks. F1 on verified cells: 86.3%.
LIVE=1 pnpm exec tsx packages/eval/src/cli.ts --set full --client live --seed 1 --columns packages/eval/fixtures/columns-all10.json --cap 5
Limit. This run counted a checked answer of “no” as a clause found, which pulled precision down to between 50% and 58% on three questions: termination for convenience, exclusivity and non-compete. The count was corrected, and those three questions were run again on 12 November 2026 (1,530 cells): precision 82.2%, 77.7% and 68.0%. Non-compete is still the weakest. The ten-question figures above have not been measured again under the corrected count.
LIVE=1 pnpm exec tsx packages/eval/src/cli.ts --set full --client live --seed 1 --columns packages/eval/fixtures/columns-weak3.json
The other measurements
Smaller, and stated with the same care.
- Three questions, 1,530 cells
- Live run on 10 November 2026: F1 on verified cells 91.2%, no failed cell. Change of control alone has precision 58.5%.
LIVE=1 pnpm exec tsx packages/eval/src/cli.ts --set full --client anthropic --seed 1 --out <dir>- The test suite
- 1,413 tests in 122 files passed on 12 November 2026, with the browser tests and the mock evaluation.
env -u LIVE pnpm check- Load
- 20,000 cells in 108.9 seconds, none failed, 85 throttle answers retried.
- Limit. Mock model, the 510 contracts repeated, no queue and no database: it measures the pipeline, not a real run.
pnpm loadtest
What these figures do not show
Read them as a measurement of one run on public contracts, not as a promise about yours.
- One seed and one model: the run was not repeated to give a range
- CUAD contracts are American and in English
- Italian was measured on an invented set only
- German, French and Spanish were run with a mock model only
- Recognition of scans was measured on generated pages, not on real scans
- The check compares words, not meaning
The check itself is on the first page.
Change one word of a quote and see what the rule does with it.