Does taking the arithmetic away from the model actually matter?

Same models. Same 61-SKU catalog. Same 30 labelled RFQs, five of them real scans. The only difference is the architecture: QuoteMind computes money in deterministic Python and re-checks it with a critic; the baseline is one agent asked to produce the whole quote.
Run: 11 Jul 2026 (today). These numbers are measured, not asserted - the harness is in src/quotemind/eval_/.

Metric	QuoteMind	Single agent
Task success	93%	40%
Price exactness	93%	40%
Caught its own problem	10%	0%
SKU top-1 accuracy	100%	98%
Line extraction F1	0.98	0.93
Errors	0	5
Cost per quote	$0.012637	$0.010989
Latency p50	31s	19s

Every case, one square each

Whether the final money was exactly right.

vi_text_001 vi_text_002 vi_text_003 vi_text_004 vi_text_005 vi_text_006 vi_text_007 vi_text_008 vi_text_009 vi_text_010 en_text_001 en_text_002 en_text_003 en_text_004 en_text_005 vi_xlsx_001 vi_xlsx_002 vi_xlsx_003 en_xlsx_001 en_xlsx_002 en_pdf_001 en_pdf_002 en_pdf_003 vi_scan_001 vi_scan_002 vi_scan_003 vi_scan_004 vi_scan_005 adv_001 adv_002

we got it right, the single agent did not both right we got it wrong

The single agent reads and matches almost as well - its SKU accuracy is within a point of ours. It gets the money wrong, and it never notices. That is what the critic is for, and it is why the model is never allowed to do the arithmetic.
← back to the review queue