Applied Machine Learning · Multi-vertical

Extraction Bake-off

Four invoice extractors. One hundred synthetic documents across three difficulty tiers. Field-level scoring against ground truth. The data tells you when to build, when to buy, and when to pay for both.

Best accuracy
99.1%
Claude vision
Docs evaluated
100
4 methods each = 400
Total spend
$4.07
Across all methods
Regex on hard
55%
The build/buy crossover

Tab 1 · Single Document

Pick a doc. Watch four methods compete.

Each column is one extractor. Green ✓ means the extracted field matches ground truth (using field-specific scoring rules — exact for IDs, fuzzy for names, numeric tolerance for money, F1 for line items). The cached view loads instantly; "Run live" hits the real Lambda with all four methods in parallel.

Filter:
Document · INV-0001
INV-0001
cleanvendor: acme_industrial

Mode

Pre-computed (instant, $0)
These results were extracted earlier and cached. The "Run live" button hits the real Lambda with all 4 methods in parallel. Costs ~$0.04, takes 5-15s, rate-limited to 5 per IP per hour.
Regex
0.0s·$0.0000
Invoice #
✓
ACM-21395
Date
✓
2026-03-10
Vendor name
✓
Acme Industrial Supply Co.
Vendor address
✓
1247 Manufacturing Blvd, Cleveland, OH…
Bill-to
✓
Rodriguez, Figueroa and Sanchez
Total
✓
199.75
Tax
✓
0.00
Line items
✓
3 items
Textract
2.5s·$0.0100
Invoice #
✓
ACM-21395
Date
✓
2026-03-10
Vendor name
✓
Acme Industrial Supply Co.
Vendor address
✓
1247 Manufacturing Blvd, Cleveland, OH…
Bill-to
✓
Rodriguez, Figueroa and Sanchez
Total
✓
199.75
Tax
✓
0.00
Line items
✓
3 items
Claude
5.4s·$0.0086
Invoice #
✓
ACM-21395
Date
✓
2026-03-10
Vendor name
✓
Acme Industrial Supply Co.
Vendor address
✓
1247 Manufacturing Blvd, Cleveland, OH…
Bill-to
✓
Rodriguez, Figueroa and Sanchez
Total
✓
199.75
Tax
✓
0.00
Line items
✓
3 items
Hybrid
6.0s·$0.0209
Invoice #
✓
ACM-21395
Date
✓
2026-03-10
Vendor name
✓
Acme Industrial Supply Co.
Vendor address
✓
1247 Manufacturing Blvd, Cleveland, OH…
Bill-to
✓
Rodriguez, Figueroa and Sanchez
Total
✓
199.75
Tax
✓
0.00
Line items
✓
3 items
Ground truth
TRUTH
Invoice #
ACM-21395
Date
2026-03-10
Vendor name
Acme Industrial Supply Co.
Vendor address
1247 Manufacturing Blvd, Cleveland, OH…
Bill-to
Rodriguez, Figueroa and Sanchez
Total
199.75
Tax
0.00
Line items
3 items