Icelandic-text OCR · icelandic-ocr-benchmark ground truth

Icelandic OCR Leaderboard

How well OCR and vision-language models transcribe Icelandic documents — a broken-out scorecard, not a single grade. Ranked on micro-averaged character error rate, read as confidence intervals, not ranks.

The test material is a ground truth sample drawn from Sigurdur/icelandic-ocr-benchmark, pages of Icelandic-language documents. The scoring core is ported, unchanged, from finebooks/bhl-ocr-eval — same diplomatic/reading CER lanes, same bootstrap CIs, same fail-closed eligibility rule.

→ one line of the ground truth as each lane scores it — diplomatic keeps case and composed accents, reading case-folds and normalizes

Type

Size

click a row for per-language CER

Model ▾	Type i	Params i ▾	CER · reading i ▾	CER · dip i ▾	Content CER i ▾	Sparse CER i ▾	Recall i ▾	Over-x i ▾	Loop % i ▾

Sparse and blank pages are where models diverge most: strong content OCR can coexist with runaway hallucination on near-empty pages, and the aggregate CER shows it.

Size vs. accuracy

params (log) vs CER · upper-left is better

Reading the board. Where 95% bootstrap CIs overlap, differences are within noise — that overlap is the finding, not a defect. CIs resample volumes (source documents), so with few volumes they are wide by design.

A page can end three ways. Scored; looped (hit its token cap mid repetition loop — excluded from the row's aggregates and counted in Loop %); or errored (the producer failed — any error makes the row ineligible for ranking under fail-closed scoring).
CER · reading folds case and normalizes punctuation (reading ability); CER · dip keeps them (diplomatic fidelity). Recall is format-immune coverage, read co-equally with CER.
This board is early. It starts with one classical engine (Tesseract 5, Icelandic language pack) scored against a 100-page sample. Rows are added as more models are run through the same pinned scoring core — nothing here is a denylist, just what has been run so far.
Docs & data: methodology, scorer and drivers live in icelandic-ocr-eval, ported from finebooks/bhl-ocr-eval. Ground truth: Sigurdur/icelandic-ocr-benchmark.