Field scores
field_scoresrelease v0.7.1CC0 1.0
About
Measured accuracy per run and field, from the benchmarks (L4_TEST.md, sfra bench, the roll-call gold). Claims from these runs say reliability = measured.
- How far to trust it
- The release passed its automatic checks before it was published (what is checked). A hand check by the editor of qipu data is under way: 36 of 38 items checked so far were right, on votes, money, places and the map. A row says what the record states; the scanned page is the authority. Details
- Licence
- CC0 1.0: use it for anything, no permission needed. The scans and other sources keep their own rights.
- Version
- v0.7.1, published 21 October 2026. A published release is never changed; corrections come as a new release.
Technical details
- Table
- field_scores
- Rows
- 36
- Columns
- 7
- Parquet
- field_scores.parquet, < 0.1 MB
- Version
- v0.7.1, 21 October 2026
- Licence
- CC0 1.0. Sources keep their own rights.
Rows
Rows by run_id
| Value | Rows | Share | |
|---|---|---|---|
| sf-v21-haiku-1014 | 18 | 50.0% | |
| sfra-appraisals-haiku55-2026-10 | 7 | 19.4% | |
| van-v21-haiku-1014 | 4 | 11.1% | |
| rollcall-parser-v1 | 3 | 8.3% | |
| bos-corpus-2026-09 | 2 | 5.6% | |
| dl-sf-facts-v1.1-haiku55-2026-10 | 1 | 2.8% | |
| andrews-sonnet55-2026-10 | 1 | 2.8% |
Rows by field
| Value | Rows | Share | |
|---|---|---|---|
| places | 3 | 8.3% | |
| money | 3 | 8.3% | |
| outcomes | 2 | 5.6% | |
| vote names | 2 | 5.6% | |
| mover | 2 | 5.6% | |
| votes | 2 | 5.6% | |
| number+action+votes | 2 | 5.6% | |
| interest | 1 | 2.8% | |
| counts | 1 | 2.8% | |
| present_use | 1 | 2.8% | |
| owner | 1 | 2.8% | |
| names to persons | 1 | 2.8% |
Rows by metric
Method and accuracy
No accuracy measure for this table yet.
The release passed its automatic checks before it was published (what is checked). A hand check by the editor of qipu data is under way: 36 of 38 items checked so far were right, on votes, money, places and the map. A row says what the record states; the scanned page is the authority.
Columns 7
| Column | Type | Can be empty | Privacy | Public copy | Description |
|---|---|---|---|---|---|
| run_id | VARCHAR | no | none | as is | The run scored (see runs).Values rollcall-parser-v1, sf-v21-haiku-1014, sfra-appraisals-haiku55-2026-10, van-v21-haiku-1014, bos-corpus-2026-09, dl-sf-facts-v1.1-haiku55-2026-10, andrews-sonnet55-2026-10, sf-enrich-v11-haiku55-2026-10, citydir-haiku55-2026-10 |
| field | VARCHAR | yes | none | as is | The field or group of fields scored (vote names, money, mover, ...).Values votes, outcomes, places, dl facts, andrews owners, sfra appraisals, action, address, appraised_value, counts, dates, finds ... |
| metric | VARCHAR | yes | none | as is | How it was scored (exact, F1 (vs Opus key), recall, Spearman, ...).Values F1, F1 (vs Opus key), Spearman, all exact, exact, exact where the roll is in the text, matched, normalized exact (18 pages), recall |
| value | DOUBLE | yes | none | as is | The score, 0-1. |
| gold_id | VARCHAR | no | none | as is | The gold set (see runs.gold_id).Values decisions_bench Vancouver gold (30 pages, 48 decisions: Opus-written, 21 decisions hand-checked), decisions_bench/gold.jsonl (SF, 40 pages: an Opus-5.5-written key, no record of a hand check) + v2 Opus key, decisions_bench/gold.jsonl roll calls (Opus-written key), scripts/22_rollcall_gold.py, ia_records/sfra/bench (18 pages) |
| against | VARCHAR | yes | none | as is | What the run was compared with: opus_key (a key written by Claude Opus), opus_key (21 hand-checked), hand / human (a person checked the rows), model_scan_check (a model re-read each sampled claim against the page image; a person did not).Values hand, human, opus_key, opus_key (21 hand-checked), model_scan_check |
| n | INTEGER | yes | none | as is | Size of the gold set (pages, items or roll calls). |
Privacy: what kind of text the column holds. Public copy: what the published file does with it; the embargo covers records after 1951.
Preview
| run_id | field | metric | value | gold_id | against | n |
|---|---|---|---|---|---|---|
| sf-v21-haiku-1014 | votes (counts) | exact | 0.99 | decisions_bench/gold.jsonl (SF, 40 pages: an Opus-5.5-writ… | opus_key | 115 |
| sf-v21-haiku-1014 | money | F1 (vs Opus key) | 0.8 | decisions_bench/gold.jsonl (SF, 40 pages: an Opus-5.5-writ… | opus_key | 115 |
| sf-v21-haiku-1014 | topics | F1 | 0.9 | decisions_bench/gold.jsonl (SF, 40 pages: an Opus-5.5-writ… | opus_key | 115 |
| sf-v21-haiku-1014 | interest | Spearman | 0.78 | decisions_bench/gold.jsonl (SF, 40 pages: an Opus-5.5-writ… | opus_key | 115 |
| van-v21-haiku-1014 | vote names | exact | 0.98 | decisions_bench Vancouver gold (30 pages, 48 decisions: Op… | opus_key (21 hand-checked) | 48 |
| rollcall-parser-v1 | counts | exact where the roll is in the text | 0.87 | decisions_bench/gold.jsonl roll calls (Opus-written key), … | opus_key | 46 |
| dl-sf-facts-v…iku55-2026-10 | dl facts | confirmed on the linked page image / checked (Wilson 95% 0… | 0.9111 | eval/model_scan_check (300 public claims re-read against t… | model_scan_check | 45 |
| andrews-sonnet55-2026-10 | andrews owners | confirmed on the linked page image / checked (Wilson 95% 0… | 0.9655 | eval/model_scan_check (300 public claims re-read against t… | model_scan_check | 29 |
Eight rows sampled at build time. Hover a shortened value for all of it.
Download
| Format | File | Size |
|---|---|---|
| Parquet | field_scores.parquet ↗ | < 0.1 MB |
| SQLite (all tables) | qipu_sf.db ↗ | 6.44 GB |
SQL
SELECT *
FROM field_scores
LIMIT 100
Run in the SQL console (DuckDB in the browser, over the Parquet file).
Python
import duckdb
duckdb.sql("SELECT * FROM 'https://tiles.qipu.org/data/releases/v0.7.1/parquet/field_scores.parquet' LIMIT 5").show()