Changelog

Releases are numbered v0.x until the schema settles. Each release is a folder (releases/<version>/) and, when published, an archive.org item. Entity ids (places, persons, decisions, instruments) never disappear between releases: a retired or merged id gets a redirect. Claim ids changed twice before any correction pointed at them (v0.3: slot hashes) and are meant to stay from v0.3 on.

v0.7.1 (20/10/2026): what happened, with its source

A patch of v0.7 with the same sources and records; ids unchanged. It follows the usability audit of 19/10 (USABILITY_AUDIT.md) and Daniel's privacy change of 19/10 (post-1951 text kept, only private people's names withheld).

• One status for every decision reading, never blank. New columns on decisions: status, status_word, status_plain, happened (decided / not_decided / failed / unknown), carried_out (unknown everywhere: a decision to pay is not a payment), status_from, matter_status, matter_happened, matter_status_from (qipu_data/status.py). 718,093 of 718,093 readings have one: decided 373,018, not decided 284,198, failed 15,682, no status on record 45,195 (was: no outcome on 269,843).
• Outcomes recovered. 103,132 blank outcomes are filled from another reading of the same printed item, the v2 extraction or the printed roll call (outcome_from says which). Blank outcomes: 253,927 to 153,886.
• Summaries no longer state open matters as done. 25,784 one-line summaries rewritten from the status ("Payment of $13,250 to X proposed", "…; passed first reading only"): 10,681 by rules, 15,103 by Claude Haiku, every line checked (same numbers, no new names). plain keeps the original; plain_fix names the rule.
• Money. Every money statement (223,558) has outcome and qualifiers.money_status (approved_to_pay 123,777, proposed 59,500, listed_for_payment 17,438, claimed 9,979, refused 1,615, recommended 550 ...) and qualifiers.money_kind (claim_against_city 21,764, demand_for_payment 32,214, city_receives 1,980, city_payment 167,600). Nothing is marked paid: no payment evidence is in the data.
• Page links. The 43,507 sfbos.org minutes links with a known page now open that page (#page=); 60,223 statements of those decisions too. 1,876 Legistar links have no page to point to.
• Roll-call votes can be cited. 1,682,168 of 1,682,353 vote statements without page_url now have one.
• One cutoff. Records up to 1951 are open, 1952 on are under the 75-year rule, in every product (qipu_data/cutoff.py, pinned to 2026: nothing moves on 1 January). privacy_rules gains as_of_year and last_open_year; the 1950-51 privacy flags are cleared (10,386 decisions, 9,513 x_decisions, 86,183 claims).
• Post-1951 text, names withheld. The public layer keeps post-1951 text and withholds only private people's names (feat/redact-names; leak gate and canary in gates/).

v0.7-rc (19/10/2026, release candidate, not published): week 1 of the target model

This candidate includes everything in v0.6.2-rc. The schema notes are in SCHEMA_V07.md. Nothing here breaks existing ids or columns; the planned breaking changes are listed there and wait.

• Frozen person and organisation ids. ids/persons.csv and ids/orgs.csv are the minted-id registries. They are append-only and kept with the code. The roster step now reads a person's id from the registry instead of re-deriving it from the name, and mints new ids only for people not yet listed. A release ships the registries as entity_ids (294 persons and the Board). The graph slice's 1,499 organisation ids are registered as provisional. Checks I7 and I8 make sure every id used is registered and that no registered id ever disappears.
• Record time. The new withdrawn table lists every id that left a release, with a reason and what replaced it. It is carried forward from release to release. It starts with the 54,152 vote claims that disappeared between v0.6 and v0.6.1: 53,960 came from roll calls that turned out to be duplicates (merge), 183 were renumbered by the re-read (reread) and 9 were no longer read (error). 53,817 of them carry replaced_by. It also lists 500 instrument ids. I6 is now an error: a claim id may leave a release only with a redirect or a withdrawn row.
• Mentions. Every statement read from a page that links a person, an organisation or an unresolved name now carries the printed name it came from, in mention_id: 4,793,900 of 4,793,900. 611,173 mention rows were minted for the roll calls of the v0.6.1 re-read. The new table place_mentions (892,915 rows) does the same for place strings, and claims.place_mention_id covers all 1,102,513 statements read from a page and placed finer than the city. Checks M3, M4 and M5 enforce this.

v0.6.2-rc (19/10/2026, release candidate, not published): fixes from the architecture review

A patch of v0.6.1 with the same sources and records. Claim ids are unchanged. The fixes come from the review of 19/10 (ARCH_REVIEW_1019.md).

• Date bounds on every dated claim. In v0.6.1, the 1,682,357 vote claims of the roll-call re-read had t_from but empty t_lo / t_hi, so any date-range query on t_lo dropped a third of the votes. They are now filled, along with 246 other claims, by the same rule as the build (qipu_data.timebounds). The new error-level check T6 stops a build that misses any.
• Impossible dates. 59 claims carried a day that is not on the calendar: 1964-04-87 (49), 2001-11-79 (8), 1912-07-00 and 1964-11-80. They now keep the month (t_precision = month). The reading is kept in qualifiers.date_as_printed, and qualifiers.date_fix says why (day_not_on_calendar). The same dates are now empty on 12 decisions and 15 x_decisions rows; the year stays. The new error-level check T2D tests real calendar dates, where T2 only tested the shape.
• Checks. S9 (row keys unique) ran as "skipped" in v0.6.1 because of a Binder Error. It now runs and passes. An error-level check that cannot run now fails the build instead of skipping. I6 now counts every vanished claim id instead of stopping at 1,000: 54,152 vote claim ids of v0.6 are not in v0.6.1 and have no redirect. I6 becomes an error in v0.7.
• Two meanings of "reliability", documented. claims.reliability (measured / unmeasured) belongs to the run. claim_reliability.tier (verified / quoted / unverified) belongs to the claim. The DuckDB now has the claim_reliability table too. The rename to run_measured is breaking, so it waits for v0.7.
• Properties. 34 of 85 properties have a Wikidata property, marked exact or close (28 before). The new columns are range_kind (machine-readable) and allowed_qualifiers (the qualifier keys seen in the data).
• JSON-LD. Entity and source JSON, and the entity and landing pages, carry @context (https://data.qipu.org/context.jsonld), @id, @type and sameAs (Wikidata, archive.org).
• New stub table org_merges (empty): the merge log for organisation names, which the graph slice fills.
• Docs. STABLE_IDS_AND_VERSIONING.md: id minting, never-reuse, redirects, and what MAJOR / MINOR / PATCH promise. By that rule, v0.6.1 should have been v0.7. The datasheet has a new coverage and bias section. Version strings come from one VERSION file.

v0.6.1 (17/10/2026): data-quality fixes on v0.6

Same sources and the same records as v0.6; this release fixes how some of them were read. Use v0.6.1 instead of v0.6.

• Public names stay in scrubbed text. The privacy filter now keeps DataSF street names, neighbourhoods, landmarks and parks, named laws, events, curated titles and 93 hand-listed public figures. Before → after, in embargoed decision texts: "Cesar Chavez" hidden in 775 of 777 → 12 (a bare name with a private namesake), "Sunshine Ordinance" 465/477 → 0, "Martin Luther King" 110/170 → 2, "Forest Knolls" 51/75 → 0. The privacy gate stays at 0 known private names.
• Distinct decisions. New column decision_group, one id per printed item, on decisions and x_decisions: 718,093 rows → 539,192 printed items. On the audit's duplicate hand checks: precision 0.92, recall 0.70.
• Dates on v2 decisions (decisions.date_from): 0 → 317,293 of 344,936, from the Journal pages' dated headings. On 15,743 rows that already had a date the method agrees 94.4% of the time.
• First readings are no longer shown as passed: the outcome is now "passed first reading" on 41,030 decisions and 47,751 outcome claims.
• Roll calls are re-read from the page text (across page breaks, single "No:" dissents, lost commas). Distinct roll calls: 123,611 → 166,347 (47,512 added, 4,776 old duplicates marked duplicate_of). 1959 roll calls with a No: 54 → 98 (hand count about 102); Zirpoli's Noes in 1959: 30 → 44 (hand count 46-48).
• Supervisors' terms are rebuilt from their votes (roles, persons.first_year/last_year).
• Enactment numbers: a number printed only after citing words loses its decision link, and the 752 decisions checked against Legistar are corrected (decision_numbers.number_check is new). Wrong on the audit sample: 30% → 21%.
• Street links are read from the item's own wording (place_role bounds, precision segment with from/to, or address). Seriously wrong on the audit sample: 53/68 → 39/68; corner links 28/46 → 20/46.
• Places: Balboa Reservoir hub with its North and South Basins (220 decisions); Army St → Cesar Chavez St (1995); Embarcadero Freeway from 1959.

Known issues (still open, planned for v0.6.2): about one numbered decision in five before 1998 still carries a neighbour's number; segment-and-side places are marked precision segment without a block; the public-figure list is a hand list of 93.

v0.6 (16/10/2026): replaces v0.5

Counts from the release: 7,834,683 claims (4,192,188 votes), 718,093 decision rows (about 465,000 distinct decisions; 344,936 from the v2 read), 470,593 places, 182,701 archive.org pages, 6,377 sources. New since v0.5: US Census tracts 1980-2020, registered business locations 1968-2026 and building permits per block (W09); IIIF annotation pages on archive.org's canvases (W03).

Privacy: use v0.6 instead of v0.5. v0.5 showed some private names the filter missed: a possessive ("settlement of [a resident]'s lawsuit"), a title before one surname ("rezoned over Mrs. [an objector]'s objection"), business names that are an owner's own name, lawsuit parties, and names left next to a withheld one. v0.6 withholds all of these:

• a business or party name that reads as a person (the owner's full name plus a trade word, initials plus a surname, a surname-first listing, "Law Office Of ...") is withheld whole, in name fields and in the SQLite file; firms named by surnames only, and national brands named after a founder, stay shown;
• city employees named as a party in a plain "employee" role in records after 1951 (reimbursements, commendations, grievances); commissioners, department heads and officers of the Board named in office stay shown;
• lawsuit parties ("X v. City", "lawsuit by ..."), family trusts, names in lists that hold a withheld name, a name part left next to "[name withheld]", ALL-CAPS full names in OCR text, and a title before one ordinary capitalised word ("Mr. Gold");
• a hand review of every name the gate flagged in the draft (1,673 cells: 182 private names, 1,491 firms, streets, buildings, places and officials) is in contracts/privacy_reviewed.yaml; all 182 are withheld.

The gate fails on each form (P1 possessives, P1_titled_name, new P1_name_value for business-name columns).

Fix: page links. Every link opens the page the archive.org viewer shows:

• x_decisions.page_url on the bos-corpus and resweep rows was 1-2 pages early on 164,070 of 227,752 rows (in v0.5 too); scan_url and decisions.source_url were right. x_appraisals.page_url was 2 pages early on all rows.
• claims.source_url on the OCR-leaf sets (Andrews directories, Democracy's Library facts) was one page late; it now equals page_url. In v0.5, 149,324 of 150,446 Democracy's Library fact links were late.
• New gate check L3: every OCR-page link opens the viewer page the leaf map gives, with a sample of quotes on the opened page.
• A model re-check against the page images now finds the fact on the linked page for 41 of 45 Democracy's Library facts and 28 of 29 Andrews owners (v0.5 links: 9 of 45 and 0 of 34); see ACCURACY_v0.6.md.
• Claims moved to the previous page (decisions v2 quoting the page before) now have that page in pages.

Known issues (found by an error audit on 16/10; fixes are planned for v0.6.1, no data changed in v0.6):

• Counts: the 718,093 decision rows come from several extraction passes, and one item can have up to five rows; they stand for about 465,000 distinct decisions. Use the 344,936 rows of the decisions v2 read for one row per item.
• First readings are shown as passed (about 48,000 rows); the outcome "passed first reading" comes in v0.6.1.
• Roll calls can be incomplete where a vote runs across a page break (18.5% of roll calls).
• Place links for "between X and Y" often sit on the wrong block, and about half of the street links are wrong.
• Decisions v2 rows have no date field yet.
• About 26% of the numbered decisions before 1998 carry the number of another item.

Held for v0.6.1: the W07 enrichment (private names in its name fields, payee privacy class) and the W08 city directories 1910-45 (too few entries verified on their cited page). Their tables are not in v0.6.

Parcel coverage: 97.6% of today's parcels are linked; the rest are retired lots with no current parcel (accepted).

W09 later decades (US Census tracts 1980-2020, registered business locations, building permits per block): each claim is keyed by its table row (claim_support.row_key).

Fix: v2 claims on the right decision. When two items on a page both contained a decisions-v2 reading's opening words, the build picked one at random (DuckDB any_value). This happened for 22,093 of 90,564 matches (a roll call quoting the title of the ordinance before it). v0.6 picks deterministically: the item holding the whole 40-character key, then the earliest match, then the lower id. Of the 36,564 vote claims that move:

• 17,407 now point to an item that holds the whole key where the v0.5 item did not;
• none move the other way.

Because a claim id hashes its decision, 43,296 v0.5 claim ids are replaced (votes 23,035, places 7,548, parties 6,772, money 4,451, other 1,490). No correction pointed at any of them (the corrections table is empty).

Privacy: less over-withholding in text, with a leak check. In v0.5, a random sample of 100 withheld spans from claims, decisions and x_site_rows text held only 9 private people. The other 91 were places (32), organisations (27), officials named in their role (11), days and events (8), public figures (6), titles of works (3) and OCR noise (4).

v0.6 keeps these in scrubbed text, under the 14/10 rules (organisations win; officials in their role are shown):

• titles of works in quotes or italics;
• known place names from our own places table;
• capitalised runs that end in a feature word ("Coit Tower", "Veterans Day", "Food Security Task Force");
• people named in an appointment ("appointed to", "reappointment of", "Rep.").

Guards that still withhold:

• anything holding a known private full name or surname;
• a first name followed by a census surname;
• a day, week or award named after someone;
• a name with a quoted nickname.

The leak check was rerun until clean. Each round drew 200 spans that v0.5 withheld and v0.6 shows, read every one, and turned each hit into a guard rule plus a test before re-drawing:

• round 1: 3 hits;
• round 2: 4 hits;
• round 3: 2 hits;
• round 4: 1 hit;
• round 5: 1 hit;
• round 6: 0 private people in 200; all were appointees, places, organisations and events.

Re-scrubbing v0.5 text with the new rules shows about 22% of the spans v0.5 withheld.

A fresh sample of 100 spans that stay withheld: private people 24, public figures 6, officials in role 8, places 25, organisations and programmes 30, days and events 2, titles 1, OCR noise 4. Over-withholding is the safe error and remains the larger one. Samples: build/plat4/leakcheck_*.json.

v0.5 (15/10/2026): San Francisco only, every page linked, door-level claims, reliability tiers

Published 15/10/2026 05:07 (https://tiles.qipu.org/data/releases/v0.5/, manifest sha256 dba8d4b6...). Counts from the public Parquet: 7,500,934 claims (4,158,385 votes), 694,233 decision rows, 469,812 places, 181,214 archive.org pages, 6,296 sources.

What changed

• San Francisco only. Vancouver moved to its own release folder (release/van); ids are checked per city (I5).
• Every page links to the page the viewer shows (page_url on every table with a page id): the OCR leaf goes through the audited leaf map (scandata, 274 volumes) to archive.org's /page/n<k>. Pages the viewer hides (covers, colour cards; 1,355 rows) have no link. Known issue: pages.iiif_url is one page early on about a third of pages; use page_url (fixed in v0.6).
• Block placement rule: a decision keeps a block only if a street named in its own text touches it, or a named site is within 400 m; 18,741 placements (7.7%) moved to a street, a site or the city.
• Door-level claims (about 840k): assessor roll (built, use, units, stories), notable DBI permits, Board permits 1906-49, the 1869 land survey and first-built estimates. Readings from the 1938/1948 aerials stay local.
• New tables: x_dl_facts (Democracy's Library SF read in full: 150,446 facts, each quote checked against the page), x_realprop_dir (Andrews Real Property Directory 1943/1949: 276,599 address lines with owners, shown under the 75-year rule), claim_reliability (one row per claim: verified / quoted / unverified, with the signals behind it; 2,582,112 quoted, 889 verified), cpi_us / cpi_ca (amounts in 2026 dollars).
• Privacy: names follow the 75-year rule field by field (officials named in their role and organisations are shown); the gate's reviewed list of places, firms and public figures named after people (contracts/privacy_reviewed.yaml) is part of the release.
• Checks: spec checks 0 fail; the contract and privacy gate passes (0 failures, 52 warnings); the entity sweep finds no private name in 673,598 record documents. A 20-claim scan spot-check: 18 quotes on the linked page; the 2 misses quote the page before (decisions v2 reads the previous page as context; v0.6 re-points them).

v0.4.1 (14/10/2026): privacy fix, republished

• Name fields under the embargo are now withheld. v0.4 showed payee names in claims.qualifiers (Parquet), in x_decisions.money[].payee (Parquet) and in decisions_v2.money (SQLite). New rule for every name field (payee, payer, party, owner, buyer, seller, counterparty ...): while the record is under the embargo (SF 75 years, Vancouver 92), organisations and public bodies are shown; person-like or unknown names read "[name withheld]". Values withheld: Parquet claims 3,097, x_decisions 3,380; SQLite decisions_v2 2,702. A new gate (42_name_fields.py, P-fields) checks it and fails the build if any remain.
• The text filter now also catches names inside JSON text (qualifiers purposes) and next to street names ("1445 Eddy [name withheld]"); 2,517 more patterns in Parquet and 1,594 in SQLite were withheld. P-public check: 0 left.
• name_mentions withdrawn from the public copy. x_mentions, ocr_files, place_relations and block_points are now documented.
• Same data as v0.4 otherwise. v0.4 files were removed from the public bucket.

v0.4 (14/10/2026, early morning): Vancouver, provenance, exports, privacy everywhere

What changed

• Vancouver (qipu:van/...): the decisions v2 run (claude-haiku-5-5, $23.95, Wayback council minutes 1995-2026 and IA scans of the 1970s): 76,652 decisions read, 14,231 items split across two pages merged with the next page's heading (their second halves redirect to the first), 62,421 decisions and 252,551 claims (outcome, concerns_place, party, moved, money, voted, refers_to, rezoned). Vancouver minutes mostly print only who opposed ("CARRIED (Ald. X opposed)"), so its votes are 12,775 no, 4,920 absent, 448 yes. Councillors are public officials by role; their names stay names as printed until a roster is loaded. Canada's 92-year rule applies to everything else (all of it is embargoed).
• Privacy, every column (coordinator spot-check): a second pass (41_public_scrub.py) filters every free-text column of every public table with a year (vote rows, roll-call quotes, decision titles, x_ tables, quotes) and the build stops if a person-name pattern is left in an embargoed row. The name finder now needs a title, an initial or a Census first name, so phrases like "Finally Passed" are no longer hidden. On the published v0.3 copy: 36,393 name patterns in the public SQLite and 90,024 in the public Parquet before; 0 and 0 after (republished 14/10 ~05:00).
• OCR provenance (review M5): ocr_files (9,416 OCR files of the cited items with md5, sha1 and mtime) and pages.ocr_file_sha1 on 115,800 of 118,891 pages.
• Exports (review M10, m8): Linked Places Format (2,148 places for the World Historical Gazetteer), 1,000 W3C Web Annotations on IIIF canvases (pre-1940 decisions), and a reviewed-but-not-uploaded QuickStatements batch adding "member of the San Francisco Board of Supervisors" (Q116005093) to 115 supervisor terms with a Journal-page reference.
• Reach: REACH.html (Democracy's Library by place: 91% placeable from metadata, but 92% of USGovernmentDocuments are court dockets placed only by district; of the other ~0.9M items ~15% name a city or county); MINUTES_INVENTORY.html (10,612 council and board minutes items on archive.org, ~1.68M pages, ~$2,100 to read with decisions v2); new-city plans for Vancouver and Oakland (95_new_city.py).
• Benchmark kit (bench/): golds, prompt, a dependency-free scorer, honest labels.
• Hand-check sheets (build/handcheck/, local): 100 held-out decisions and 300 vote names, stratified by decade, for a person to check against the scans.
• MCP server: places_at and showcase tools; citations documented.

Checks: 41 pass, 5 warn, 1 skipped (S9: the runner skips it when the previous release is attached), 0 fail; the public privacy check passes.

Still weak

• SF decisions v2 is still the 1,747 decisions of the run's first parquet (3 of 52 batches collected; the runner writes its parquet at the end).
• Vancouver has no places beyond the city (no address register loaded), and no council roster.
• The name finder is a pattern with a first-name list: it still misses a lone surname ("Smith gets a permit") and over-hides some phrases that start with a first name.
• The public explorer's privacy filter runs a Python function over millions of cells: the export takes ~30 minutes.

v0.3 (14/10/2026, night): the spec v1.0 fixes, the place graph, the SF proof

What changed

• Adopted the architect's spec v1.0 P0 list (spec/HANDOFF.md 1-6): content-hash claim ids (no run id, no resolved place) with claim_support (one row per supporting extraction row); decisions v2 readings attach to their v1 decision (1,030 v0.2 v2 ids redirect to v1 ids in id_redirects); search scores moved out of confidence into qualifiers.score; place precision on the closed list plus place_role (at / near / along / about); licence on claims is a short code, the prose lives in media.rights_note; vocabulary tables (properties, place_types, cities, privacy_rules, events); places.type_id; the spec checks run in every build and stop the publish on an error.
• Also from the spec: EDTF time with t_lo / t_hi; Sanborn lines are building observations (not built); the 1906 fire is affected_by the event; nearby photos and camps are place_role = near with distance_m; rent carries its period where the row prints one; private names carry privacy_class and public_after_year (sf 75 years, van 92); runs.gold_id, field_scores from the benchmarks, claims.reliability measured / unmeasured; media get media_kind, created_edtf (from the item, not the beat) and use_policy.
• The place graph: place_geometries (459,606 dated geometries: DataSF parcels and assessor blocks as polygons, EAS address points, street centerlines per street, the 41 neighbourhoods, the 1906 burn zone, the showcase features) and place_relations (1.28 M: within / crosses / overlaps from geometry, on_street / within parcel / within zoning district declared), computed every build; no fixed hierarchy. places_at(lon, lat, date) returns every place containing a point at a date, most specific first (680 Haight today: address, parcel 0847-019, block 0847, Haight Ashbury; Market and 4th on 20 April 1906: block, parcels, South of Market, the burn zone).
• Today's zoning districts as places (from the DataSF parcel attribute), parcels declared within them.
• The SF proof: six places (Islais Creek and its watershed, Hunters Point Shipyard, the Embarcadero Freeway, Haight Street, 1805 Geary / the Fillmore with the Western Addition A-2 area, Mission Bay) with what the Archive holds about each across collections, found by metadata only (archive.org search, Open Library, Wayback CDX; no model): 3,382 hits become sources and concerns_place / depicted_in / recorded_at claims with a confidence by where the name matched and a rights note by collection. A timeline page per place on the explorer, and "click anywhere" (a point to its block, neighbourhood and features, with their records).
• Explorer: a DuckDB-WASM SQL console over the Parquet release (sql.html).

• From the independent review (spec/REVIEW_1.md): privacy in every public product: the public SQLite, the public Parquet and the entity JSON withhold private people's names for 75 years in SF (92 in Vancouver), names whose class is unknown included; person-like names inside model-written summaries of embargoed records become "[name withheld]"; owners and parties show their role ("an owner (name withheld until 2042)"); raw OCR quotes of embargoed records are not published. The full tables stay local. Model-written lines are labelled (summary_by, a note on every result) and the site gives a takedown route. Page ids now use IA's leaf number (<identifier>:<leafNum>, the jp2 / IIIF number), with the BookReader index kept as display_n for links. Claim ids are 128 bits and hash the slot (subject, property, object, page, role), not the reading. Benchmark labels say what was measured (agreement with Opus-written keys; only the appraisal values and 21 Vancouver decisions were hand-checked). Vote confidence is the measured 0.87 (0.6 where the printed counts disagree). Sentences paraphrased from Wikipedia carry CC BY-SA 4.0; OpenSFHistory photos are link-only in our products.
• The earlier staging copies (v0.1, v0.2) were taken offline on 14/10 at 02:50 because they showed private names; they are not republished.

Checks (spec/tests/run_checks.py on v0.3, previous release v0.2): 40 pass, 5 warn (T3 219 claims dated outside their volume's catalogue years; T4 7 claims before their place existed; V4 4 roll calls where one printed name stayed unmatched; V6 903 rents without a printed period; I6 claim ids of v0.2 changed with the move to content hashes), 0 fail; 2 skipped (S9: the runner skips it when a previous release is attached, it passes when run alone; R11: no corrections yet).

Still weak

• Feature geometry is a circle around the Wikidata point (marked approximate); real boundaries (sewersheds, the shipyard, the freeway alignment, the made land) are the next layers.
• The metadata search catches namesakes: "The Fillmore" brings concerts at other Fillmores, "Hunters Point" brings the wider neighbourhood; those have lower confidence but are not removed.
• Democracy's Library hits are few for these places by metadata alone (Hunters Point 22, Mission Bay 3, Islais Creek 0): DL's IGS items rarely name them in title or subject; their pages do.
• Claim ids changed from v0.1/v0.2 (content hashes from now on): nothing pointed at them yet (no corrections).

v0.2 (14/10/2026, night)

What changed

• Decisions v2 (claude-haiku-5-5, schema v2.1, one call per page) ingested as its own run next to v1, never over it: 1,747 SF decisions so far (the run is still landing; Vancouver's first batches had not landed). x_decisions now holds v1 and v2 rows (v2 columns: names in the roll call, parties with roles, money, places with cross streets, zoning, related numbers, dates, interest).
• New claims from v2: 11,789 votes (100% resolved to supervisors by the roll-call matcher), 790 parties, 570 amounts, 763 places, 307 movers, 157 references, 24 rezonings. 1,030 v2 decisions are recognised as the same decision as a v1 item (same_as); 114 v1 decisions took their outcome from that v2 twin.
• Vancouver has its own namespace (qipu:van/...); its votes stay names as printed until a council roster is loaded.
• Media registry: 1,405 photos, films, recordings, clips and documents the map shows (timelines, covers, hood stories, block items, films, clips), each with credit, licence, holder and a rights note (OpenSFHistory/WNP: ask before public use; Live Music Archive: non-commercial), and 5,088 depicted_in / recorded_at claims tying them to their place and year.
• The Redevelopment Agency appraisals (Western Addition A-2): 3,189 rows as x_appraisals, 14,823 claims (appraised_at, assessed_at, rent, sold_for, building, use, units, owned_by).
• Democracy's Library items read for their places (x_dl_tags, 1,131 items): 1,289 concerns_place claims.
• Products: the map's block decision files regenerated from the claims, compared with the map's files (build/products/v0.2/DIFF.md; the map is not touched).

Why: the map's whole content becomes traceable (every photo and film has a holder and rights), and the richer v2 extraction joins the same structure without replacing v1.

Still weak

• v2 coverage is what landed by 02:07 (2 of 52 SF batches); rebuild to take more.
• The v2 runner stamps CAD on SF amounts and links pages by OCR leaf; both are corrected in the release, not in the run files.
• v2 money has no direction (paid or received), so it is money, not paid/received.
• The map's two layers disagree with each other on the one-line summary for 3,989 of 37,758 block rows (the block index line vs the curated site line); the dataset keeps the curated one.
• DuckDB is capped at 2 GB, but the build still peaks at about 3.5 GB of memory (Python + DuckDB).

v0.1 (14/10/2026)

First release, built only from data qipu already had (the map's files and the decision extraction runs).

• Registry: sources (3,150 cited sources; Democracy's Library membership by collection), pages (97,874 archive.org pages with scan and IIIF links), runs.
• Claims (2,236,656): Board decisions from the corpus, the map's block index, the re-sweep (claude-haiku-5-5) and the site's curated rows; block items (assessor year built, Sanborn lines, 1906 fire, photos, stories); street and neighbourhood timelines; doors; dateline; archive items linked to places; 1,482,785 votes.
• Roll calls: 123,611 printed roll calls parsed from the Journals, 1906-1999; 99.4% of 1,491,968 names matched to 294 supervisors (official roster, Wikidata ids).
• Places: 387,193 with stable ids (DataSF EAS addresses, parcels, blocks, streets, neighbourhoods, landmarks). Instruments: 72,621 printed numbers.
• Explorer at data.qipu.org with ten saved questions; resolvable JSON for every place, person and instrument; read-only MCP server.
• Known limits: see README.