engineering/arborist
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
arborist/bench/results
History
russell@unturf.com 3aae8119a6
#000065 step 1: routing helper + meta field + pre-migration snapshot 
Three pieces, all read-only or additive — no shard mutation, no
schema-version bump:

1. shard_for_document(document_root, M) in arborist/document.py.
   Pure function: int(document_root[:8], 16) % M. 22 tests cover
   determinism, range-bounds, near-uniform distribution (±5pp at
   N=20k), and seven lock-in fixtures so peers will disagree
   loudly if anyone changes the formula.

2. corpus_shard_count meta field + get/set helpers in store.py.
   Lives in the existing key/value meta table; SCHEMA_VERSION
   stays at v9.8.0 (the DDL doesn't change and source_root is
   layout-independent, so cache records survive a reshard).
   Legacy shards (without the field) return None; reshard tool
   populates it on every target shard at migration time.

3. Pre-migration snapshot captured to
   bench/results/pre-migration-snapshot.json:
     docs        3,468,392  (3,468,226 globally unique)
     chunks      6,235,764
     edges      90,593,537
     audit       3,468,403
   This is the reference set post-reshard row counts must match.

4. Audit-event extraction script writes all 3.47M events from
   all 4 shards to /tmp/audit-events.ndjson (2.0 GB) for the
   Option-A canonical-chain consolidation step. Verifies chain
   integrity on extract — all 4 source chains report 0 breaks.

5. Fixed a wrong chunk count in docs/corpus-history.md
   (had ~3.54M/shard; actual is ~1.56M/shard) and added the
   edge-count column (~22.6M/shard, 90.6M total). 6.24M chunks
   total, not 14.12M.

Tests: 29 new pass (22 routing + 7 meta). No existing tests
touched.
2026-05-26 13:13:44 -04:00
..
5f-threshold-calibration-2026-05-11.md #000047 — close: delta_aggregator knob on ForkScore (Option D) 2026-05-11 08:27:38 -04:00
baseline-falsification-hard.json #000048 step 2.1 — verify_quotes entity salient-token-disagreement gate 2026-05-11 13:57:45 -04:00
citation-aliases-batch-2026-05-10.md citation-aliases batch: 18 -> 35 / 92 records resolve (38%) 2026-05-10 06:58:27 -04:00
claim-pack-retrieval-lift-2026-05-09.md bench/results: claim-pack retrieval lift measurement (2026-05-09) 2026-05-09 13:38:04 -04:00
full-bench-qa-2026-05-10.md bench/full-bench-qa: no regression after substrate+Phase 3 sprint (675 cells) 2026-05-10 13:42:12 -04:00
full-warrant-resolution-2026-05-10.md docs: refresh stale alias counts (residual 40/54 → 74) 2026-05-10 12:27:23 -04:00
grinstead-snell-and-cascade-tuning-2026-05-10.md Grinstead-Snell ingest + cascade tuning batch: 68 -> 77/92 (84%) 2026-05-10 08:03:11 -04:00
hilbert-pillar-iv-fully-resolved-2026-05-09.md ticket #000042: 4 geometry aliases resolve all 18 Hilbert pillar IV records 2026-05-09 20:18:58 -04:00
jaggedness_2026-05-21T01-56-17Z.json feat(#000060): bench/jaggedness.py — deterministic retrieval jaggedness instrument 2026-05-21 08:38:07 -04:00
lambda-and-pm-additions-2026-05-10.md PLFA + SF-LF + PM Vol 1 ingest: 62 -> 68/92 (74%) — pillar IX nearly closed 2026-05-10 07:45:40 -04:00
medium-confidence-aliases-and-peano-2026-05-10.md medium-conf aliases + Peano ingest + title backfill: 35 -> 54/92 (59%) 2026-05-10 07:14:44 -04:00
nli-shadow-grid-n1-fixtures-vs-realstrict.json #000049 §7 #23: grid sweep flips the verdict — the lexical-candidate NLI veto PASSES the gate 2026-05-12 18:12:36 -04:00
nli-shadow-grid-n1-synth-vs-realstrict.json #000049 §7 #23: grid sweep flips the verdict — the lexical-candidate NLI veto PASSES the gate 2026-05-12 18:12:36 -04:00
nli-shadow-grid-n1-synth-vs-synth.json #000049 §7 #23: grid sweep flips the verdict — the lexical-candidate NLI veto PASSES the gate 2026-05-12 18:12:36 -04:00
nli-shadow-grid-n3-synth-vs-realstrict.json #000049 §7 #24: n=3 confirmation — §7 #23 was a small-FP-sample artifact; the no-stone-unturned mega-grid found the config that survives 2026-05-12 18:38:43 -04:00
nli-shadow-grid-n5-synth-vs-realstrict.json #000049 §7 #25: n=5 confirmation (444 STRICT cells) — verdict settles; the large models are the robust ones, MiniLM-cost-pick overturned 2026-05-12 19:42:55 -04:00
nli-shadow-grid-pooled808-synth-vs-realstrict.json #000049 §7 #26: mining for real recombinations + the pooled-808-STRICT check — verdict settles on margin@0.995 2026-05-12 21:11:35 -04:00
nli-shadow-grid-realctx25-vs-pooled808.json #000049 §7 #27: recall-side realism check — synthetic 28/28 was a 4x over-estimate; bart-large-mnli/k=12/max/θc=0.999 is the new fp=0 leader on real haystacks at 0.48 recall (vs §7 #26's claimed 28/28 → 0.24 on real) 2026-05-13 06:35:36 -04:00
nli-shadow-sweep-benchqa-n1.json #000049 §7 #22: speedup (batch + cuda auto-detect + ONNX-int8 export) + the gate-item-4 verdict at proper n 2026-05-12 17:21:32 -04:00
nli-shadow-sweep-smoke-live.json #000049 §7 #21: candidate-clause restriction in ShadowNLI.check — helps, doesn't close it 2026-05-12 14:34:44 -04:00
nli-shadow-sweep.json #000049 §7 #21: candidate-clause restriction in ShadowNLI.check — helps, doesn't close it 2026-05-12 14:34:44 -04:00
nli-strict-fires-mining-pooled.jsonl #000049 §7 #26: mining for real recombinations + the pooled-808-STRICT check — verdict settles on margin@0.995 2026-05-12 21:11:35 -04:00
phase3-live-validation-2026-05-10.md Phase 3 live validation: 6/7 probes reach EVIDENCE-WARRANTED 2026-05-10 09:46:10 -04:00
phase3-warrant-chain-A-B-2026-05-10.md bench: #000031 Phase 3 A/B finds mechanism dormant on warrant-targeted fixture 2026-05-10 09:37:27 -04:00
phase3-warrant-chain-A-B-aggressive-2026-05-10.md bench: aggressive warrant fixture confirms Phase 3 is rescue-only, not default-path 2026-05-10 09:53:21 -04:00
phase3-warrant-chain-paraphrase-investigation-2026-05-10.md bench: Phase 3 paraphrase fixture investigation — empirically dormant on current corpus 2026-05-10 10:06:23 -04:00
pre-migration-snapshot.json #000065 step 1: routing helper + meta field + pre-migration snapshot 2026-05-26 13:13:44 -04:00
progressive-and-2026-05-09.md bench: progressive-AND fixture + 2026-05-09 A/B baseline report 2026-05-10 06:35:18 -04:00
prometheus-sigma-sweep-dryrun-2026-05-10.md #000037 §22: dry-run swap to sweep_weights() + §15.4 added 2026-05-10 19:22:03 -04:00
prometheus-sigma-triggers-2026-05-10-npower.md three-thread session output: stale TODOs, N-power probe, ForkScore Phase 1c 2026-05-10 07:46:35 -04:00
prometheus-sigma-triggers-2026-05-10.md #000037 §12 Trigger 2 fires empirically — Phase 1 gate satisfied 2026-05-10 17:55:19 -04:00
prometheus-sigma-triggers-2026-05-11.md #000012 Phase 1c follow-through: wire #000037 §12 Trigger 1 probe to fork_score_branches 2026-05-11 06:56:09 -04:00
real-shard-baseline.json modified: .gitignore 2026-05-08 16:38:09 -04:00
real-shard-baseline.md modified: .gitignore 2026-05-08 16:38:09 -04:00
relevance-aboutness-candidatebench-round2.json #000052 §3.2.1 round-2: extend size spectrum to 7 models (30MB-560MB, ~20×) — real model science from a wider sweep 2026-05-13 10:05:56 -04:00
relevance-aboutness-candidatebench.json #000052 §3.2.1: candidate-bench landed (4 models × 26 pairs on the 4090 — proper bench, not the 6-pair smoke) 2026-05-13 09:57:26 -04:00
relevance-shadow-sweep-pooled808-cleaned.json #000052 §3.2.2 refinement: claim-lattice metadata cleaning — STRICT FP drops universally (-0.5 to -5.6 pts), L-2 down to 1.5% 2026-05-13 15:01:50 -04:00
relevance-shadow-sweep-pooled808.json #000052 §3.2.2 step 2: relevance shadow sweep on 808 pooled bench-qa STRICT — universal walk-back, runtime veto NOT VIABLE on this design 2026-05-13 14:17:38 -04:00
shard-count-sweep-2026-05-26T16-20-48Z.csv #000065: pin M=4 + bench script + SQLite-alternative decision tree 2026-05-26 12:40:48 -04:00
shard-count-sweep-2026-05-26T16-31-34Z.csv #000065: pin M=4 + bench script + SQLite-alternative decision tree 2026-05-26 12:40:48 -04:00
textbook-base-knowledge-upgrade-2026-05-09.md textbook ingest: 4 base-knowledge additions for pillars I/II/III/IX 2026-05-09 20:38:32 -04:00
three-more-textbooks-2026-05-10.md 3 more textbooks (Russell PoM + Laplace + Dedekind): 54 -> 62/92 (67%) 2026-05-10 07:31:40 -04:00
warrant-chain-render-2026-05-09.md bench/results: warrant-chain render tail measurement (2026-05-09) 2026-05-09 18:00:36 -04:00
witness-sweep.json fan-out: witness automation · #000018 analysis · #000033 pillar VII (Claude) 2026-05-09 14:36:39 -04:00