engineering/arborist
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
arborist/tests
History
russell@unturf.com ea39455d75
v8: ticket #000012 Phase 1a — ForkScore consumes the new bench substrate 
Pure scoring function over (parent, child) BatteryResult bundles. Closes
the scoring half of fox's 2026-05-08 frontier note ("how does an organism
mutation become canonical?") — the canonicalization half (validator
state, acceptance, fork choice) stays under #000012 as the v8 paper.

Formula:

  ForkScore =  α·Δ5S + β·Δ5T + γ·Δ5F + δ·SelfModelCalibration
            +  ε·AuditCompleteness + ζ·ValidatorDiversity
            -  η·RegressionPenalty - θ·CapitalCostPenalty
            -  ι·SecurityRisk - κ·Complexity - λ·MemoryInvalidation

Consumes every metric this session shipped:

- 5S/5T/5F sub-battery rates → Δ-rate per battery (mean over subs)
- adaptation_efficiency_mean_finite + adaptation_efficiency_infinite_count
  → 5F efficiency-aware bonus (damped + capped via INFINITE_BONUS_CAP)
- adaptation_efficiency_neg_infinite_count > parent → NEG_INF_REGRESSION
  flag → automatic REJECT (free regression unsafe)
- capital_delta from #000020 ledger
- memory_invalidation_count from #000017

Verdict thresholds:

- ACCEPT  when score >= SIGNAL_FLOOR (5pp; matches docs/bench-maxing.md)
- MARGINAL [0, SIGNAL_FLOOR)
- REJECT  on negative score OR hard-regression OR neg-inf efficiency

Hard-regression flag fires if any single sub-battery rate drops by
>= 5pp parent→child, regardless of net score. CLI exits 1 on REJECT
so CI gates run `arborist v8 score` directly.

Surface:

- arborist/v8/{fork_score,weights}.py
- WeightSet dataclass with α…λ + DEFAULT_WEIGHTS (single-validator
  tuned: ζ=0, ι=0, κ=0; η=2.0 weighted heavier than improvement
  weights; θ=0.5 modest cost penalty)
- weights_from_dict accepts "lambda" key (Python reserved word)
- bench_result_to_metrics adapter from runner --all JSON
- CLI: arborist v8 score --parent P.json --child C.json [--weights W.json]
  with override flags for SelfModelCalibrationGain, AuditCompleteness,
  capital_delta, memory_invalidation_count, etc.

Reference: docs/v8-fork-score.md (formula + term semantics + verdict
matrix + Phase-1a vs Phase-1b boundary).

Tests: tests/test_v8_fork_score.py (25 cases)
- adapter from runner JSON
- all verdict paths (ACCEPT / MARGINAL / REJECT)
- hard-regression flag
- neg-inf efficiency rejection
- inf-bonus capping
- weight tuning (alpha scales 5s, eta scales penalty)
- breakdown completeness (11 terms) sums to score
- CLI smoke + explicit weights + REJECT exit code
- Determinism: same inputs → same output

Full suite: 1186 passed, 36 skipped.

#000012 status: in progress (Phase 1a landed, consensus paper still open).
2026-05-08 07:18:22 -04:00
..
crawler aborist/arborist 2026-05-07 09:31:49 -04:00
fixtures verify: strip wikitext from context before substring matching 2026-04-28 15:48:07 -04:00
__init__.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_bench_batteries.py tests: unit + integration + functional coverage for this session's surface 2026-05-07 21:15:14 -04:00
test_bench_qa_sweep.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_burn.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_burn_doc.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_capital.py capital: land ticket #000020 (8-capital-form cost ledger) 2026-05-07 16:41:48 -04:00
test_claim_lattice.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_cli_render.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_cli_session.py tests: unit + integration + functional coverage for this session's surface 2026-05-07 21:15:14 -04:00
test_compress.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_concepts.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_dag.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_directives.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_distill.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_distill_recursive.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_evict.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_frame.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_grok_source.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_html_source.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_ingest.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_inspect.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_journal.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_keys.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_llm_context_base.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_loss_report.py loss_report: land ticket #000022 (adapter LossReport sidecar) 2026-05-07 17:58:50 -04:00
test_memory_root.py memory: land ticket #000017 (memory-root lifelong learning summary) 2026-05-07 16:46:41 -04:00
test_merkle.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_aead.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_chain.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_cli.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_cli_pull.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_wire.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_wire_e2e.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_metacognition.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_migration_audit_mode.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_pi_star.py tests: unit + integration + functional coverage for this session's surface 2026-05-07 21:15:14 -04:00
test_providence_source.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_qa.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_qa_quality_live.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_quantifier_caps.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_quantifier_classifier.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_quantifier_phase4.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_quantifier_reminder.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_query.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_reclassify.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_repair.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_resume.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_selfmodel.py selfmodel: land ticket #000014 (identity record + falsification) 2026-05-07 16:36:34 -04:00
test_session_integration.py tests: unit + integration + functional coverage for this session's surface 2026-05-07 21:15:14 -04:00
test_session_migrations.py tests: unit + integration + functional coverage for this session's surface 2026-05-07 21:15:14 -04:00
test_snapshot.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_soft_preflight.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_tfidf.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_v8_fork_score.py v8: ticket #000012 Phase 1a — ForkScore consumes the new bench substrate 2026-05-08 07:18:22 -04:00
test_vcs_source.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_verify.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_verify_json.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_versioned_ingest.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_warrant.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_wikipedia_old.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_wikipedia_xml.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_wikitext.py aborist/arborist 2026-05-07 09:31:49 -04:00