russell@unturf.com
7871e1fdfe
Three streams. Two land in the repo; one lands in fox's Downloads
(existing g4 packs live there too).
Witness sweep automation
========================
`bench/scripts/witness_sweep_cron.sh` — schedulable harness that
runs `make bench-witness-sweep` + `make bench-witness-divergence`
back-to-back and (with --commit) auto-commits any new divergence
fixtures + sweep result. Fail-closed: pre-commit hook failure
aborts; non-main-branch refuses to commit.
$ bench/scripts/witness_sweep_cron.sh # dry run
$ bench/scripts/witness_sweep_cron.sh --commit # auto-commit
$ bench/scripts/witness_sweep_cron.sh --commit --push # full pipeline
Recommended cron (every 6 hours; LLM cost scales with frequency):
0 */6 * * * \
bench/scripts/witness_sweep_cron.sh --commit --branch main \
>> /var/log/arborist-witness.log 2>&1
Smoke this commit: dry-run produced 5 fresh divergences (10
cumulative across two extraction passes). The witness-sweep + 5F
extractor pipeline runs unattended now; calibration-data corpus
will grow over time.
#000018 — adversarial soft-hash covert-channel analysis
========================================================
`docs/soft-hash-channel-analysis.md` — formal threat model + channel-
capacity definition + cryptographic reduction + mitigation table
+ residual analysis. Doc-only deliverable per the ticket's hard
constraint.
Headline result: under threat models T1 (data poisoner) and T2
(gradient adversary), the channel is bounded by SHA-256 partial-
preimage hardness when φ (the anchor map) is independent of
parameter geometry. T3 (hyperparameter adversary) requires
M2 (per-checkpoint anchor nonce) for boundedness.
Recommended mitigation: M2 (per-checkpoint nonce) — 32 bytes added
to the snapshot manifest, no training-ergonomics regression,
closes T1+T2 channels under any φ choice and bounds T3 by
per-checkpoint window. Fall back to M1 (PRG-based φ) if M2-only
deployment surfaces structural concerns. M3 (drop anchor entirely)
stays in reserve as the strict-construction fallback.
Three open questions (§9): Hessian alignment under φ_linear,
PRG choice for φ_PRG, and explicit T3 per-window bound. Each is a
follow-up ticket.
Ticket #000018 status: closed · landed 2026-05-09 (analysis doc).
v7 § 9.10 spec amendment proposed in §7 of the analysis.
#000033 — pillar VII (combinatorics), Claude-authored
======================================================
NOT committed to the arborist repo (the existing g4-v2 packs live
in `/home/fox/Downloads/` too — that's the operator's bundle
location). Two new bundle files at:
/home/fox/Downloads/axiomsclaude-vii-v1.json (7 axioms)
/home/fox/Downloads/theoremsclaude-vii-v1.json (7 theorems)
Pillar VII covers combinatorial counting — the gap between Grok's
pillars VI and IX in the v2 packs:
axioms (7): addition principle · multiplication principle ·
pigeonhole principle · factorial definition ·
binomial coefficient definition · Pascal's rule ·
empty-set / boundary axiom
theorems (7): binomial theorem · inclusion-exclusion (counting
form) · hockey-stick identity · Vandermonde's
identity · Catalan number closed form · stars-and-
bars · strong pigeonhole
Each record in the dual-thread format the existing g4 packs use
(Δ symbolic LaTeX + ∇ verbose prose + ∇ concise + sigil + formal
language + role + status + source_reference + date + foundational
group + category + subfield). Per fox's directive: explicit
authorship metadata everywhere — `authored_by: Claude (Anthropic)
— model claude-opus-4-7`. NOT Grok-generated; no silent invention.
Each record carries `pi_star_ref: combinatorics@v1` so the kernel
binding is explicit. Theorems list `depends_on_axioms` arrays so
each theorem cites the foundation axioms it bottoms out on.
Smoke test (committed alongside):
$ arborist --db /tmp/test.db ingest --source claim_pack \\
--bundle /home/fox/Downloads/axiomsclaude-vii-v1.json \\
--bundle /home/fox/Downloads/theoremsclaude-vii-v1.json
→ 14 docs, 14 chunks, 0 cross-bundle edges
Source attributions: Stanley EC1, Brualdi Introductory
Combinatorics, Knuth TAOCP Vol 1, plus historical sources where
applicable (Pascal 1654, Vandermonde 1772, Dirichlet 1834, Catalan
1838, Feller 1950 for stars-and-bars).
Tests: 1636 passed, 37 skipped (no regressions; pillar VII
ingestion smoke covered above).
124 lines
No EOL
4.1 KiB
JSON
124 lines
No EOL
4.1 KiB
JSON
{
|
|
"schema_version": "witness-sweep-v1",
|
|
"timestamp_utc": "2026-05-09T18:27:23+00:00",
|
|
"shards_dir": "/home/fox/.arborist/shards",
|
|
"model": "adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic",
|
|
"endpoint": "https://hermes.ai.unturf.com/v1",
|
|
"summary": {
|
|
"questions": 8,
|
|
"by_agreement_label": {
|
|
"LLM-DIVERGED": 5,
|
|
"STRICT-WITNESSED": 3
|
|
},
|
|
"divergence_count": 5,
|
|
"divergence_rate": 0.625,
|
|
"wall_ms_median": 159.2,
|
|
"wall_ms_max": 1099.3
|
|
},
|
|
"queries": [
|
|
{
|
|
"question": "0.1 + 0.2",
|
|
"status": "cache_hit",
|
|
"audit_mode": "CANONICAL_PROJECTION",
|
|
"pi_star_ref": "arithmetic@v1",
|
|
"canonical_answer": "3/10",
|
|
"agreement_label": "LLM-DIVERGED",
|
|
"llm_raw": "1/10",
|
|
"llm_canonical_match": false,
|
|
"cache_status": "ok",
|
|
"wall_ms": 1099.3,
|
|
"audit_event_hash": "3b25ad3cd28a9505d96e22c85ae42ab182008ee6584b3a74c881dba07e3eab28"
|
|
},
|
|
{
|
|
"question": "1/3 + 1/6",
|
|
"status": "cache_hit",
|
|
"audit_mode": "CANONICAL_PROJECTION",
|
|
"pi_star_ref": "arithmetic@v1",
|
|
"canonical_answer": "1/2",
|
|
"agreement_label": "STRICT-WITNESSED",
|
|
"llm_raw": "1/2",
|
|
"llm_canonical_match": false,
|
|
"cache_status": "ok",
|
|
"wall_ms": 159.2,
|
|
"audit_event_hash": "7677efdeb3b2ad7f60cf4312d2f4825c9fa949ddcbc06434ee5fc4ea50a86e3e"
|
|
},
|
|
{
|
|
"question": "2**10",
|
|
"status": "cache_hit",
|
|
"audit_mode": "CANONICAL_PROJECTION",
|
|
"pi_star_ref": "arithmetic@v1",
|
|
"canonical_answer": "1024/1",
|
|
"agreement_label": "STRICT-WITNESSED",
|
|
"llm_raw": "1024",
|
|
"llm_canonical_match": false,
|
|
"cache_status": "ok",
|
|
"wall_ms": 134.7,
|
|
"audit_event_hash": "ed570bd6127a0b7143bb5c0f8cf76daf37a2092fe84350c0cab1ee2a2bf0410b"
|
|
},
|
|
{
|
|
"question": "A IMPL B",
|
|
"status": "cache_hit",
|
|
"audit_mode": "CANONICAL_PROJECTION",
|
|
"pi_star_ref": "logic-kernel@v1",
|
|
"canonical_answer": "(NOT A OR B)",
|
|
"agreement_label": "LLM-DIVERGED",
|
|
"llm_raw": "TRUE",
|
|
"llm_canonical_match": false,
|
|
"cache_status": "ok",
|
|
"wall_ms": 147.0,
|
|
"audit_event_hash": "0f6fb926fd72150c5a0d8121a004b1f66edf3c8ee4bc15c52168f3a7869786fe"
|
|
},
|
|
{
|
|
"question": "(NOT B) IMPL (NOT A)",
|
|
"status": "cache_hit",
|
|
"audit_mode": "CANONICAL_PROJECTION",
|
|
"pi_star_ref": "logic-kernel@v1",
|
|
"canonical_answer": "(NOT A OR B)",
|
|
"agreement_label": "LLM-DIVERGED",
|
|
"llm_raw": "TRUE",
|
|
"llm_canonical_match": false,
|
|
"cache_status": "ok",
|
|
"wall_ms": 260.2,
|
|
"audit_event_hash": "1b17f6e39edf2bb3d8dce003e20f71df9a3b9bd1b885f8a6e4b78e43ec122c31"
|
|
},
|
|
{
|
|
"question": "A OR NOT A",
|
|
"status": "cache_hit",
|
|
"audit_mode": "CANONICAL_PROJECTION",
|
|
"pi_star_ref": "logic-kernel@v1",
|
|
"canonical_answer": "TRUE",
|
|
"agreement_label": "STRICT-WITNESSED",
|
|
"llm_raw": "TRUE",
|
|
"llm_canonical_match": false,
|
|
"cache_status": "ok",
|
|
"wall_ms": 129.1,
|
|
"audit_event_hash": "5929ffe218bb8a97c391df70cfffed810f66d02c4a54bdce2569755200bd8d0d"
|
|
},
|
|
{
|
|
"question": "(x+1)**2",
|
|
"status": "cache_hit",
|
|
"audit_mode": "CANONICAL_PROJECTION",
|
|
"pi_star_ref": "algebra-symbolic@v1",
|
|
"canonical_answer": "Add(Pow(Symbol('x'), Integer(2)), Mul(Integer(2), Symbol('x')), Integer(1))",
|
|
"agreement_label": "LLM-DIVERGED",
|
|
"llm_raw": "1",
|
|
"llm_canonical_match": false,
|
|
"cache_status": "ok",
|
|
"wall_ms": 196.1,
|
|
"audit_event_hash": "b36f1a830aae8fec514f95f3104ba4473bc6febb98137016c5d0816f859ba440"
|
|
},
|
|
{
|
|
"question": "x**2 + 2*x + 1",
|
|
"status": "cache_hit",
|
|
"audit_mode": "CANONICAL_PROJECTION",
|
|
"pi_star_ref": "algebra-symbolic@v1",
|
|
"canonical_answer": "Add(Pow(Symbol('x'), Integer(2)), Mul(Integer(2), Symbol('x')), Integer(1))",
|
|
"agreement_label": "LLM-DIVERGED",
|
|
"llm_raw": "1",
|
|
"llm_canonical_match": false,
|
|
"cache_status": "ok",
|
|
"wall_ms": 147.3,
|
|
"audit_event_hash": "d307d6990527ddc6af73ea3cb894d4986ea3003106723fe74e22282f16eb62c6"
|
|
}
|
|
]
|
|
} |