qa: dedup-mode policy + JIT fidelity — agents pick fast-cache vs audit-grade
Different agents have different value functions on the same providence
records. Today aborist bakes one canonicalization policy into
governance_policy_hash and calls it universal. This adds two knobs that
let agents express their preference without breaking provenance.
- aborist/qa/keys.py: `canonical_question(q, mode=...)` and
`question_hash(q, mode=...)` accept "strict" (NFC + ws-collapse only;
every variant gets its own hash) or "equivalence_class" (default —
additionally lowercase + trailing-punct strip + article strip).
New constants: QUESTION_DEDUP_MODES, FIDELITY_MODES, defaults.
- aborist/qa/query.py + aborist/qa/runner.py: each call computes the
primary cache_key under policy["question_dedup"]. New `fidelity`
parameter:
"strict" only primary cache_key checked
"equivalence_class" primary first; if miss AND alternate mode
produces a different cache_key, try alternate
Cross-silo fallback works because _ckey_for_mode rewrites
policy["question_dedup"] to the alternate mode before computing
governance_policy_hash — so the fallback ckey matches what an agent
under that mode would have written. Lookups can find each other's
records when fidelity permits. Result dict gains `lookup_path` ∈
{"strict", "equivalence_class", "strict_fallback",
"equivalence_class_fallback", "miss"}.
- aborist/cli.py: `--question-dedup` and `--fidelity` flags on the
`query` subcommand. Human render annotates cache_hit lines that
came from a fallback ckey ("cached via equivalence_class_fallback").
- tests/test_query.py: three new regressions
* strict-policy + strict-fidelity: variants get distinct cache_keys
* cross-silo fallback: strict-policy reads eq-class-policy records
via fidelity=equivalence_class
* strict-fidelity refuses fallback (audit-grade): cache miss even
when the alternate silo has a hit
- CLAUDE.md: dedup-mode and fidelity convention bullets updated.
This is V1 + V2 of the substrate move sketched in conversation: write
policy determines which silo a record lives in; read fidelity determines
how loosely an agent walks across silos. Records stay exactly-keyed
(provenance hard); routing is per-agent (preference soft). Same
verifier_no_diagnostics discipline holds — no soft signal enters the
hard chain.
442 tests pass.
This commit is contained in:
parent
1e8d2d2dba
commit
aefc032749
6 changed files with 431 additions and 90 deletions
|
|
@ -87,7 +87,8 @@ make chain-check-shards # audit-chain break count per shard (0 = intact)
|
|||
- **Chunker default = `tok-512-v1`.** Changing the default bumps `chunking_version` and **stales every prior cache record**. Add a new chunker as a new `name` instead.
|
||||
- **Canonicalization = `norm-v1`** (NFC, collapsed whitespace). Same rule.
|
||||
- **Schema = `v9.8.0`.** Same rule.
|
||||
- **`question_hash` canonicalization** drops trailing punctuation (ASCII `.?!,;:`, CJK `?!。、`, ellipsis `…`) AND standalone English articles (`the`, `a`, `an`) after lowercase + NFC + whitespace-collapse. So `"who is X"`, `"who is X?"`, `"Who Is X."`, `"who is the X"`, `"who is an X?"` all hash to the same value. Tokens are matched as exact lowercase strings, so substrings like `"thesis"` (contains `the`) stay untouched. Conservative on purpose — only ASCII English articles. Bumping `_QUESTION_TRAILING_STRIP` or `_QUESTION_ARTICLE_STRIP` orphans prior records whose canonical question contained newly-stripped tokens; they live as history but won't be re-hit on lookup.
|
||||
- **`question_hash` is dedup-mode-aware.** `question_hash(q, mode=...)` accepts `"strict"` (NFC + ws-collapse only — case-sensitive, punctuation-sensitive, article-sensitive; every variant gets its own hash) or `"equivalence_class"` (default — additionally lowercase + trailing-punct strip + standalone-article strip; collapses `"who is X"`, `"who is X?"`, `"Who Is X."`, `"who is the X"`, `"who is an X?"` to one hash). Mode lives in `policy["question_dedup"]` so it folds into `governance_policy_hash`; agents under different modes write records under different cache_keys.
|
||||
- **JIT lookup fidelity.** Per-call `fidelity` parameter on `query()`/`ask()` decouples lookup tolerance from write policy. `"strict"` checks only the primary cache_key (audit-grade — record reuse only on exact match). `"equivalence_class"` (default) tries primary first, then the alternate dedup mode's cache_key as a fallback so a fast-cache agent can reuse records written under either policy. Result includes `lookup_path` ∈ `{"strict", "equivalence_class", "strict_fallback", "equivalence_class_fallback", "miss"}`. Cross-silo fallback works because the helper rewrites `policy["question_dedup"]` to the alternate mode when computing the fallback ckey, so `governance_policy_hash` matches what an agent under that mode would have written. CLI: `--question-dedup`, `--fidelity` flags on `query` (and the `ask` API params).
|
||||
- **`audit_mode` is decided by the verifier, never asserted unconditionally.** Four layered strategies in `aborist/qa/verify.py`, tried in order; first to find evidence classifies the answer:
|
||||
1. **quote** — model wrapped claims in double quotes per system prompt. Sequential pairing: 1st & 2nd `"`, 3rd & 4th, etc. (NOT regex pairing — that captures inter-pair prose as a phantom span when the model writes `"title" prose "quote"`).
|
||||
2. **span** — bullet/sentence lines from the answer appear verbatim in context. Catches models that quote inline without `"..."` marks.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue