Today `question_hash` canonicalizes (NFC, ws-collapse, lowercase) but
preserves trailing punctuation. So "who is X?" and "who is X" produce
different hashes, different cache_keys, different providence_cache
records — the cache misses on what's semantically the same question.
Add `rstrip(".?!,;:")` as the third canonicalization step inside
question_hash specifically. Local to question hashing — chunk leaf
hashes go through `canonicalize()` directly and are unaffected.
Equivalence class after this:
"who is X" \
"who is X?" |
"who is X." | -> all same question_hash, all same cache_key
"who is X!" |
"Who Is X" /
Internal punctuation is preserved on purpose: "X, then Y" carries
meaning that "X then Y" doesn't, even though both have the same
content tokens.
One-time impact: prior cached records whose canonical question ended
in punctuation become orphans on lookup (re-derive on next ask).
History stays on disk; nothing burned automatically.
New test file tests/test_keys.py with 13 tests pinning the question_hash
equivalence class plus light coverage of the other dim hashes
(model_profile, conversation, governance, cache_key 8-dim invariant).
286 passed, 1 skipped.
russell@unturf.com
a51ca712a0