arborist/aborist
russell@unturf.com e5ffa6c6fd
qa(inspect): wordlist union (US+UK) + configurable supplemental dictionaries
Three changes that shape the same lever:

(1) The metaphor-cue wordlist now unions /usr/share/dict/words +
    /usr/share/dict/american-english + /usr/share/dict/british-english.
    The Debian split made the prior 'just symlink to american-english'
    miss British spellings (colour, organisation, realise) which
    silently became false negatives on British-speaker questions.
    Union: 102,485 → 104,305 entries on this machine. ~1,820 added
    British-specific entries.

(2) Supplemental dictionary support: operators can layer
    domain-specific vocabulary into the morphological substrate.
    Two paths:

      - Env var: ABORIST_METAPHOR_DICTS=/path/a:/path/b
      - Programmatic: register_metaphor_dictionary(path)

    Each supplemental dict is one word per line. The cue suffix
    tests (-ly stem, -ing stem, -est stem) then resolve domain
    stems automatically — adding 'aerodynamic' to a custom dict
    makes 'aerodynamically' classify as adverbial without code
    changes.

    Use case: 'a tree with its own vocabulary' — an aviation
    forest, a medical corpus, a legal-domain shard each carries
    jargon the standard wordlist doesn't cover. Register once,
    suffix tests pick up domain stems forever.

(3) README gains a 'Sidecar diagnostics' section with a table of
    the three sidecars (deflection, title-relevance, metaphor-
    deflection) plus a 'Metaphor-deflection cue dictionary' subsection
    explaining the derivation rule, the load order, and the per-
    forest vocabulary configurability. Architecturally documents
    why the rule is *derived* from the union (Phase-2 lesson) and
    not hand-curated.

3 new tests in tests/test_inspect.py:
  - register_metaphor_dictionary unions a custom path's words
  - ABORIST_METAPHOR_DICTS env var supplements with two paths
  - re-registering same path is idempotent

763/34 tests pass.
2026-05-02 14:27:24 -04:00
..
concepts qa(retrieval): documents_fts FTS5 index — title search 1250× faster 2026-05-02 06:57:10 -04:00
distill storage cheats + TF-IDF retrieval fix 2026-04-27 17:24:51 -04:00
mesh mesh: per-peer audit chain-of-claims tracking on the wire 2026-04-28 19:41:25 -04:00
qa qa(inspect): wordlist union (US+UK) + configurable supplemental dictionaries 2026-05-02 14:27:24 -04:00
search qa(retrieval): synonym-aware FTS5 OR-fallback restores brain-tech surfacing 2026-05-01 22:56:50 -04:00
sources docs: consolidate self-reference design — flat MVP + fact-Core distillation 2026-05-02 12:39:42 -04:00
__init__.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
cli.py qa: tighten TITLE_MISMATCH + DEFLECTION demotes from emergent-log review 2026-05-02 13:33:48 -04:00
compress.py compress: thread-local zstd contexts; bench: --concurrency flag 2026-05-02 07:36:33 -04:00
document.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
evict.py storage cheats + TF-IDF retrieval fix 2026-04-27 17:24:51 -04:00
ingest.py storage cheats + TF-IDF retrieval fix 2026-04-27 17:24:51 -04:00
merkle.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
progress.py progress reporter + structured benchmark 2026-04-27 11:37:20 -04:00
snapshot.py add corpus-level snapshots: single-hash identity for the forest 2026-04-27 21:29:10 -04:00
source.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
store.py speed: pytest-xdist, bench smoke, concurrency default; UTF surrogate fix 2026-05-02 09:29:40 -04:00
wikitext.py verify: strip wikitext from context before substring matching 2026-04-28 15:48:07 -04:00