Commit graph

3 commits

Author SHA1 Message Date
8d6961fcc1
aborist/arborist
modified:   .gitlab-ci.yml
	modified:   bench/qa_questions.txt
	modified:   bench/qa_sweep.py
	modified:   bench/run.sh
	modified:   docs/TICKETS.md
	modified:   docs/_source/README.md
	modified:   docs/_source/_ext/makefile_targets.py
	modified:   docs/_source/api/cli.rst
	modified:   docs/_source/api/distill.rst
	modified:   docs/_source/api/mesh.rst
	modified:   docs/_source/api/qa.rst
	modified:   docs/_source/api/retrieval.rst
	modified:   docs/_source/api/storage.rst
	modified:   docs/_source/api/substrate.rst
	modified:   docs/_source/concepts.rst
	modified:   docs/_source/conf.py
	modified:   docs/_source/cookbook.rst
	modified:   docs/_source/index.rst
	modified:   docs/_source/license.rst
	modified:   docs/_source/quickstart.rst
	modified:   docs/bench-maxing.md
	modified:   docs/benchmarks.md
	modified:   docs/cti-architecture.md
	modified:   docs/diagrams/aborist-modules.dot
	modified:   docs/diagrams/aborist-modules.svg
	modified:   docs/diagrams/mesh-data-flow.dot
	modified:   docs/diagrams/mesh-epoch-lifecycle.dot
	modified:   docs/diagrams/mesh-epoch-lifecycle.svg
	modified:   docs/diagrams/mesh-group-decisions.dot
	modified:   docs/diagrams/mesh-group-decisions.svg
	modified:   docs/diagrams/mesh-identity-stack.dot
	modified:   docs/diagrams/mesh-secret-envelope.dot
	modified:   docs/mesh.md
	modified:   docs/qa-modes-bench.md
	modified:   docs/seven-point-program.md
	modified:   docs/tickets/ticket-000001-retrieval-keywords-audit-gap.md
	modified:   docs/tickets/ticket-000002-reference-frame-polarity-contract.md
	modified:   docs/tickets/ticket-000003-anchor-class-warrant.md
	modified:   docs/tickets/ticket-000005-label-ladder-migration.md
	modified:   docs/tickets/ticket-000006-bench-emergent-findings.md
	modified:   docs/tickets/ticket-000007-query-layer-hyphen-fold.md
	modified:   docs/tickets/ticket-000008-broad-quantifier-preflight-guard.md
	modified:   docs/tickets/ticket-000009-quantifier-preflight-dag-binding.md
	modified:   docs/tickets/ticket-000010-metacognition-preflight-guard.md
	modified:   docs/tickets/ticket-000011-soft-preflight-hint-sidecar.md
	modified:   scripts/backfill_concepts.py
	modified:   scripts/bench_emergent.py
	modified:   tests/crawler/test_async_web_fetcher.py
	modified:   tests/crawler/test_bridge.py
	modified:   tests/crawler/test_web_fetch.py
	modified:   tests/test_bench_qa_sweep.py
	modified:   tests/test_burn.py
	modified:   tests/test_burn_doc.py
	modified:   tests/test_claim_lattice.py
	modified:   tests/test_cli_render.py
	modified:   tests/test_compress.py
	modified:   tests/test_concepts.py
	modified:   tests/test_dag.py
	modified:   tests/test_directives.py
	modified:   tests/test_distill.py
	modified:   tests/test_distill_recursive.py
	modified:   tests/test_evict.py
	modified:   tests/test_frame.py
	modified:   tests/test_grok_source.py
	modified:   tests/test_html_source.py
	modified:   tests/test_ingest.py
	modified:   tests/test_inspect.py
	modified:   tests/test_journal.py
	modified:   tests/test_keys.py
	modified:   tests/test_llm_context_base.py
	modified:   tests/test_merkle.py
	modified:   tests/test_mesh.py
	modified:   tests/test_mesh_aead.py
	modified:   tests/test_mesh_chain.py
	modified:   tests/test_mesh_cli.py
	modified:   tests/test_mesh_cli_pull.py
	modified:   tests/test_mesh_wire.py
	modified:   tests/test_mesh_wire_e2e.py
	modified:   tests/test_metacognition.py
	modified:   tests/test_migration_audit_mode.py
	modified:   tests/test_providence_source.py
	modified:   tests/test_qa.py
	modified:   tests/test_qa_quality_live.py
	modified:   tests/test_quantifier_caps.py
	modified:   tests/test_quantifier_classifier.py
	modified:   tests/test_quantifier_phase4.py
	modified:   tests/test_quantifier_reminder.py
	modified:   tests/test_query.py
	modified:   tests/test_reclassify.py
	modified:   tests/test_repair.py
	modified:   tests/test_resume.py
	modified:   tests/test_snapshot.py
	modified:   tests/test_soft_preflight.py
	modified:   tests/test_tfidf.py
	modified:   tests/test_vcs_source.py
	modified:   tests/test_verify.py
	modified:   tests/test_verify_json.py
	modified:   tests/test_versioned_ingest.py
	modified:   tests/test_warrant.py
	modified:   tests/test_wikipedia_old.py
	modified:   tests/test_wikipedia_xml.py
	modified:   tests/test_wikitext.py
2026-05-07 09:31:49 -04:00
d24291bc8b
qa(#000008): classifier fix — count-question short-circuit + bounded fixtures
Caught by the 2026-05-03 dry-run distribution review across the
73-question bench set (§10.11.3 step 2):

  intensity     pre-fix    post-fix
  SINGULAR      61 (84%)   65 (89%)
  MANY           4 ( 5%)    0 ( 0%)   ← all 4 were `how many X?`
  ALL            1 ( 1%)    1 ( 1%)
  COMPREHENSIVE  1 ( 1%)    1 ( 1%)
  OPEN_REQUEST   5 ( 7%)    5 ( 7%)
  SMALL_NUM      1 ( 1%)    1 ( 1%)

Defect: `how many states are there?` matched the bare `\bmany\b`
pattern in MANY rung — wrong. `how many X?` is a count-question
SHAPE, asking for ONE numeric answer ("50"), not enumeration of
many things. Cap should be 1 (SINGULAR), not 8 (Hermes MANY).

Fix: count-question short-circuit in classify_question_quantifier()
that returns SINGULAR for `^\s*(?:and\s+|but\s+|so\s+)?how (?:many|much)\b`.
Anchored at start so buried `how many` (e.g. "list all the states;
how many are there?") doesn't suppress the rest of the question's
quantifier markers — the leading `list all` still wins.

9 new tests pin: count questions classify SINGULAR, leading
conjunctions don't break the short-circuit, buried `how many` does
NOT short-circuit (verifies anchor is leading-only).

Bonus — Finding 2 from the dry-run review: zero bounded universals
in bench fixture. Adds two:

  name all members of the beatles
  list all planets in the solar system

Both classify ALL · scope_bound_hint=bounded so the §10.1 bounded-
vs-unbounded distinction has live bench coverage. Without these,
--reject-broad correctness on bounded universals has no automated
test fixture.

915 tests passing (9 new); 36 skipped.
2026-05-03 08:29:36 -04:00
926b05ed97
qa(#000008): Phase 1 — pure quantifier classifier (dry-run wired)
Lands aborist/qa/quantifier.py with classify_question_quantifier(),
a pure function mapping a question string onto the ten-rung
intensity ladder (ticket #000008 §2):

  ABSENT < SINGULAR < PROPORTIONAL < SMALL_NUM_EXPLICIT
       < COMPARATIVE_BOUND < FEW < MANY < ABSENT < ALL
       < OPEN_REQUEST < COMPREHENSIVE

Returns intensity, matched_token, explicit_count, is_broad,
operational_shape, scope_bound_hint, classifier_version. Pure: no
I/O, no model call, no retrieval call.

Highest-intensity-wins arbitration: COMPREHENSIVE strictly stronger
than OPEN_REQUEST (both > ALL). Catches "tell me everything about
all wars" → COMPREHENSIVE rather than dropping to one of the softer
shape detectors.

Scope_bound_hint heuristic (§10.1): bounded vs unbounded universals.
"All members of the Beatles" → bounded (corpus-known finite set).
"Winners of all major sports" → unbounded (scope undefined). Year-
anchored questions ("…in 2024") bound the universe to one event.
Heuristic only — corpus-arity check left for future refinement.

Wired into query() right after policy resolution. Both miss and
cache-hit paths surface quantifier_intensity, quantifier_matched
_token, scope_bound_hint, quantifier_explicit_count on the result
dict. claim_cap_applied is None until Phase 2 lands the cap-
application gate (default-off per §10.11.3 dry-run discipline).

61 new tests cover every rung, scope-bound detection (bounded /
unbounded / unknown), highest-wins arbitration, and regression
fixtures (factoid/wh-questions don't over-classify as broad).
2026-05-03 07:23:44 -04:00