ticket #000039 §13.5: amend test plan with CLI subprocess coverage
Phase 1 plan §13.5 named four test files (3 unit + 1 integration)
but did NOT name CLI subprocess tests for the three new CLI
surfaces §13.4 introduces:
- arborist ingest --embed (flag on existing subcommand)
- arborist query --retrieval={fts5|vec|hybrid} (new flag)
- arborist vec rebuild (new subcommand)
Per docs/calculator-test-patterns.md §6 (codified earlier today
in commit 0725eb4 from the three-module pattern bench): import-only
tests miss argparse + main() drift. Yesterday's substrate refactor
caught this hazard three times — fork_score.py import (85be5eb),
Makefile bench-fork-score target (209d670), .gitlab-ci.yml job
+ script (b320e27). Each fix was 1-3 lines, but each had been
shipped to main + would have surfaced as a noisy CI failure on
next pipeline run.
§13.5 now adds three CLI subprocess test files:
- tests/test_cli_vec_rebuild.py
- tests/test_cli_ingest_embed_flag.py
- tests/test_cli_query_retrieval_flag.py
Each gated via pytest.importorskip("sqlite_vec") so they skip
cleanly when [vec] extras absent. Pattern matches fox's
test_cli_baseline_runs_clean / test_cli_invalid_input_exits_2
in tests/test_t3_bound_calculator.py (the exemplar for
calculator-style CLI tests).
§13.7 size estimate revised: 4 test files → 7 test files (+3 CLI
subprocess), 250 → 400 test LOC. CLI subprocess tests are
~30-40 LOC each (boilerplate + tmp_path + subprocess.run +
JSON parse). Phase 1 total grows from ~550+250 → ~550+400 LOC.
Doc-only edit; doesn't unblock or block fox's §13.8 four
decisions — the test-plan addition is mechanical discipline,
not a scope change. Phase 1 still gates on the four §13.8
decisions before any code lands.
Cross-ref: docs/calculator-test-patterns.md §6 (CLI subprocess
pattern) + the three substrate-rename defect commits caught by
that pattern in retrospect (85be5eb / 209d670 / b320e27).
This commit is contained in:
parent
10b81e187d
commit
7ea0fdf2b1
1 changed files with 32 additions and 3 deletions
|
|
@ -735,6 +735,31 @@ Unit tests:
|
|||
fields fold into `governance_policy_hash`; flipping any one
|
||||
invalidates prior cached records.
|
||||
|
||||
CLI surface tests (per
|
||||
``docs/calculator-test-patterns.md`` §6 — subprocess invocation
|
||||
catches argparse + main() drift the import-only tests miss; this
|
||||
hazard surfaced three times during the 2026-05-10 substrate
|
||||
rename refactor in yesterday's commit chain, so we pin the
|
||||
pattern):
|
||||
|
||||
- `tests/test_cli_vec_rebuild.py` — subprocess invocation of
|
||||
``arborist vec rebuild --db <tmp_path/test.db> --batch 4``
|
||||
against a 12-chunk synthetic shard; assert exit 0, JSON
|
||||
status output if any, row-count parity in
|
||||
``chunk_vecs``. Skipped via ``pytest.importorskip`` when
|
||||
``[vec]`` extras absent.
|
||||
- `tests/test_cli_ingest_embed_flag.py` — subprocess invocation
|
||||
of ``arborist ingest --source html --embed --author X``
|
||||
against a tmp shard; assert ``--embed`` flag wires through
|
||||
to the embedder; row count after ingest matches; vec rows
|
||||
match chunk rows 1:1.
|
||||
- `tests/test_cli_query_retrieval_flag.py` — subprocess
|
||||
invocation of ``arborist query --retrieval=vec`` and
|
||||
``--retrieval=hybrid`` and ``--retrieval=fts5`` against a
|
||||
pre-embedded shard fixture; assert all three exit 0 and
|
||||
produce structurally-distinct retrieval candidate sets
|
||||
(vec ≠ fts5 on at least one query).
|
||||
|
||||
Integration tests (gated on `[vec]` extras installed; skipped
|
||||
otherwise):
|
||||
- `tests/test_vec_smoke_integration.py` — runs the §13.2 smoke on
|
||||
|
|
@ -754,14 +779,18 @@ otherwise):
|
|||
|
||||
- New files: 2 (embed.py, search/vec.py) ~ 200 LOC.
|
||||
- Patches: 4 (store.py, query.py, cli.py, Makefile) ~ 100 LOC.
|
||||
- Tests: 4 files ~ 250 LOC + fixtures.
|
||||
- Tests: 7 files ~ 400 LOC + fixtures (4 unit + 3 CLI subprocess
|
||||
per the §13.5 amendment 2026-05-10 + 1 integration). The CLI
|
||||
subprocess tests are ~30-40 LOC each — boilerplate + tmp_path
|
||||
+ subprocess.run + JSON parse.
|
||||
- Phase-1 doc append in `docs/`: this §13 + a journey-note bench
|
||||
result file.
|
||||
- pyproject.toml: 1 stanza.
|
||||
|
||||
Total: ~550 LOC + ~250 test LOC. Single substantial commit if all
|
||||
Total: ~550 LOC + ~400 test LOC. Single substantial commit if all
|
||||
tests pass + smoke succeeds; otherwise broken into the natural
|
||||
gates (smoke → embed.py → vec.py → integration tests → bench).
|
||||
gates (smoke → embed.py → vec.py → CLI subprocess tests →
|
||||
integration tests → bench).
|
||||
|
||||
### 13.8 Decisions fox needs to make to unblock Phase 1
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue