russell@unturf.com
823e8da1ff
Two follow-up defects from 2026-06-14 factory triage:
(1) DLQ runner classifier did not recognize "vram-oversized" reason text
introduced by foxhop dispatcher pre-flight (commit 8d45e0d on the foxhop
side). 65 of 87 rDLQ cells got escalated as class=unknown instead of a
properly named bucket. Adds pattern + escalate-class entry + reducer
test case mapped to (vram-oversized sim no — bin is dead weight on this
card, salvage skipped).
(2) sweep-doctrine reducers had no per-test timeout. K=5 doctrine tests
(test-k5-apply-forward-ipmul + 4 siblings) ran lumbda at 99% CPU for
2h43m on a remote node without ever emitting their DOCTRINE verdict
line — accumulating 30+ runaway lumbda procs under two stuck `make
sweep-doctrine` invocations. run.sh + run-parallel.sh now wrap our
lumbda invocation in `timeout ${SWEEP_DOCTRINE_TEST_TIMEOUT_S:-300}`;
hit exits 124, our existing "no DOCTRINE line" branch logs HARNESS-FAIL.
K=5 substrate has a documented non-terminating compute defect AND a
load-time buffer overflow (commit 6d18c59 on foxhop). Bisect deferred
per ticket 0007 in foxhop tree; needs qemu apparatus we currently lack.
|
||
|---|---|---|
| .. | ||
| lib.lsp | ||
| README.md | ||
| run-parallel.sh | ||
| run.sh | ||
sweep-doctrine — TCRAUDT reducer library
Test-Case Reducers Are Underappreciated Debugging Tools. Source: Laurie Tratt 2026-01 essay, building on Zeller delta-debugging + Regehr C-reducer. Reduce a failing probe to a 5-line test, commit it as institutional memory. Tests outlive every sweep agent's working memory.
Discipline
5 phases (mandatory for every defect):
- Reproduce at full production scope first — verify defect, capture failing output.
- Reduce inputs to smallest case that still triggers — peel flags, widths, iters, params.
- Commit reducer to
tests/sweep-doctrine/— NEVER inruns/(runs/ stays ephemeral; reducers outlive sweeps). - Fix defect.
- Verify reducer flips EXPECTED-FAIL → PASS.
make sweep-doctrineexits 0.
Test format
Each test-*.lsp prints exactly one verdict line:
DOCTRINE <name> PASS got=<actual> expected=<expected>
DOCTRINE <name> EXPECTED-FAIL got=<actual> expected=<expected> ; known open defect, alarm acknowledged
DOCTRINE <name> FAIL got=<actual> expected=<expected> ; regression — gate fails
Shared helpers in lib.lsp — assertions, fresh-register builders, expected-truth-table comparators.
Runner
make -C ~/git/lumbda sweep-doctrine # local, all reducers
make -C ~/git/lumbda sweep-doctrine REMOTE=user@host # via SSH
Backed by run.sh (serial) + run-parallel.sh (xargs -P fan-out).
When reduction stops shrinking, escalate
Reducer that pins a trigger pair without root cause = building on sand — gate stays green while wrong-output ships. Escalate:
- Unit — fire suspect primitive on register state mimicking callsite geometry. Compare bits to reference.
- Integration — emit one iter of suspect operation, read every intermediate register. First divergent register names broken step.
- Functional — full operation at small width with
record-register-state!at iter N. Iter where divergence first appears names failure mode.
Reducer = alarm. EXPECTED-FAIL = alarm acknowledged, not silenced. Don't promote any change touching a primitive whose reducer stays EXPECTED-FAIL.
Hard rules
- Reducers belong in
tests/sweep-doctrine/; ephemeral sweep artifacts (probe scratch, dispatched cells, per-sweep RESULTS) belong in consumer-sideruns/directories. Never invert. make sweep-doctrineexit 0 iff every test produced expected outcome (PASS for positive controls, EXPECTED-FAIL for known open defects).
License
AGPLv3.