feat(#000057): arborist+qwen enablement — multi-engine JSON-schema + per-model extras pass-through
Two surgical fixes unblock 'arborist with synthesis LLM = Qwen-on-
llama.cpp' as a viable arm in the control sweep. Pre-existing
docstring said 'Arborist×Qwen needs proof-path guided_json+extra_body
surgery — coupled follow-up'; this is that follow-up.
Fix 1 — multi-engine structured-output extras
The runner / query JSON-mode paths previously sent only vLLM's
'guided_json' key for the claim_lattice schema. llama.cpp silently
drops it, leaving Qwen un-enforced (the parse-tolerant fallback did
all the work). Helper
claim_lattice_structured_output_extras() in arborist/qa/verify.py
now returns a dict carrying the schema under all three engine
conventions:
- guided_json (vLLM grammar-constrained sampling)
- json_schema (llama.cpp native shorthand)
- response_format (OpenAI-spec, honoured by llama.cpp and newer vLLM)
Each engine recognises its own key and silently drops the others.
Used at both inference call sites (runner.py:740, query.py:3324).
Hermes/vLLM path is unchanged — it picks up 'guided_json' and
ignores the other two.
Fix 2 — query() accepts user-supplied extra_body, merges with defaults
query() grew a keyword-only extra_body parameter (default None).
Per-model knobs (Qwen's {'chat_template_kwargs': {'enable_thinking':
False}} toggle, future template knobs) can flow from the caller to
the synthesis chat-completion call. Schema-enforcement extras are
added inside query() and merge under user keys — common case is
disjoint namespaces, but if a caller wants to override 'guided_json'
they can.
bench/control_sweep.py now passes MODELS[arborist_ref]['extra']
through to query() in the arborist branch, so --arborist-ref
qwen-nothink runs with reasoning disabled and --arborist-ref
qwen-think runs with reasoning enabled. Phase 1's arborist arm with
--arborist-ref=hermes is unaffected (MODELS['hermes']['extra'] is
None, merges to no-op).
Tests
+ 3 new in tests/test_verify_json.py covering helper default shape,
alternate-schema reuse, and query()'s new extra_body parameter
220 affected tests still green (verify / claim_lattice / judge /
runner suite)
pytest test_verify_json: 27/27
Next: small smoke run --arborist-ref qwen-nothink against 4-8 items
to confirm end-to-end before any full sweep. Phase 2 (qwen-think solo)
still running in background, unaffected — it doesn't touch the
arborist arm.
This commit is contained in:
parent
3450e8a281
commit
c6621ee700
5 changed files with 141 additions and 11 deletions
|
|
@ -189,10 +189,21 @@ def _process_item(idx: int, it: dict, variants: list[str],
|
|||
"judge_raw": (v.raw or "")[:700]})
|
||||
if arborist_on:
|
||||
try:
|
||||
# Pass through MODELS[arborist_ref]["extra"] so
|
||||
# per-model knobs (e.g. Qwen's enable_thinking
|
||||
# toggle under chat_template_kwargs) reach the
|
||||
# synthesis call in the proof-path. Without this,
|
||||
# an arborist-ref=qwen-nothink sweep would still
|
||||
# emit reasoning tokens because the toggle never
|
||||
# crosses query()'s boundary. JSON-schema extras
|
||||
# are added inside query() via
|
||||
# claim_lattice_structured_output_extras() and
|
||||
# merge with this per-model extras dict.
|
||||
r = query(question=q_asked, qa_db=qa_db,
|
||||
chat_client=cl(arborist_ref),
|
||||
model_id=MODELS[arborist_ref]["model"],
|
||||
shards_dir=shards_dir,
|
||||
extra_body=MODELS[arborist_ref]["extra"],
|
||||
policy=dict(DEFAULT_QUERY_POLICY,
|
||||
answer_mode="claim_lattice"))
|
||||
arb_raw = (r.get("raw_answer")
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue