fix: Qwen3 defaults to enable_thinking=False — was returning empty answers
Root cause of 'arborist abstains on everything with qwen' (fox 2026-05-21): Qwen3 thinking-on default burns the entire token budget on hidden <think> reasoning over a 20K RAG context and returns EMPTY message.content (measured: 768/768 completion tokens, content '') -> every arborist answer UNGROUNDED. Bench harnesses passed enable_thinking=False via the MODELS dict, but the CLI + control_ab did not, so the quality bench was measuring a thinking-budget-exhaustion artifact, not abstention. OpenAICompatibleClient now defaults Qwen3 to enable_thinking=False unless a caller set it explicitly (reasoning-variant path passes True, preserved). Verified: same France query goes empty/UNGROUNDED -> STRICT 'Nicolas Sarkozy' with the flag. Fixes every caller (CLI, control_ab). 5 tests; full suite 2547 passed. Today's qwen QUALITY numbers are void and need re-running; energy numbers stand (real inference happened regardless).
This commit is contained in:
parent
2fd3523777
commit
39c040cacc
2 changed files with 56 additions and 0 deletions
34
tests/test_qwen_thinking_default.py
Normal file
34
tests/test_qwen_thinking_default.py
Normal file
|
|
@ -0,0 +1,34 @@
|
|||
"""Qwen3 enable_thinking=False default (2026-05-21).
|
||||
|
||||
Qwen3 thinking-on default burns the token budget on hidden <think> over a
|
||||
large RAG context and returns empty content → every answer UNGROUNDED.
|
||||
The client forces enable_thinking=False for Qwen3 unless a caller set it.
|
||||
"""
|
||||
from arborist.qa.client import _qwen3_no_think_default as f
|
||||
|
||||
|
||||
def test_qwen3_gets_no_think_by_default():
|
||||
eb = f("Qwen3.6-27B-UD-Q4_K_XL.gguf", None)
|
||||
assert eb["chat_template_kwargs"]["enable_thinking"] is False
|
||||
|
||||
|
||||
def test_qwen3_merges_into_existing_extra_body():
|
||||
eb = f("Qwen3.6-27B", {"guided_json": {"x": 1}})
|
||||
assert eb["guided_json"] == {"x": 1}
|
||||
assert eb["chat_template_kwargs"]["enable_thinking"] is False
|
||||
|
||||
|
||||
def test_explicit_thinking_true_is_preserved():
|
||||
# the reasoning-variant path must keep thinking ON.
|
||||
eb = f("Qwen3.6-27B", {"chat_template_kwargs": {"enable_thinking": True}})
|
||||
assert eb["chat_template_kwargs"]["enable_thinking"] is True
|
||||
|
||||
|
||||
def test_non_qwen_untouched():
|
||||
assert f("adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic", None) is None
|
||||
eb = {"foo": 1}
|
||||
assert f("Hermes-3", eb) is eb # unchanged object
|
||||
|
||||
|
||||
def test_empty_model_untouched():
|
||||
assert f("", None) is None
|
||||
Loading…
Add table
Add a link
Reference in a new issue