From 787c243179eb5ddace68a170804be49685b91b64 Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Mon, 27 Apr 2026 13:04:31 -0400 Subject: [PATCH] strict attribution rules in query system prompt MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Even with rivalry exclusion, sources like 'CPU_monthly' (a roundup discussing both AMD and Intel CPUs in the same article) sneak through because their title matches "CPU" without naming a competitor. Hermes then conflated facts: "the fastest AMD CPU is the 3.2 GHz Pentium 4." Tightened the DEFAULT_QUERY_POLICY system prompt with four explicit rules: 1. A single source may discuss multiple competitors. Attribute facts only to the entity the source explicitly names. 2. If the question asks about ONE entity (e.g., "fastest AMD CPU"), do not include competitor facts as if they were facts about the asked entity. 3. AMD PR-rating numbers (3200+, 2500+) are NOT clock speeds in MHz. Quote the source's own wording rather than interpreting model numbers. 4. Say "I don't know based on the provided sources" when the sources don't cover the answer. Don't speculate. Demo on the same AMD query (post-distill, cores now in retrieval): before: "the fastest AMD CPU is the 3.2 GHz Pentium 4 from May 2003" after: "the fastest AMD CPU mentioned is the Athlon XP 3200+ which was released in May 2003. The sources do not contain information about CPUs released after 2003." Sources include AMD, AMD_5x86, AMD_Duron + their cores, CPU_design, CPU_monthly. Hermes correctly attributes Athlon XP 3200+ to AMD and honestly bounds the corpus's coverage at 2003. Bumping the system prompt invalidates the governance_policy_hash and therefore every prior cache_key — by design, per v9.8 admissibility (prompt change = different governance = fresh inference). 66 tests still passing. --- aborist/qa/query.py | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/aborist/qa/query.py b/aborist/qa/query.py index 308d0f8..cd90706 100644 --- a/aborist/qa/query.py +++ b/aborist/qa/query.py @@ -132,10 +132,23 @@ def _filter_by_title_relevance(hits: list, question: str) -> list: DEFAULT_QUERY_POLICY = { "system_prompt": ( "You are answering a question using ONLY the sources provided below. " - "Each source is delimited by '=== Source: ===' headers. " - "Cite which source supports each claim by quoting it inline. " - "If the sources do not contain the answer, say 'I don't know based on " - "the provided sources.' Do not speculate beyond what is written." + "Each source is delimited by '=== Source: ===' headers.\n\n" + "ATTRIBUTION RULES:\n" + "1. A single source may discuss MULTIPLE products, companies, or " + "competitors. Read carefully and only attribute facts to the entity " + "the source explicitly names for that fact.\n" + "2. If the question asks about ONE specific entity (e.g., 'fastest " + "AMD CPU'), do NOT include facts about competitors (Intel, Pentium) " + "as if they were facts about the asked entity. They are different " + "products even when discussed in the same article.\n" + "3. When referencing model numbers like 'Athlon XP 3200+', remember " + "that AMD's PR-rating numbers (3200+, 2500+) are NOT the clock speed " + "in MHz. Quote the source's own wording for clock speeds rather than " + "interpreting model numbers.\n" + "4. Cite which source supports each claim by quoting it inline.\n" + "5. If the sources do not contain the answer, say 'I don't know " + "based on the provided sources.' Do not speculate beyond what the " + "sources say." ), "temperature": 0.1, "top_p": 1.0,