# arborist QA-quality benchmark question set.
#
# One question per line. `#`-prefixed lines and blank lines ignored.
# Designed for the 2003-05-16 Wikipedia cur snapshot — questions must
# resolve against ~2003 article content. Stress-tests across the
# failure-mode shapes the verifier needs to handle.

# narrow factoid — well-anchored single-fact questions
what is the capital of france?
who wrote GNU linux?
when was a programming language named Python created?
who painted the mona lisa?
a bridge between new london & groton?
who wrote the play hamlet?
what is the chemical symbol for gold?

# broad descriptive — encyclopedic shape, prone to mode-collapse
tell me about connecticut
tell me about the C programming language
tell me about method man?
tell me all there is to know about york england?
tell me about the roman empire
describe the structure of DNA

# under-specified "all" — the word "all" reads to Hermes-3-8B as
# license to enumerate every adjacent fact in training prior, which
# in claim_lattice_pointer mode degrades to free-form prose with zero
# `[E\d+]` tags (FORMAT_COLLAPSED soft-demote, 2026-05-02). Stronger
# models (Qwen / GPT-4 family) plausibly recover format discipline
# under the same prompt — bench coverage of this shape lets us
# measure cross-model resilience. Ticket #000008 (broad-quantifier
# preflight guard) proposes upstream classification + per-model
# claim ceiling.
winners of all major sports?

# bounded universals — finite, corpus-known answer sets. Ticket
# #000008 §10.1 splits broad universals into bounded vs unbounded.
# These should classify ALL but with `scope_bound_hint: "bounded"`,
# meaning --reject-broad does NOT reject and the cap is the natural
# bound. Without these fixtures, the bounded-vs-unbounded distinction
# has no live bench coverage.
name all members of the beatles
list all planets in the solar system

# entity list — invites lazy-anchor on a magnet chunk
what dinosaurs were in the first jurassic park film?
who are the members of the beatles?
name simpsons family members including pets?
list of obelisk in connecticut
what are the planets of our solar system?
who were the original seven mercury astronauts?

# relationship / multi-fact
who is supermans girlfriend?
who is bilbo baggins's nephew?
what is the relationship between linux & unix?
who is veronica ballestrini & what month was she born?
who is luke skywalker's father and sister?

# comparison — multi-entity, prone to attribution drift
what's the difference between linux and bsd?
how does intel compare to amd?
what is the difference between http and ftp?
mac vs windows for software development

# niche / partial — corpus may be thin
what is the boltzmann constant?
who invented the doppler effect?

# date / time — when-questions stress year/date grounding
when did world war 2 end?
what year did the berlin wall fall?
when was the first moon landing?
when did the soviet union dissolve?
in what year was the magna carta signed?

# quantity — numeric grounding, prone to confabulated digits
how many states are in the united states?
how many bones are in the adult human body?
what is the population of japan?
how many wives did henry the eighth have?
how many moons does jupiter have?

# geographic — where / what country / location grounding
where is mount kilimanjaro located?
what country is the city of prague in?
in which ocean is madagascar?
where does the nile river begin?
what continent is egypt on?

# cause / effect — why-questions, narrative grounding
why did the titanic sink?
what caused the chernobyl disaster?
why did the dinosaurs go extinct?
what triggered world war 1?

# entity disambiguation — common names that collide with many entities
who is michael jordan?
who is george bush?
who is john smith of jamestown?
what is the matrix?
who is paul of tarsus?

# synonym / paraphrase robustness — same fact, different phrasing.
# Pairs probe whether equivalence_class question dedup collapses these
# at write & whether retrieval grounds them identically.
who founded microsoft?
who is the founder of microsoft?
when was the eiffel tower built?
what year was the eiffel tower constructed?

# leading / forensic — probe whether retrieval surfaces the right
# fact when the question's premise contradicts the popular narrative.
# Elevation question expected answer: north (per geographic surveys);
# corpus grounding may be shaky — coverage check can pass on
# topical-token overlap without the cited span containing the
# elevation fact, so this also stress-tests lazy-anchor STRICT.
which side is the ground elevation highest throughout the span of the great wall of china, the north or south?
whales are endangered from over hunting
did napoleon really die on saint helena?

# out-of-corpus — should land UNGROUNDED honestly
who is a benevolent dictator for life for mars?
what year does our cold fusion breakthrough happen?
who won the 2024 us presidential election?
what is the latest version of the ipad pro?

# allusion / reference frame — the diagnostic signal is a verbatim
# multi-token sequence, not the individual content tokens. Pure-BM25
# on tokens will surface literal-frame articles (Oceania the region,
# Asia the continent) instead of the reference (Nineteen Eighty-Four).
# The phrase-pattern retrieval route (commit 2026-05-01) targets this
# class. Bench reveals prevalence of the failure mode + whether the
# fix generalizes beyond Orwell.
has oceania always been at war with east asia?
who said may the force be with you?
what does winter is coming mean?
what is the meaning of rosebud?
who said to be or not to be that is the question?
what does the cake is a lie reference?

# hyphenation / orthography — query and corpus use different hyphen
# conventions for the same concept. FTS5 `unicode61` splits on hyphen
# at index AND query time, so `bi-polar` (the orthography many
# English readers actually use) tokenizes to [bi, polar] while the
# Wikipedia article `Bipolar disorder` tokenizes to [bipolar]. Two
# non-overlapping token sets for the same medical condition. The
# query-layer hyphen-fold (Ticket #000007, 2026-05-02) closes this
# class by emitting joined-no-hyphen variants on the query side and
# adding a hyphen-anchor accept path in `_filter_by_title_relevance`.
# Pre-fix: only Bi-Polar (album) / Bi-Polar Blues / BI disambiguation
# surfaced; the medical-condition cluster never reached the LLM.
# Post-fix: `Bipolar disorder` lands in top-K and grounds the answer.
bi-polar is rare?
