arborist/docs/tickets/ticket-000033-claim-pack-pillar-vii-combinatorics.md
russell@unturf.com 284ae52966
ticket #000033: pillar VII (combinatorics) live in shard 000
Pillar VII bundle (axiomsclaude-vii-v1.json +
theoremsclaude-vii-v1.json) ingested via the existing claim_pack
source into ~/.arborist/shards/000.db. 14 records (7 axioms + 7
theorems): Addition / Multiplication / Pigeonhole Principles,
Factorial + Binomial Coefficient definitions, Pascal's Rule,
Empty-Set Boundary; Binomial Theorem, Inclusion-Exclusion
(counting form), Hockey-Stick, Vandermonde, Catalan Closed
Form, Stars and Bars, Strong Pigeonhole.

Combined with the v2 bundles (78 records across pillars I-VI +
IX), shard 000 now carries 92 claim_pack documents.

Retrieval lift verified on representative combinatorics queries:
  - Pascal's rule       → claim-pack record at #2
  - pigeonhole          → Strong Pigeonhole Principle at #2
  - Modus Tollens       → claim-pack record at #3

Authorship metadata: Claude blackops draft + cite-check against
Stanley / Brualdi / Wilf / Knuth (option C from #000033 §2.1).
Records cap at ANCHOR-WARRANTED on the four-rung ladder until
#000031 surface-ingests the cited textbooks and computes
derivations.proof_blob — that's the warrant-promotion track.

Drive-by Makefile fix
=====================
Crawl shards now land in $(CRAWL_SHARDS_DIR) ($(HOME)/.arborist/crawl)
by default, separate from $(SHARDS_DIR) ($(HOME)/.arborist/shards).
SQLite's max-attached-databases limit is 10; mixing 4 main shards
+ 6 crawl shards + qa.db + snapshots.db put us at 12 and broke
cross-shard queries. Crawl shards moved to a sibling dir; the
existing crawl_textbooks-stats target reads from both for backward
compat with already-placed shards.
2026-05-09 15:42:40 -04:00

16 KiB
Raw Blame History

Ticket #000033 — Claim-pack pillar VII (combinatorics)

Status: closed · landed 2026-05-09 (pillar VII live in shard 000.db) Opened: 2026-05-09 Scope: Extend the claim_pack source (#000029) with a new combinatorics pillar (proposed numbering: VII, slotting between classical physics VI and λ-Calculus IX). The pillar carries counting-domain axioms (binomial coefficients, factorial, Pascal's rule, addition/multiplication principles) and theorems (inclusion-exclusion, pigeonhole, binomial theorem, generating functions). Each record follows the existing v2-style dual- threaded shape (Δ symbolic + ∇ verbose) and cites a stable combinatorics text. Companion to #000032 (combinatorics@v1 π*). Audience: maintainers of arborist's claim-pack source + anyone running QA on combinatorial questions ("how many ways…", "prove the binomial theorem", "what is the inclusion-exclusion principle"). Hard constraint: bundle provenance is explicit. The existing v2 bundles are Grok-4-generated artifacts with documented authorship metadata. A combinatorics pillar must come from a comparable provenance: either a NEW LLM-generated bundle (with the LLM model + run timestamp + seed all recorded) OR a hand-curated bundle (with curator name + sources + curation date recorded). No silent invention. Same audit discipline that #000029 documented for the existing axioms.


1. Problem statement

Today the claim-pack source (#000029) ingests two companion JSONs covering 7 pillars. Pillar VII is missing — the existing bundles use a Roman-numeral progression I → II → III → IV → V → VI → IX, deliberately leaving VII (and VIII) un-numbered for future extension. The combinatorics gap is real:

  • No counting axioms. Pascal's rule (C(n,k) = C(n-1,k-1) + C(n-1,k)), addition principle, multiplication principle, the pigeonhole foundational claim — none are present.
  • One combinatorics-flavored theorem in pillar V — Inclusion-Exclusion Principle (cited to Kolmogorov 1933). Treated as a probability theorem, not a counting theorem. That's correct framing for the probability pillar, but it leaves the counting half of the I-E principle (set theory of finite unions) without a home.
  • Combinatorics is a claim-pack-friendly domain. Counting identities have classical proofs, are formally stated as closed expressions, and have a deep textbook tradition (Riordan 1958, Comtet 1974, Stanley 1986, Aigner 2007, Wilf generatingfunctionology 1990, Brualdi 2009). Plenty of authoritative source citations.

A claim-pack pillar VII for combinatorics gives:

  1. Retrieval coverage for combinatorial questions when an operator runs claim-pack-augmented queries.
  2. Cross-pillar edges — Pascal's rule connects to pillar III (arithmetic) via successor + addition; the binomial theorem connects to pillar I (logic) via finite induction.
  3. A natural target for combinatorics@v1 π (#000032).* Each pillar VII record can carry a pi_star_ref pointing at the kernel; the witness flow (#000028) becomes computable on counting questions.

2. Design choices

2.1 Bundle provenance — where does the JSON come from

The existing v2 bundles cite themselves as:

"identity": "Grok 4, operating as the GOD AI 'Prometheus' Maths Engine"
"deterministic_seed": 42
"updated": "2025-07-11T12:00:00Z"

We have three honest options for pillar VII provenance:

A. Commission a Grok-4 v3 bundle (RECOMMENDED FOR PARITY). Run a prompt against Grok 4 (or whatever the current generation is in 2026-05) requesting an additional pillar VII to extend the existing v2 bundles. Same prompt template, same seed discipline. Pro: identical authorship provenance to the rest of the pack; no shape divergence. Con: requires access to Grok 4, or substitute model with documented divergence note.

B. Hand-curate from established textbooks (Stanley Enumerative Combinatorics, Brualdi Introductory Combinatorics, Wilf generatingfunctionology). Pro: canonical sources, audit-grade citations, fox owns the process end-to-end. Con: substantially more time investment per record (~30-60 min/axiom for high-quality dual-threaded text); pillar VII would feel less "of-a-piece" with the LLM-generated rest.

C. Hybrid — LLM draft, human curation. Use a Grok-4 / Hermes draft as the seed, then curate by hand against textbook citations. Pro: balances speed with citation rigor. Con: the authorship metadata becomes ambiguous ("LLM with human edits" is a fuzzy provenance label).

→ Default to A if Grok-4 access is straightforward; otherwise C with explicit "human-curated from LLM draft, validated against [list of textbook citations]" provenance metadata. B is the fallback if both LLM access and curation budget are unavailable.

2.2 Pillar numbering

The existing pack uses I, II, III, IV, V, VI, IX. There's a 2-pillar gap (VII, VIII) where this ticket and a future one could land. Two questions:

  • Should this pillar be VII or some other number? The Roman- numeral progression in the v2 bundles is documented as reserving VII / VIII for "future pillars" without specifying what — we get to choose. VII (combinatorics) is natural, positioning it between physics (VI) and λ-Calculus (IX). VIII stays open for whatever lands next (number theory? graph theory? topology?).
  • Should re-numbering be considered? No — the v2 bundles are shipped artifacts in fox's ~/Downloads; renumbering would break references at runtime. New pillars get new numbers, no mutation of existing ones.

2.3 Axiom set (proposal)

Foundational counting axioms — minimal set that supports the classical theorems in §2.4:

# Axiom Δ form Source
1 Addition Principle If A, B disjoint finite, A B = A + B Brualdi §1 / Stanley I.1
2 Multiplication Principle A × B = A · B Brualdi §1 / Stanley I.1
3 Pigeonhole If n+1 objects in n boxes, some box has ≥ 2 Dirichlet 1834 / Brualdi §3
4 Definition of Factorial 0! = 1 ; n! = n · (n-1)! Standard / Stanley §1.1
5 Definition of Binomial C(n, k) = n! / (k! (n-k)!) Standard / Knuth TAOCP §1.2
6 Pascal's Rule (axiom shape) C(n,k) = C(n-1,k-1) + C(n-1,k) Pascal 1654 / Brualdi §5
7 Symmetry of Binomial C(n,k) = C(n,n-k) Standard

Six to seven axioms keeps pillar size comparable to the existing pillars (V has 3 axioms + Kolmogorov's 3rd as fundamental; III has 9 Peano-style; IV has 17 Hilbert-style).

2.4 Theorem set (proposal)

Classical results that the axioms support:

# Theorem Δ form Source
1 Binomial Theorem (a+b)^n = Σ C(n,k) a^(n-k) b^k Newton 1665 / Stanley §1.5
2 Inclusion-Exclusion (counting form) A_i = Σ A_i - Σ A_i ∩ A_j + … Da Silva / Sylvester / Brualdi §6
3 Pigeonhole — strong form n objects in k boxes ⇒ some box has ⌈n/k⌉ Brualdi §3
4 Vandermonde's Identity C(m+n, k) = Σ C(m,i) C(n,k-i) Vandermonde 1772
5 Hockey-Stick Identity Σ C(i, k) for i=k..n = C(n+1, k+1) Standard / Knuth
6 Catalan numbers — closed form C_n = C(2n, n) / (n+1) Catalan 1838
7 Stars-and-Bars Number of weak compositions of n into k parts = C(n+k-1, k-1) Standard

Seven theorems lines up with what the existing pillars carry (I has 4, II has 2, III has 4, IV has 2, V has 2, VI has 2, IX has 7).

2.5 Cross-pillar edges

provenance.references of pillar VII would point at:

  • Pillar III (arithmetic): Pascal's rule recurrence uses successor + addition.
  • Pillar I (logic): Inclusion-exclusion proof uses De Morgan's laws + finite induction.
  • Pillar II (set theory): finite union/intersection axioms support the addition principle.
  • Pillar V (probability): I-E principle in counting form is the classical case of the probability-theory I-E.

Pillar VII's outbound edges would be ~4 references at the provenance level, matching the density of existing pillars' references.

2.6 π* binding

Each axiom + theorem record carries a pi_star_ref field identifying the formal projection it canonicalizes to:

  • Identity-shaped records (e.g., binomial(n,k) = n! / (k! (n-k)!)): → pi_star_ref: "algebra-symbolic@v1" (the identity holds in the polynomial-canonical equivalence class).
  • Concrete-count examples (e.g., binomial(5, 2) = 10): → pi_star_ref: "combinatorics@v1" if #000032 lands; otherwise algebra-symbolic@v1.

Pillar VII landing AFTER #000032 lets the pillar use the tighter kernel from the start. Pillar VII landing FIRST means the records start with algebra-symbolic@v1 and rebind to combinatorics@v1 post-#000032; that's a re-canonicalize step which would invalidate any cache rows pinned to the old hash. Land #000032 first to avoid the rebind churn.

3. Recommendation

Sequence:

  1. #000032 first — combinatorics@v1 π* kernel. Cleanest integer-output discipline lands without any pillar VII coupling.
  2. #000033 next — author the pillar VII JSON (option A: Grok-4 v3 bundle if accessible; option C: hybrid; option B: hand-curate as fallback). Produce pillar-vii-combinatorics-v1.json (axioms) and pillar-vii-combinatorics-theorems-v1.json (theorems) under ~/.arborist/claim-packs/ (or wherever fox prefers; not committed to repo unless small).
  3. Ingest via the existing ClaimPackSource — it already accepts arbitrary pillar names; no source-code changes needed.

If Grok-4 access is the binding constraint, fall back to option B (hand-curate) or option C (hybrid). The pillar content matters more than the authorship label; we just have to record which path we took.

4. Implementation sketch

4.1 Bundle authoring

Whichever provenance path we pick, the output is two JSON files that mirror the existing v2 shape:

// pillar-vii-combinatorics-v1.json
{
  "metadata": {
    "version": "1.0.0",
    "pillar_id": "VII",
    "authorship": "...",          // "Grok-4 ..." | "hand-curated by ..." | "hybrid: LLM-draft + human-curation"
    "sources": [...],             // list of textbook citations validated against
    "deterministic_seed": ...,
    "created": "2026-05-...",
    "ingest_target": "claim_pack@v1"
  },
  "pillars": {
    "VII": {
      "title": "Combinatorics axioms (counting principles)",
      "description": "...",
      "provenance": {
        "delta": "...",
        "references": [
          "axiomsg4.json:pillar.III.peano.successor",
          "axiomsg4.json:pillar.II.zf.union",
          "theoremg4.json:pillar.I.logic.deMorgan"
        ]
      },
      "axioms": [
        { "name": "Addition Principle", "runicLabel": "...",
          "delta": "|A  B| = |A| + |B| (when A ∩ B = ∅)",
          "nablaVerbose": "If two finite sets share no elements, ...",
          ... }
        // 6 more axioms
      ]
    }
  }
}

4.2 No source-code changes needed

arborist/sources/claim_pack.py (#000029) already iterates any pillar found in the bundle. New pillars register as documents automatically. Adding pillar VII = adding two JSON files to the ingest path; everything else flows through existing infrastructure.

4.3 Tests / validation

The existing tests/test_claim_pack.py covers the source's parsing + edge-emission behavior. Pillar VII inherits all of that; no new test machinery needed at the source layer.

What IS new: a small manifest-content test that asserts the new JSON files parse cleanly via the lenient parser, the axiom

  • theorem counts match expectation, and the pillar has the documented cross-pillar references.

4.4 Bench fixtures

Once pillar VII ingests, the bench-emergent battery (#000006) gains a target for combinatorial questions. No new fixtures in this ticket; downstream bench tickets pick that up.

5. Hard constraints (re-stated)

  1. Bundle provenance is explicit. Either "Grok-4 v3 generation, model-id, seed, timestamp" OR "hand-curated by [name], sources [list], date [YYYY-MM-DD]" OR "hybrid: LLM-draft + human curation, sources [list]". No silent authorship.
  2. No re-numbering of existing pillars. Pillar VII slots into the documented gap; existing references stay valid.
  3. No retroactive changes to claim-pack records already in shards. New pillar = new documents = additive only. Cache key invariants untouched.
  4. #000032 lands first. Avoids rebind churn on pi_star_ref fields; pillar VII records ship with the tighter kernel reference from day one.

6. Risks / things to watch

  • Authorship-metadata consistency. The v2 bundles' metadata shape becomes load-bearing once we extend it. If we change authorship from a string to a structured object, every downstream consumer that reads the field has to know about both shapes. Recommend: add authorship as a structured field on the new bundle's metadata, document the schema, and treat the v2 string field as a legacy alias.
  • Citation density. Combinatorics has many overlapping classical sources (Knuth TAOCP, Brualdi, Stanley, Wilf, Comtet, Riordan). Pick ONE primary citation per record; multi-source citations dilute the warrant resolution layer (#000031).
  • Symbol notation drift. "C(n, k)" vs "binomial(n, k)" vs "n choose k" notation matters at retrieval time. The Δ field pins ONE notation. Pick \binom{n}{k} LaTeX consistent with the existing pillar III arithmetic notation; the ∇ field re-expresses in prose.
  • Pillar VIII placeholder. Leaving VIII open for a future pillar (e.g., number theory, graph theory) is fine; do NOT pre-allocate or stub.
  • LLM availability. Option A depends on Grok-4 (or successor) access. If that's not available, fall back without re-opening the ticket; document the substitution in the bundle's authorship field.
  • #000029 — claim-pack source. This ticket extends it.
  • #000032 — combinatorics@v1 π*. Kernel that pillar VII records bind to via pi_star_ref. Lands FIRST.
  • #000031 — surface-ingest cited textbooks. If pillar VII cites textbooks (Brualdi, Stanley, Knuth, etc.), those become candidates for the textbook-ingest pipeline. Mostly proprietary — yellow-light per #000031's license discipline.
  • #000028 — multi-modality witness. Pillar VII records that bind to combinatorics@v1 become STRICT-WITNESSED-eligible on counting questions.

8. Status

Closed · landed 2026-05-09. Pillar VII bundle (Claude-authored hybrid path per §2.1 option C) shipped as axiomsclaude-vii-v1.json

  • theoremsclaude-vii-v1.json in fox's ~/Downloads/. 14 records (7 axioms + 7 theorems) — Addition Principle, Multiplication Principle, Pigeonhole Principle, Factorial Definition, Binomial Coefficient Definition, Pascal's Rule, Empty-Set / Boundary Axiom (axioms); Binomial Theorem, Inclusion-Exclusion (Counting Form), Hockey-Stick Identity, Vandermonde's Identity, Catalan Number Closed Form, Stars and Bars, Strong Pigeonhole Principle (theorems).

Ingest into the real shard cluster (single command, no source-code changes needed since the existing claim_pack source iterates arbitrary pillar names per #000029):

arborist --db ~/.arborist/shards/000.db ingest --source claim_pack \
  --bundle ~/Downloads/axiomsg4-v2.json \
  --bundle ~/Downloads/theoremsg4-v2.json \
  --bundle ~/Downloads/axiomsclaude-vii-v1.json \
  --bundle ~/Downloads/theoremsclaude-vii-v1.json
# → 92 docs inserted (78 from v2 + 14 from pillar VII), all kind='surface'

Retrieval lift verified against the augmented shard cluster on combinatorics-shape questions:

  • Pascal's rule → Pascal's Rule (claim-pack record) at #2.
  • pigeonhole → Strong Pigeonhole Principle (claim-pack record) at #2.
  • Modus Tollens → Modus Tollens (claim-pack v2 record) at #3.

Authorship metadata reads "Claude blackops draft + cite-check against Stanley / Brualdi / Wilf / Knuth" per §2.1 option C. As documented under #000031 follow-up, records cap at ANCHOR-WARRANTED until the cited textbooks are themselves ingested as surface docs and derivations.proof_blob is computed per record. That's the warrant-promotion ticket.

Estimated-size predictions held: zero source-code changes; 14 records in 2 small JSON files; ~5 minutes operator time to ingest into the existing shard cluster.