docs(concepts): design reference + 1.6% storage-tax rationale

New `docs/concept-relations-design.md`: architecture reference for
the per-shard concept_relations layer that replaced the legacy
frozenset module (commit 5fd458a). Covers:

- Why phase 1 (hand-curated frozensets) didn't scale.
- Append-only schema + the three by-construction properties (idempotent
  re-derivation via UNIQUE, per-shard storage, Merkle-orthogonal).
- Built-in `link_reciprocity_synonym` extractor reading the existing
  `edges` table — no new crawler, works for Wikipedia AND HTML sites.
- Measured storage: 95.58 MB across 4 wiki shards (3.47M docs,
  10.75M resolved edges, 55,148 reciprocal pairs, 289,848 synonyms),
  4m16s wall-clock backfill. 1.6% tax on the 6 GB corpus.
- Three storage compactions considered & rejected, each with the
  specific trade-off it loses on (drop idx_concept_evid → painful
  purge debugging; BLOB source_root → schema inconsistency; FK
  normalization → JOIN in retrieval hot path).
- How-to: backfill, manual add, purge.
- Adding new extractors.
- Deferred follow-ons (CLI commands, Wikipedia See-also extractor,
  category extractor, hatnote extractor).

CLAUDE.md item 5 in the retrieval-pipeline list updated to point at
the new module path (aborist/concepts/) and the design doc.

TICKETS.md reference list updated to mention the new design doc.

2026-05-01 21:39:09 -04:00

3.2 KiB

Raw Blame History

Aborist tickets — index

Aborist uses a flat ticket-NNNNNN-<slug>.md convention for actionable proposals: defects with a known root cause, design proposals awaiting go/no-go, or scoped enhancements queued for future implementation.

Convention

Filename: docs/ticket-NNNNNN-<slug>.md. The ID is a six-digit zero-padded integer; the slug is hyphen-lowercase and short enough to stay readable on a 100-column terminal.
Numbering: monotonic, never reused. Pick the next free integer greater than every existing ticket. The current next ID is on the Next ID line below; bump it when you open a ticket and commit the bump in the same commit as the new ticket file.

Header: every ticket starts with

# Ticket #NNNNNN — <short title>

**Status:** open · awaiting go/no-go      (or "in progress", "closed")
**Opened:** YYYY-MM-DD
**Scope:** <one-liner — what this ticket is for>
**Audience:** <who reads this>
**Hard constraint:** <invariants this ticket commits to NOT break>

Body: problem statement, design choices with trade-offs, recommendation, implementation sketch, scope boundaries, status. See ticket-000001-retrieval-keywords-audit-gap.md for a worked template.
Closure: when a ticket lands, flip Status to closed · landed in commit <sha> and leave the file in place. Do not delete tickets; they are the design log.

Distinction from other docs

docs/ also contains architecture references and bench journey notes that are NOT tickets — they describe state of the world rather than proposing change. Examples that stay un-numbered:

cti-architecture.md (architecture reference)
mesh.md, mesh-deploy.md (architecture + ops reference)
naming-deferral.md (naming convention reference)
qa-modes-bench-2026-04-30.md (point-in-time bench journal)
verifier-semantic-gap-design.md, self-reference-thought-chains-design.md (older design docs that pre-date the ticket convention; leave as-is unless retroactively promoting one to a ticket adds value)
concept-relations-design.md (architecture reference for the per-shard concept_relations synonym/rivalry layer + the 1.6% storage-tax rationale; landed 2026-05-01)

If a doc proposes change AND awaits a decision AND has scoped implementation cost, it's a ticket. Otherwise it's reference.

Index

Newest first. Update on every open/close.

ID	Title	Status	Opened	Directive
#000005	Label ladder migration (POINTER-LINKED → …)	closed · 2026-05-02	2026-05-01	D7
#000004	Directive coverage in bench summary	closed · `acd1f9c`	2026-05-01	D8
#000003	Anchor-class warrant generalization (Module H+)	closed · 2026-05-02	2026-05-01	D6
#000002	Reference-Frame Polarity Contract (Module L)	closed · 2026-05-02	2026-05-01	D3
#000001	Retrieval-keywords audit gap	closed · 2026-05-02	2026-05-01	D4

Next ID

000006

3.2 KiB Raw Blame History