# `aborist.document`

The data structures every source produces and every storage layer
consumes. Three core types: `Document`, `Edge`, `Chunker`.

## `Document`

A single ingest unit: a Wikipedia article, an HTML page, a Grok
conversation, a git commit message, etc. Carries both the raw
content AND the version tags that determine its identity:

```python
@dataclass(frozen=True)
class Document:
    document_uri: str           # canonical URI (or stable surrogate for non-URI sources)
    raw_content: str            # source-of-truth bytes pre-canonicalization
    kind: str                   # 'surface' / 'core' / 'visual' / etc.
    chunking_version: str       # e.g. 'tok-512-v1' — pinned by the Chunker
    canonicalization_version: str  # e.g. 'norm-v1' — pinned by canonicalize()
    schema_version: str         # e.g. 'v9.8.0' — store schema generation
    title: str | None = None
    edges: list[Edge] = ()      # outbound link graph
    metadata: dict = ...        # source-specific opaque payload
```

`document_root` is computed at ingest time as the Merkle root over
the canonicalized chunks. Two peers ingesting the same source +
running the same `chunking_version` + `canonicalization_version`
get bit-identical `document_root`s — the v9.8 admissibility property.

## `Edge`

One outbound link. `aborist/sources/wikipedia.py` emits one Edge
per `[[wikilink]]`; `aborist/sources/html_page.py` emits one per
`<a href>`. The link graph IS the corpus topology — `concepts/extract.py`
later reads `edges` rows to derive synonym relations from
reciprocal links (no separate crawler needed).

```python
@dataclass(frozen=True)
class Edge:
    src_root: str   # source document_root
    dst_uri: str    # always present
    dst_root: str   # '' (unresolved) until the dst doc is also ingested
    edge_type: str  # 'wikilink' / 'href' / 'citation' / 'derived_from' / ...
    anchor: str     # chunk index or fragment, '' if N/A
```

## `Chunker`

ABC with one method `chunk(text: str) -> list[str]`. Default impl
is `TokenChunker` (`name='tok-512-v1'`) — splits on token-rough
windows so the resulting chunks are predictable for downstream FTS5
indexing & for the LLM context budget.

**Changing the chunker bumps `chunking_version` AND stales every
prior cache record** (chunking is one of the 8 cache_key dimensions).
Don't redefine `tok-512-v1`; add a new chunker as a new `name`
instead.

## Diagrams

![module graph](../diagrams/aborist-modules.svg)
![ingest pipeline](../diagrams/ingest-pipeline.svg)

## Source

[`aborist/document.py`](../../aborist/document.py)