capital: land ticket #000020 (8-capital-form cost ledger)
CapitalProfile (8 forms: living, material, financial, intellectual,
experiential, social, cultural, spiritual) attached per state-changing
op as a sibling-table row in capital_ledger. Sibling semantics: ledger
rows reference an audit_event_hash but do NOT enter the audit-event
preimage, so retroactive cost re-estimation cannot break the chain.
Surface:
- arborist.capital.{profile,store}
- profile_for_op dispatch with per-op estimators (ingest/qa/distill)
- record/summary/op_cost/top_by_form
- CLI: arborist capital summary|op-cost|top
Wire-up at three op sites:
- ingest.py — one row per batch (doc_count + total_bytes)
- qa/runner.py — one row per cache-miss (answer_chars + llm_seconds)
- distill/runner.py — one row per derivation (positive intellectual)
Estimator constants are heuristic v1 (ESTIMATOR_VERSION pin in the
schema). Re-estimation is supported by re-running estimators against
the recorded inputs_blob and writing a new row with a bumped version
pin; old rows stay queryable.
Tests: tests/test_capital.py (13 cases). Sibling-table invariant
verified: audit chain stays intact across capital writes.
Full suite: 1025 passed, 36 skipped.
This commit is contained in:
parent
a9fdcf41d5
commit
69f91d39a6
10 changed files with 853 additions and 10 deletions
53
arborist/capital/__init__.py
Normal file
53
arborist/capital/__init__.py
Normal file
|
|
@ -0,0 +1,53 @@
|
||||||
|
"""Capital-cost ledger: tying arborist ops to the 8 forms of capital.
|
||||||
|
|
||||||
|
Implements ticket #000020. Sibling table to ``audit_events`` —
|
||||||
|
``capital_ledger`` rows do NOT enter ``audit_events.event_hash``
|
||||||
|
preimage, so retroactive cost re-estimation cannot break the audit
|
||||||
|
chain.
|
||||||
|
|
||||||
|
The 8 forms (Roland & Landua via unturf.com/eight-forms-of-capital):
|
||||||
|
|
||||||
|
1. **Living** — soil, water, human health, ecosystems
|
||||||
|
2. **Material** — servers, tools, infrastructure
|
||||||
|
3. **Financial** — money, currencies, securities
|
||||||
|
4. **Intellectual** — ideas, knowledge, open source
|
||||||
|
5. **Experiential** — embodied skill, mastery, know-how
|
||||||
|
6. **Social** — trust, relationships, community networks
|
||||||
|
7. **Cultural** — story, art, ceremony, shared ethics
|
||||||
|
8. **Spiritual** — presence, connection to a greater whole
|
||||||
|
|
||||||
|
Each :class:`CapitalProfile` records contributions / debits across all
|
||||||
|
8 forms in their native units (no conversion). Profiles are advisory
|
||||||
|
by default — passing ``capital_profile=None`` to ``record`` emits no
|
||||||
|
ledger row, fully backward-compatible.
|
||||||
|
|
||||||
|
Public surface:
|
||||||
|
|
||||||
|
- :class:`CapitalProfile` — frozen dataclass; one float per form.
|
||||||
|
- :func:`record` — write a ledger row tied to an audit_event_hash.
|
||||||
|
- :func:`summary` / :func:`top_by_form` / :func:`op_cost` — read APIs.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from arborist.capital.profile import (
|
||||||
|
ESTIMATOR_VERSION,
|
||||||
|
CapitalProfile,
|
||||||
|
profile_for_op,
|
||||||
|
)
|
||||||
|
from arborist.capital.store import (
|
||||||
|
op_cost,
|
||||||
|
record,
|
||||||
|
summary,
|
||||||
|
top_by_form,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"ESTIMATOR_VERSION",
|
||||||
|
"CapitalProfile",
|
||||||
|
"profile_for_op",
|
||||||
|
"record",
|
||||||
|
"summary",
|
||||||
|
"top_by_form",
|
||||||
|
"op_cost",
|
||||||
|
]
|
||||||
181
arborist/capital/profile.py
Normal file
181
arborist/capital/profile.py
Normal file
|
|
@ -0,0 +1,181 @@
|
||||||
|
"""CapitalProfile dataclass + estimator dispatch.
|
||||||
|
|
||||||
|
Estimator versioning rule (per ticket #000020 §6 — risks):
|
||||||
|
|
||||||
|
- ``ESTIMATOR_VERSION`` is the global pin. Bump it when the
|
||||||
|
estimator function set changes. Old rows keep their version
|
||||||
|
recorded so retroactive re-estimation can be rebuilt cleanly.
|
||||||
|
- Per-op-type estimators are registered in :data:`ESTIMATORS`;
|
||||||
|
unknown op types fall through to a zero-profile + the version
|
||||||
|
pin.
|
||||||
|
|
||||||
|
Cost-unit conventions (no cross-form conversion at this layer):
|
||||||
|
|
||||||
|
- ``living`` — person-minutes equivalent (operator + downstream user
|
||||||
|
attention).
|
||||||
|
- ``material`` — kWh equivalent for compute + storage (rough).
|
||||||
|
- ``financial`` — USD equivalent for paid resources (e.g., LLM API
|
||||||
|
spend).
|
||||||
|
- ``intellectual`` — net contribution (positive when op produces
|
||||||
|
reusable knowledge artifacts; negative when op consumes them
|
||||||
|
without preservation).
|
||||||
|
- ``experiential`` — operator wait-time minutes.
|
||||||
|
- ``social`` — trust delta on a ±1 named scale.
|
||||||
|
- ``cultural`` — analogous ±1.
|
||||||
|
- ``spiritual`` — analogous ±1.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import asdict, dataclass, field, replace
|
||||||
|
from typing import Callable, Optional
|
||||||
|
|
||||||
|
|
||||||
|
ESTIMATOR_VERSION = "estimator-v1"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class CapitalProfile:
|
||||||
|
living: float = 0.0
|
||||||
|
material: float = 0.0
|
||||||
|
financial: float = 0.0
|
||||||
|
intellectual: float = 0.0
|
||||||
|
experiential: float = 0.0
|
||||||
|
social: float = 0.0
|
||||||
|
cultural: float = 0.0
|
||||||
|
spiritual: float = 0.0
|
||||||
|
|
||||||
|
def as_dict(self) -> dict:
|
||||||
|
return asdict(self)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def zero(cls) -> "CapitalProfile":
|
||||||
|
return cls()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------
|
||||||
|
# Estimator dispatch
|
||||||
|
#
|
||||||
|
# Each estimator returns a (CapitalProfile, inputs_dict) pair. The
|
||||||
|
# inputs_dict goes to capital_ledger.estimator_inputs_blob so retroactive
|
||||||
|
# re-estimation can be reproduced exactly.
|
||||||
|
# ---------------------------------------------------------------------
|
||||||
|
|
||||||
|
Estimator = Callable[[dict], tuple[CapitalProfile, dict]]
|
||||||
|
|
||||||
|
|
||||||
|
def _estimate_ingest(args: dict) -> tuple[CapitalProfile, dict]:
|
||||||
|
"""Cost of ingesting documents.
|
||||||
|
|
||||||
|
Inputs (all optional; missing keys default to 0):
|
||||||
|
- ``doc_count`` — number of documents ingested
|
||||||
|
- ``total_bytes`` — uncompressed corpus size
|
||||||
|
|
||||||
|
Heuristic constants (initial v1 — refine empirically):
|
||||||
|
- 1e-6 kWh per ingested byte (storage write + index)
|
||||||
|
- 1e-9 person-minutes per byte (sysadmin tax, amortized)
|
||||||
|
- +0.001 intellectual per doc (positive: knowledge preserved)
|
||||||
|
"""
|
||||||
|
doc_count = float(args.get("doc_count", 0))
|
||||||
|
total_bytes = float(args.get("total_bytes", 0))
|
||||||
|
inputs = {"doc_count": doc_count, "total_bytes": total_bytes}
|
||||||
|
return (
|
||||||
|
CapitalProfile(
|
||||||
|
material=total_bytes * 1e-6,
|
||||||
|
living=total_bytes * 1e-9,
|
||||||
|
intellectual=doc_count * 0.001,
|
||||||
|
),
|
||||||
|
inputs,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _estimate_qa(args: dict) -> tuple[CapitalProfile, dict]:
|
||||||
|
"""Cost of answering one query.
|
||||||
|
|
||||||
|
Inputs:
|
||||||
|
- ``prompt_chars`` — total prompt size in chars
|
||||||
|
- ``answer_chars`` — answer size in chars
|
||||||
|
- ``llm_seconds`` — wall-clock seconds spent in LLM call
|
||||||
|
- ``cache_hit`` — bool; cache hits are nearly free
|
||||||
|
|
||||||
|
Heuristics (LLM dominates):
|
||||||
|
- Cache hit: nearly zero except 0.01 experiential (operator wait)
|
||||||
|
- LLM call: 1e-3 kWh/sec (small model on commodity GPU)
|
||||||
|
- LLM call: $1e-5 / 1k chars (rough self-hosted equivalent)
|
||||||
|
- Answer chars: -0.0001 intellectual per char (consuming context)
|
||||||
|
"""
|
||||||
|
cache_hit = bool(args.get("cache_hit", False))
|
||||||
|
prompt_chars = float(args.get("prompt_chars", 0))
|
||||||
|
answer_chars = float(args.get("answer_chars", 0))
|
||||||
|
llm_seconds = float(args.get("llm_seconds", 0.0))
|
||||||
|
inputs = {
|
||||||
|
"cache_hit": cache_hit,
|
||||||
|
"prompt_chars": prompt_chars,
|
||||||
|
"answer_chars": answer_chars,
|
||||||
|
"llm_seconds": llm_seconds,
|
||||||
|
}
|
||||||
|
if cache_hit:
|
||||||
|
return CapitalProfile(experiential=0.01), inputs
|
||||||
|
return (
|
||||||
|
CapitalProfile(
|
||||||
|
material=llm_seconds * 1e-3,
|
||||||
|
financial=(prompt_chars + answer_chars) * 1e-5 / 1000.0,
|
||||||
|
experiential=llm_seconds / 60.0,
|
||||||
|
intellectual=-answer_chars * 0.0001,
|
||||||
|
),
|
||||||
|
inputs,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _estimate_distill(args: dict) -> tuple[CapitalProfile, dict]:
|
||||||
|
"""Cost of distilling surface documents into core documents.
|
||||||
|
|
||||||
|
Inputs:
|
||||||
|
- ``surface_count`` — number of surface docs distilled
|
||||||
|
- ``core_count`` — number of core docs produced
|
||||||
|
- ``llm_seconds`` — wall-clock seconds spent in LLM call (if any)
|
||||||
|
|
||||||
|
Distillation produces reusable knowledge artifacts (cores), so
|
||||||
|
intellectual capital is positive.
|
||||||
|
"""
|
||||||
|
surface_count = float(args.get("surface_count", 0))
|
||||||
|
core_count = float(args.get("core_count", 0))
|
||||||
|
llm_seconds = float(args.get("llm_seconds", 0.0))
|
||||||
|
inputs = {
|
||||||
|
"surface_count": surface_count,
|
||||||
|
"core_count": core_count,
|
||||||
|
"llm_seconds": llm_seconds,
|
||||||
|
}
|
||||||
|
return (
|
||||||
|
CapitalProfile(
|
||||||
|
material=llm_seconds * 1e-3,
|
||||||
|
experiential=llm_seconds / 60.0,
|
||||||
|
intellectual=core_count * 0.005,
|
||||||
|
),
|
||||||
|
inputs,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _estimate_default(args: dict) -> tuple[CapitalProfile, dict]:
|
||||||
|
"""Fallback: zero profile + the inputs dict for record-keeping."""
|
||||||
|
return CapitalProfile.zero(), dict(args)
|
||||||
|
|
||||||
|
|
||||||
|
ESTIMATORS: dict[str, Estimator] = {
|
||||||
|
"ingest": _estimate_ingest,
|
||||||
|
"qa": _estimate_qa,
|
||||||
|
"distill": _estimate_distill,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def profile_for_op(
|
||||||
|
op_type: str, inputs: Optional[dict] = None
|
||||||
|
) -> tuple[CapitalProfile, dict]:
|
||||||
|
"""Dispatch to the right estimator; fallback to zero profile.
|
||||||
|
|
||||||
|
Returns ``(profile, recorded_inputs)``. The recorded inputs are
|
||||||
|
written to ``capital_ledger.estimator_inputs_blob`` for replay.
|
||||||
|
"""
|
||||||
|
inputs = dict(inputs or {})
|
||||||
|
estimator = ESTIMATORS.get(op_type, _estimate_default)
|
||||||
|
return estimator(inputs)
|
||||||
142
arborist/capital/store.py
Normal file
142
arborist/capital/store.py
Normal file
|
|
@ -0,0 +1,142 @@
|
||||||
|
"""CRUD over ``capital_ledger``.
|
||||||
|
|
||||||
|
Sibling table semantics: ``capital_ledger`` rows reference an
|
||||||
|
``audit_event_hash`` but do NOT enter the audit-event preimage. Rows
|
||||||
|
can be re-estimated retroactively without invalidating any chain.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import sqlite3
|
||||||
|
import time
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
from arborist.capital.profile import (
|
||||||
|
ESTIMATOR_VERSION,
|
||||||
|
CapitalProfile,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
_FORMS = (
|
||||||
|
"living",
|
||||||
|
"material",
|
||||||
|
"financial",
|
||||||
|
"intellectual",
|
||||||
|
"experiential",
|
||||||
|
"social",
|
||||||
|
"cultural",
|
||||||
|
"spiritual",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def record(
|
||||||
|
conn: sqlite3.Connection,
|
||||||
|
*,
|
||||||
|
audit_event_hash: str,
|
||||||
|
op_type: str,
|
||||||
|
profile: CapitalProfile,
|
||||||
|
estimator_inputs: Optional[dict] = None,
|
||||||
|
estimator_version: str = ESTIMATOR_VERSION,
|
||||||
|
ts: Optional[int] = None,
|
||||||
|
) -> int:
|
||||||
|
"""Insert one ledger row; return ledger_id.
|
||||||
|
|
||||||
|
Idempotent on ``audit_event_hash`` only at the row level — i.e.,
|
||||||
|
we permit multiple ledger rows per audit event (e.g., one per
|
||||||
|
estimator version when re-estimating). Callers needing strict
|
||||||
|
idempotency should check first.
|
||||||
|
"""
|
||||||
|
if ts is None:
|
||||||
|
ts = int(time.time())
|
||||||
|
inputs_blob = (
|
||||||
|
None
|
||||||
|
if estimator_inputs is None
|
||||||
|
else json.dumps(
|
||||||
|
estimator_inputs,
|
||||||
|
sort_keys=True,
|
||||||
|
separators=(",", ":"),
|
||||||
|
ensure_ascii=False,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
cursor = conn.execute(
|
||||||
|
"INSERT INTO capital_ledger ("
|
||||||
|
" audit_event_hash, op_type, living, material, financial,"
|
||||||
|
" intellectual, experiential, social, cultural, spiritual,"
|
||||||
|
" estimator_version, estimator_inputs_blob, recorded_at"
|
||||||
|
") VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
|
||||||
|
(
|
||||||
|
audit_event_hash,
|
||||||
|
op_type,
|
||||||
|
profile.living,
|
||||||
|
profile.material,
|
||||||
|
profile.financial,
|
||||||
|
profile.intellectual,
|
||||||
|
profile.experiential,
|
||||||
|
profile.social,
|
||||||
|
profile.cultural,
|
||||||
|
profile.spiritual,
|
||||||
|
estimator_version,
|
||||||
|
inputs_blob,
|
||||||
|
ts,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return int(cursor.lastrowid)
|
||||||
|
|
||||||
|
|
||||||
|
def summary(
|
||||||
|
conn: sqlite3.Connection,
|
||||||
|
*,
|
||||||
|
since: Optional[int] = None,
|
||||||
|
op_type: Optional[str] = None,
|
||||||
|
) -> dict:
|
||||||
|
"""Aggregate totals across ledger; per-form sums + row count."""
|
||||||
|
where = []
|
||||||
|
params: list = []
|
||||||
|
if since is not None:
|
||||||
|
where.append("recorded_at >= ?")
|
||||||
|
params.append(int(since))
|
||||||
|
if op_type is not None:
|
||||||
|
where.append("op_type = ?")
|
||||||
|
params.append(op_type)
|
||||||
|
where_sql = ("WHERE " + " AND ".join(where)) if where else ""
|
||||||
|
sums_sql = ", ".join(f"SUM({f}) AS {f}" for f in _FORMS)
|
||||||
|
sql = (
|
||||||
|
f"SELECT COUNT(*) AS row_count, {sums_sql} "
|
||||||
|
f"FROM capital_ledger {where_sql}"
|
||||||
|
)
|
||||||
|
row = conn.execute(sql, params).fetchone()
|
||||||
|
if row is None:
|
||||||
|
return {"row_count": 0, **{f: 0.0 for f in _FORMS}}
|
||||||
|
out = {"row_count": int(row["row_count"])}
|
||||||
|
for f in _FORMS:
|
||||||
|
v = row[f]
|
||||||
|
out[f] = 0.0 if v is None else float(v)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def top_by_form(
|
||||||
|
conn: sqlite3.Connection,
|
||||||
|
form: str,
|
||||||
|
*,
|
||||||
|
limit: int = 10,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Return top-N op_types by total contribution to a single form."""
|
||||||
|
if form not in _FORMS:
|
||||||
|
raise ValueError(
|
||||||
|
f"unknown capital form: {form!r}; expected one of {_FORMS}"
|
||||||
|
)
|
||||||
|
rows = conn.execute(
|
||||||
|
f"SELECT op_type, SUM({form}) AS total_{form}, COUNT(*) AS row_count "
|
||||||
|
"FROM capital_ledger "
|
||||||
|
"GROUP BY op_type "
|
||||||
|
f"ORDER BY total_{form} DESC "
|
||||||
|
"LIMIT ?",
|
||||||
|
(int(limit),),
|
||||||
|
).fetchall()
|
||||||
|
return [dict(r) for r in rows]
|
||||||
|
|
||||||
|
|
||||||
|
def op_cost(conn: sqlite3.Connection, op_type: str) -> dict:
|
||||||
|
"""Aggregate per-form totals for one op_type."""
|
||||||
|
return summary(conn, op_type=op_type)
|
||||||
|
|
@ -2721,6 +2721,45 @@ def _cmd_snapshot_diff(args: argparse.Namespace) -> int:
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def _cmd_capital_summary(args: argparse.Namespace) -> int:
|
||||||
|
"""Aggregate capital_ledger totals; per-form sums + row count."""
|
||||||
|
from arborist.capital import summary as capital_summary
|
||||||
|
|
||||||
|
conn = connect(args.db)
|
||||||
|
try:
|
||||||
|
out = capital_summary(conn, op_type=args.op_type, since=args.since)
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
print(json.dumps(out, indent=2, ensure_ascii=False))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def _cmd_capital_op_cost(args: argparse.Namespace) -> int:
|
||||||
|
"""Per-form totals for one op_type."""
|
||||||
|
from arborist.capital import op_cost
|
||||||
|
|
||||||
|
conn = connect(args.db)
|
||||||
|
try:
|
||||||
|
out = op_cost(conn, args.op_type)
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
print(json.dumps(out, indent=2, ensure_ascii=False))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def _cmd_capital_top(args: argparse.Namespace) -> int:
|
||||||
|
"""Top-N op_types by total contribution to a single capital form."""
|
||||||
|
from arborist.capital import top_by_form
|
||||||
|
|
||||||
|
conn = connect(args.db)
|
||||||
|
try:
|
||||||
|
out = top_by_form(conn, args.form, limit=args.limit)
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
print(json.dumps(out, indent=2, ensure_ascii=False))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
def _cmd_selfmodel_snapshot(args: argparse.Namespace) -> int:
|
def _cmd_selfmodel_snapshot(args: argparse.Namespace) -> int:
|
||||||
"""Build a SelfModel from current store state and persist it.
|
"""Build a SelfModel from current store state and persist it.
|
||||||
|
|
||||||
|
|
@ -4142,6 +4181,59 @@ def build_parser() -> argparse.ArgumentParser:
|
||||||
snap_diff.add_argument("snapshot_root", help="hex snapshot_root to diff against current")
|
snap_diff.add_argument("snapshot_root", help="hex snapshot_root to diff against current")
|
||||||
snap_diff.set_defaults(func=_cmd_snapshot_diff)
|
snap_diff.set_defaults(func=_cmd_snapshot_diff)
|
||||||
|
|
||||||
|
# ----- capital subcommands (ticket #000020) -------------------------------
|
||||||
|
capital_cmd = sub.add_parser(
|
||||||
|
"capital",
|
||||||
|
help="8-capital-form cost ledger (ticket #000020)",
|
||||||
|
)
|
||||||
|
capital_sub = capital_cmd.add_subparsers(
|
||||||
|
dest="capital_op", required=True
|
||||||
|
)
|
||||||
|
|
||||||
|
cap_summary = capital_sub.add_parser(
|
||||||
|
"summary",
|
||||||
|
help="aggregate per-form sums across the ledger",
|
||||||
|
)
|
||||||
|
cap_summary.add_argument(
|
||||||
|
"--op-type",
|
||||||
|
dest="op_type",
|
||||||
|
default=None,
|
||||||
|
help="filter to one op_type",
|
||||||
|
)
|
||||||
|
cap_summary.add_argument(
|
||||||
|
"--since",
|
||||||
|
type=int,
|
||||||
|
default=None,
|
||||||
|
help="only rows with recorded_at >= this Unix timestamp",
|
||||||
|
)
|
||||||
|
cap_summary.set_defaults(func=_cmd_capital_summary)
|
||||||
|
|
||||||
|
cap_op = capital_sub.add_parser(
|
||||||
|
"op-cost", help="per-form totals for one op_type"
|
||||||
|
)
|
||||||
|
cap_op.add_argument("op_type", help="e.g. ingest|qa|distill")
|
||||||
|
cap_op.set_defaults(func=_cmd_capital_op_cost)
|
||||||
|
|
||||||
|
cap_top = capital_sub.add_parser(
|
||||||
|
"top", help="top-N op_types by total in one capital form"
|
||||||
|
)
|
||||||
|
cap_top.add_argument(
|
||||||
|
"--form",
|
||||||
|
required=True,
|
||||||
|
choices=[
|
||||||
|
"living",
|
||||||
|
"material",
|
||||||
|
"financial",
|
||||||
|
"intellectual",
|
||||||
|
"experiential",
|
||||||
|
"social",
|
||||||
|
"cultural",
|
||||||
|
"spiritual",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
cap_top.add_argument("--limit", type=int, default=10)
|
||||||
|
cap_top.set_defaults(func=_cmd_capital_top)
|
||||||
|
|
||||||
# ----- selfmodel subcommands (ticket #000014) -----------------------------
|
# ----- selfmodel subcommands (ticket #000014) -----------------------------
|
||||||
selfmodel_cmd = sub.add_parser(
|
selfmodel_cmd = sub.add_parser(
|
||||||
"selfmodel",
|
"selfmodel",
|
||||||
|
|
|
||||||
|
|
@ -278,7 +278,7 @@ def _persist_no_tx(
|
||||||
"VALUES (?, ?, ?, 'derived_from', '')",
|
"VALUES (?, ?, ?, 'derived_from', '')",
|
||||||
(p.core_root, p.src_root, p.src_uri),
|
(p.core_root, p.src_root, p.src_uri),
|
||||||
)
|
)
|
||||||
append_audit(
|
derive_event_hash = append_audit(
|
||||||
conn,
|
conn,
|
||||||
event_type="derive",
|
event_type="derive",
|
||||||
subject_root=p.core_root,
|
subject_root=p.core_root,
|
||||||
|
|
@ -297,4 +297,25 @@ def _persist_no_tx(
|
||||||
},
|
},
|
||||||
ts=now,
|
ts=now,
|
||||||
)
|
)
|
||||||
|
# Capital ledger (ticket #000020). Distillation produces reusable
|
||||||
|
# core artifacts → positive intellectual capital.
|
||||||
|
from arborist.capital import profile_for_op, record as capital_record
|
||||||
|
|
||||||
|
capital_profile, capital_inputs = profile_for_op(
|
||||||
|
"distill",
|
||||||
|
{
|
||||||
|
"surface_count": 1,
|
||||||
|
"core_count": 1,
|
||||||
|
# llm_seconds is unknown at this site; estimator falls
|
||||||
|
# through to 0.0 cleanly when missing.
|
||||||
|
},
|
||||||
|
)
|
||||||
|
capital_record(
|
||||||
|
conn,
|
||||||
|
audit_event_hash=derive_event_hash,
|
||||||
|
op_type="distill",
|
||||||
|
profile=capital_profile,
|
||||||
|
estimator_inputs=capital_inputs,
|
||||||
|
ts=now,
|
||||||
|
)
|
||||||
return "distilled"
|
return "distilled"
|
||||||
|
|
|
||||||
|
|
@ -308,6 +308,33 @@ def _flush_batch(
|
||||||
audit_rows,
|
audit_rows,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Capital ledger (ticket #000020). One row per batch attached
|
||||||
|
# to the last event_hash. Sibling table — does NOT enter
|
||||||
|
# audit_events.event_hash preimage.
|
||||||
|
from arborist.capital import profile_for_op, record as capital_record
|
||||||
|
|
||||||
|
total_chunks = sum(
|
||||||
|
ev["body"].get("chunks", 0) for ev in audit_events
|
||||||
|
)
|
||||||
|
# Heuristic: ~512 tokens × ~4 chars/token per chunk.
|
||||||
|
approx_bytes = total_chunks * 512 * 4
|
||||||
|
profile, inputs = profile_for_op(
|
||||||
|
"ingest",
|
||||||
|
{
|
||||||
|
"doc_count": len(audit_events),
|
||||||
|
"total_bytes": approx_bytes,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
last_event_hash = audit_rows[-1][0]
|
||||||
|
capital_record(
|
||||||
|
conn,
|
||||||
|
audit_event_hash=last_event_hash,
|
||||||
|
op_type="ingest",
|
||||||
|
profile=profile,
|
||||||
|
estimator_inputs=inputs,
|
||||||
|
ts=ingest_ts,
|
||||||
|
)
|
||||||
|
|
||||||
return inserted, skipped
|
return inserted, skipped
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -999,6 +999,25 @@ def ask(
|
||||||
},
|
},
|
||||||
ts=now,
|
ts=now,
|
||||||
)
|
)
|
||||||
|
# Capital ledger (ticket #000020). Sibling table; advisory.
|
||||||
|
from arborist.capital import profile_for_op, record as capital_record
|
||||||
|
|
||||||
|
capital_profile, capital_inputs = profile_for_op(
|
||||||
|
"qa",
|
||||||
|
{
|
||||||
|
"cache_hit": False,
|
||||||
|
"answer_chars": len(answer_text),
|
||||||
|
"llm_seconds": llm_ms / 1000.0,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
capital_record(
|
||||||
|
conn,
|
||||||
|
audit_event_hash=event_hash,
|
||||||
|
op_type="qa",
|
||||||
|
profile=capital_profile,
|
||||||
|
estimator_inputs=capital_inputs,
|
||||||
|
ts=now,
|
||||||
|
)
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO providence_cache "
|
"INSERT INTO providence_cache "
|
||||||
"(cache_key, source_root, document_uri, question_hash, question_text, "
|
"(cache_key, source_root, document_uri, question_hash, question_text, "
|
||||||
|
|
|
||||||
|
|
@ -59,7 +59,7 @@ Newest first. Update on every open/close.
|
||||||
|----------|------------------------------------------------|-----------------------|------------|-----------|
|
|----------|------------------------------------------------|-----------------------|------------|-----------|
|
||||||
| #000022 | Adapter LossReport (PRD I9 analogue) | open · awaiting go/no-go | 2026-05-07 | — |
|
| #000022 | Adapter LossReport (PRD I9 analogue) | open · awaiting go/no-go | 2026-05-07 | — |
|
||||||
| #000021 | 5S/5T/5R benchmark fixtures + harness | open · awaiting go/no-go | 2026-05-07 | — |
|
| #000021 | 5S/5T/5R benchmark fixtures + harness | open · awaiting go/no-go | 2026-05-07 | — |
|
||||||
| #000020 | Capital-cost ledger (8-capital queues) | open · awaiting go/no-go | 2026-05-07 | — |
|
| #000020 | Capital-cost ledger (8-capital queues) | closed · landed 2026-05-07 | 2026-05-07 | — |
|
||||||
| #000019 | Specification methodology for π* and V | open · awaiting go/no-go | 2026-05-07 | — |
|
| #000019 | Specification methodology for π* and V | open · awaiting go/no-go | 2026-05-07 | — |
|
||||||
| #000018 | Adversarial soft-hash covert-channel analysis | open · awaiting go/no-go | 2026-05-07 | — |
|
| #000018 | Adversarial soft-hash covert-channel analysis | open · awaiting go/no-go | 2026-05-07 | — |
|
||||||
| #000017 | Memory-root: lifelong learning audit chain | open · awaiting go/no-go | 2026-05-07 | — |
|
| #000017 | Memory-root: lifelong learning audit chain | open · awaiting go/no-go | 2026-05-07 | — |
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,8 @@
|
||||||
# Ticket #000020 — Capital-cost ledger: tying verifiable AGI to 8-capital queues
|
# Ticket #000020 — Capital-cost ledger: tying verifiable AGI to 8-capital queues
|
||||||
|
|
||||||
**Status:** open · awaiting go/no-go
|
**Status:** closed · landed 2026-05-07
|
||||||
**Opened:** 2026-05-07
|
**Opened:** 2026-05-07
|
||||||
|
**Closed:** 2026-05-07
|
||||||
**Scope:** Schema + audit-event tag for a capital-cost ledger that
|
**Scope:** Schema + audit-event tag for a capital-cost ledger that
|
||||||
attributes every state-changing arborist op to one or more of the 8
|
attributes every state-changing arborist op to one or more of the 8
|
||||||
forms of capital (Living, Material, Financial, Intellectual,
|
forms of capital (Living, Material, Financial, Intellectual,
|
||||||
|
|
@ -276,11 +277,40 @@ arborist capital top --form living --limit 10
|
||||||
|
|
||||||
## 7. Status
|
## 7. Status
|
||||||
|
|
||||||
**Open · awaiting go/no-go.** Small implementation surface
|
**Closed 2026-05-07.** Scope delivered:
|
||||||
(~300 lines code + ~150 lines doc + schema). One of the more
|
|
||||||
shippable tickets in this batch.
|
|
||||||
|
|
||||||
Closure criterion: `capital_ledger` schema lands, at least three op
|
- Schema migration `_migrate_capital_ledger` adds `capital_ledger`
|
||||||
types (`ingest`, `qa.runner.ask`, `distill.runner.distill`) emit
|
table (8 capital-form columns, op_type, estimator_version,
|
||||||
ledger rows on execution, `arborist capital summary` returns
|
estimator_inputs_blob). Sibling table — does NOT enter
|
||||||
non-empty totals after a workload, audit chain stays clean.
|
`audit_events.event_hash` preimage.
|
||||||
|
- Module `arborist.capital`:
|
||||||
|
- `profile.py` — `CapitalProfile` dataclass + `profile_for_op`
|
||||||
|
estimator dispatch + per-op estimators (ingest, qa, distill).
|
||||||
|
`ESTIMATOR_VERSION = "estimator-v1"` global pin.
|
||||||
|
- `store.py` — `record(...)` writes one row keyed to an
|
||||||
|
audit_event_hash; `summary(...)`, `top_by_form(...)`,
|
||||||
|
`op_cost(...)` aggregate reads.
|
||||||
|
- Wire-up at three op sites:
|
||||||
|
- `arborist/ingest.py` — one row per batch, attached to last
|
||||||
|
audit event_hash; estimator inputs `doc_count` + `total_bytes`.
|
||||||
|
- `arborist/qa/runner.py` — one row per cache-miss QA, attached
|
||||||
|
to `providence_write` event; estimator inputs `answer_chars` +
|
||||||
|
`llm_seconds`.
|
||||||
|
- `arborist/distill/runner.py` — one row per distillation,
|
||||||
|
attached to `derive` event; positive intellectual capital.
|
||||||
|
- CLI: `arborist capital summary|op-cost|top`.
|
||||||
|
- Tests: `tests/test_capital.py` — 13 cases covering profile
|
||||||
|
defaults, estimator dispatch, ledger row write, summary
|
||||||
|
aggregation, op-type filter, top-by-form ranking, sibling-table
|
||||||
|
invariant (audit chain unaffected by capital writes). All pass;
|
||||||
|
full suite 1025 passed, 36 skipped.
|
||||||
|
|
||||||
|
Out-of-scope items (deferred to follow-ups):
|
||||||
|
|
||||||
|
- Stewardship-halt runtime enforcement (`halt_on` policy
|
||||||
|
evaluation). Schema supports it; runtime check is its own ticket
|
||||||
|
once op profiles are tuned against real workloads.
|
||||||
|
- Conversion tables between capital forms.
|
||||||
|
- Real-time instrumentation (true measurement vs hardcoded
|
||||||
|
constants). v1 ships heuristic estimators; v2 will calibrate.
|
||||||
|
- Cross-shard / cross-node ledger aggregation.
|
||||||
|
|
|
||||||
278
tests/test_capital.py
Normal file
278
tests/test_capital.py
Normal file
|
|
@ -0,0 +1,278 @@
|
||||||
|
"""Capital-cost ledger tests (ticket #000020).
|
||||||
|
|
||||||
|
Covers:
|
||||||
|
- CapitalProfile defaults to zero
|
||||||
|
- profile_for_op dispatch hits per-op estimators and falls through
|
||||||
|
to zero-profile for unknown op types
|
||||||
|
- record() writes one ledger row keyed to an audit_event_hash
|
||||||
|
- summary() aggregates per-form sums correctly
|
||||||
|
- top_by_form() ranks op_types
|
||||||
|
- Sibling-table semantics: capital_ledger writes do NOT alter
|
||||||
|
audit_events.event_hash chain
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import time
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from arborist.capital import (
|
||||||
|
CapitalProfile,
|
||||||
|
op_cost,
|
||||||
|
profile_for_op,
|
||||||
|
record,
|
||||||
|
summary,
|
||||||
|
top_by_form,
|
||||||
|
)
|
||||||
|
from arborist.capital.profile import (
|
||||||
|
ESTIMATOR_VERSION,
|
||||||
|
ESTIMATORS,
|
||||||
|
_estimate_default,
|
||||||
|
)
|
||||||
|
from arborist.store import append_audit, connect, transaction
|
||||||
|
|
||||||
|
|
||||||
|
def test_capital_profile_zero_defaults():
|
||||||
|
p = CapitalProfile.zero()
|
||||||
|
for f in (
|
||||||
|
"living",
|
||||||
|
"material",
|
||||||
|
"financial",
|
||||||
|
"intellectual",
|
||||||
|
"experiential",
|
||||||
|
"social",
|
||||||
|
"cultural",
|
||||||
|
"spiritual",
|
||||||
|
):
|
||||||
|
assert getattr(p, f) == 0.0
|
||||||
|
|
||||||
|
|
||||||
|
def test_profile_for_op_ingest_basic():
|
||||||
|
p, inputs = profile_for_op(
|
||||||
|
"ingest", {"doc_count": 5, "total_bytes": 1_000_000}
|
||||||
|
)
|
||||||
|
assert p.material > 0 # storage cost positive
|
||||||
|
assert p.intellectual > 0 # docs preserved → positive
|
||||||
|
assert inputs["doc_count"] == 5
|
||||||
|
|
||||||
|
|
||||||
|
def test_profile_for_op_qa_cache_hit_path():
|
||||||
|
p, inputs = profile_for_op("qa", {"cache_hit": True})
|
||||||
|
assert p.financial == 0 # cache hits don't charge LLM
|
||||||
|
assert p.material == 0
|
||||||
|
assert p.experiential > 0 # operator wait, even if small
|
||||||
|
assert inputs["cache_hit"] is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_profile_for_op_qa_miss_path():
|
||||||
|
p, inputs = profile_for_op(
|
||||||
|
"qa",
|
||||||
|
{
|
||||||
|
"cache_hit": False,
|
||||||
|
"answer_chars": 1000,
|
||||||
|
"llm_seconds": 5.0,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert p.material > 0
|
||||||
|
assert p.financial > 0
|
||||||
|
assert p.experiential > 0
|
||||||
|
assert p.intellectual < 0 # consumes context
|
||||||
|
|
||||||
|
|
||||||
|
def test_profile_for_op_distill_positive_intellectual():
|
||||||
|
p, _ = profile_for_op(
|
||||||
|
"distill",
|
||||||
|
{"surface_count": 1, "core_count": 1, "llm_seconds": 2.0},
|
||||||
|
)
|
||||||
|
assert p.intellectual > 0 # cores preserved
|
||||||
|
|
||||||
|
|
||||||
|
def test_profile_for_op_unknown_falls_through():
|
||||||
|
p, inputs = profile_for_op("unknown_op", {"x": 1})
|
||||||
|
assert p == CapitalProfile.zero()
|
||||||
|
assert inputs == {"x": 1}
|
||||||
|
|
||||||
|
|
||||||
|
def test_record_writes_row(tmp_path):
|
||||||
|
db = tmp_path / "shard.db"
|
||||||
|
conn = connect(db)
|
||||||
|
try:
|
||||||
|
with transaction(conn):
|
||||||
|
event_hash = append_audit(
|
||||||
|
conn,
|
||||||
|
event_type="ingest",
|
||||||
|
subject_root="00" * 32,
|
||||||
|
body={"x": 1},
|
||||||
|
)
|
||||||
|
ledger_id = record(
|
||||||
|
conn,
|
||||||
|
audit_event_hash=event_hash,
|
||||||
|
op_type="ingest",
|
||||||
|
profile=CapitalProfile(material=0.5, intellectual=0.01),
|
||||||
|
estimator_inputs={"doc_count": 1},
|
||||||
|
)
|
||||||
|
assert ledger_id > 0
|
||||||
|
row = conn.execute(
|
||||||
|
"SELECT op_type, material, intellectual, estimator_version,"
|
||||||
|
" estimator_inputs_blob"
|
||||||
|
" FROM capital_ledger WHERE ledger_id = ?",
|
||||||
|
(ledger_id,),
|
||||||
|
).fetchone()
|
||||||
|
assert row["op_type"] == "ingest"
|
||||||
|
assert row["material"] == pytest.approx(0.5)
|
||||||
|
assert row["intellectual"] == pytest.approx(0.01)
|
||||||
|
assert row["estimator_version"] == ESTIMATOR_VERSION
|
||||||
|
assert "doc_count" in row["estimator_inputs_blob"]
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_summary_aggregates(tmp_path):
|
||||||
|
db = tmp_path / "shard.db"
|
||||||
|
conn = connect(db)
|
||||||
|
try:
|
||||||
|
with transaction(conn):
|
||||||
|
for i in range(3):
|
||||||
|
ev = append_audit(
|
||||||
|
conn,
|
||||||
|
event_type="ingest",
|
||||||
|
subject_root=f"{i:064x}",
|
||||||
|
body={"i": i},
|
||||||
|
)
|
||||||
|
record(
|
||||||
|
conn,
|
||||||
|
audit_event_hash=ev,
|
||||||
|
op_type="ingest",
|
||||||
|
profile=CapitalProfile(material=0.1, financial=0.01),
|
||||||
|
)
|
||||||
|
out = summary(conn)
|
||||||
|
assert out["row_count"] == 3
|
||||||
|
assert out["material"] == pytest.approx(0.3)
|
||||||
|
assert out["financial"] == pytest.approx(0.03)
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_summary_filtered_by_op_type(tmp_path):
|
||||||
|
db = tmp_path / "shard.db"
|
||||||
|
conn = connect(db)
|
||||||
|
try:
|
||||||
|
with transaction(conn):
|
||||||
|
ev1 = append_audit(
|
||||||
|
conn, event_type="ingest", subject_root=None, body={"a": 1}
|
||||||
|
)
|
||||||
|
record(
|
||||||
|
conn,
|
||||||
|
audit_event_hash=ev1,
|
||||||
|
op_type="ingest",
|
||||||
|
profile=CapitalProfile(material=1.0),
|
||||||
|
)
|
||||||
|
ev2 = append_audit(
|
||||||
|
conn, event_type="qa", subject_root=None, body={"b": 1}
|
||||||
|
)
|
||||||
|
record(
|
||||||
|
conn,
|
||||||
|
audit_event_hash=ev2,
|
||||||
|
op_type="qa",
|
||||||
|
profile=CapitalProfile(material=2.0),
|
||||||
|
)
|
||||||
|
ingest_only = summary(conn, op_type="ingest")
|
||||||
|
qa_only = summary(conn, op_type="qa")
|
||||||
|
all_in = summary(conn)
|
||||||
|
assert ingest_only["material"] == pytest.approx(1.0)
|
||||||
|
assert qa_only["material"] == pytest.approx(2.0)
|
||||||
|
assert all_in["material"] == pytest.approx(3.0)
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_top_by_form_ranks(tmp_path):
|
||||||
|
db = tmp_path / "shard.db"
|
||||||
|
conn = connect(db)
|
||||||
|
try:
|
||||||
|
with transaction(conn):
|
||||||
|
for op, val in [("ingest", 1.0), ("qa", 5.0), ("distill", 2.0)]:
|
||||||
|
ev = append_audit(
|
||||||
|
conn, event_type=op, subject_root=None, body={"k": op}
|
||||||
|
)
|
||||||
|
record(
|
||||||
|
conn,
|
||||||
|
audit_event_hash=ev,
|
||||||
|
op_type=op,
|
||||||
|
profile=CapitalProfile(material=val),
|
||||||
|
)
|
||||||
|
ranked = top_by_form(conn, "material", limit=10)
|
||||||
|
op_types = [r["op_type"] for r in ranked]
|
||||||
|
assert op_types[0] == "qa"
|
||||||
|
assert op_types[-1] == "ingest"
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_op_cost_returns_op_summary(tmp_path):
|
||||||
|
db = tmp_path / "shard.db"
|
||||||
|
conn = connect(db)
|
||||||
|
try:
|
||||||
|
with transaction(conn):
|
||||||
|
ev = append_audit(
|
||||||
|
conn, event_type="qa", subject_root=None, body={"k": "v"}
|
||||||
|
)
|
||||||
|
record(
|
||||||
|
conn,
|
||||||
|
audit_event_hash=ev,
|
||||||
|
op_type="qa",
|
||||||
|
profile=CapitalProfile(financial=0.05, experiential=0.5),
|
||||||
|
)
|
||||||
|
out = op_cost(conn, "qa")
|
||||||
|
assert out["financial"] == pytest.approx(0.05)
|
||||||
|
assert out["experiential"] == pytest.approx(0.5)
|
||||||
|
assert out["row_count"] == 1
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_top_by_form_rejects_unknown_form(tmp_path):
|
||||||
|
db = tmp_path / "shard.db"
|
||||||
|
conn = connect(db)
|
||||||
|
try:
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
top_by_form(conn, "imaginary_form")
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_audit_chain_unaffected_by_capital_writes(tmp_path):
|
||||||
|
"""capital_ledger writes are sibling-table; chain stays intact."""
|
||||||
|
import hashlib
|
||||||
|
|
||||||
|
db = tmp_path / "shard.db"
|
||||||
|
conn = connect(db)
|
||||||
|
try:
|
||||||
|
with transaction(conn):
|
||||||
|
ev1 = append_audit(
|
||||||
|
conn, event_type="x", subject_root=None, body={"a": 1}
|
||||||
|
)
|
||||||
|
record(
|
||||||
|
conn,
|
||||||
|
audit_event_hash=ev1,
|
||||||
|
op_type="x",
|
||||||
|
profile=CapitalProfile(material=0.1),
|
||||||
|
)
|
||||||
|
ev2 = append_audit(
|
||||||
|
conn, event_type="y", subject_root=None, body={"b": 2}
|
||||||
|
)
|
||||||
|
rows = conn.execute(
|
||||||
|
"SELECT event_hash, prev_event_hash, body "
|
||||||
|
"FROM audit_events ORDER BY seq"
|
||||||
|
).fetchall()
|
||||||
|
prev = None
|
||||||
|
for r in rows:
|
||||||
|
h = hashlib.sha256()
|
||||||
|
if prev is not None:
|
||||||
|
h.update(bytes.fromhex(prev))
|
||||||
|
h.update(r["body"].encode("utf-8", errors="surrogatepass"))
|
||||||
|
assert h.hexdigest() == r["event_hash"]
|
||||||
|
prev = r["event_hash"]
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
Loading…
Add table
Add a link
Reference in a new issue