journal: emit unfirehose/1.0 JSONL for queries + bench cycles

aborist now writes one JSONL session per `make query` invocation
and per `bench-emergent` cycle to:

    ~/.aborist/unfirehose/{project-slug}/{session-uuid}.jsonl

Unfirehose's native-harness auto-discovery picks up any
~/.{name}/unfirehose/ directory (see ingest.ts:discoverNativeHarnesses)
without registration — once a session lands, the unfirehose watcher
debounces, ingests, and exposes it in the dashboard alongside
Claude Code / Fetch / uncloseai sessions.

Schema: unfirehose/1.0 (per ~/git/unfirehose-nextjs-logger/docs/
unfirehose-schema.md). Each session file:

    line 1   type=session  (header — id, projectId, firstPrompt,
                            harness="aborist", harnessVersion)
    line 2   type=message role=user
    line 3   type=message role=assistant
                          content=[text]
                          model=hermes-3-llama-3.1-8b-fp8-dynamic
                          provider=hermes
                          durationMs=<wall>
                          aborist_meta={audit_mode, n_verified/n_quotes,
                            cache_key, cache_status, lookup_path,
                            violations, sources, timings_ms, answer_mode}
    line 4   type=message role=system subtype=session_end durationMs

aborist-specific extras (verifier verdict, sources, timings) ride
under namespaced ``aborist_meta`` so the canonical fields stay clean
for off-the-shelf consumers; per the spec, unknown fields are
ignored downstream.

Bench-emergent cycles emit an additional system init message at
the start of each session noting the 3 random words, marking the
session as a generator-driven cycle vs a normal user query.

Failure-isolation: journal write is wrapped in a broad try/except
at every call site. A journaling bug must NEVER break the query
or bench loop.

Tests: 10 new in tests/test_journal.py (slug encoding, session
header, parent-id chain, session_end on close, aborist_meta
passthrough, usage block, idempotent close). Full suite: 663 passed.

Live verified: `make query Q="what is photosynthesis?"` produced
a 4-line JSONL with STRICT 3/3, all sources + timings populated,
ready for unfirehose ingestion.
This commit is contained in:
russell@unturf.com 2026-05-02 15:19:39 -04:00
parent 529434ec67
commit d3ad520529
No known key found for this signature in database
4 changed files with 504 additions and 0 deletions

View file

@ -203,6 +203,51 @@ def append_log(entry: dict, log_path: Path) -> None:
f.write(json.dumps(entry, ensure_ascii=False) + "\n")
def emit_unfirehose_session(entry: dict, model_id: str) -> None:
"""Write one unfirehose/1.0 session for this bench cycle.
Same auto-discovery path as `aborist.cli._emit_query_journal`:
~/.aborist/unfirehose/{slug}/{session_uuid}.jsonl. Each cycle gets
its own session file (one prompt → one answer = one session).
Failures must NEVER break the bench loop — wrapped at the call
site.
"""
from aborist.journal import SessionWriter
aborist_meta = {
"audit_mode": entry.get("audit_mode"),
"verifier_method": entry.get("verifier_method"),
"n_quotes": entry.get("n_quotes"),
"n_verified": entry.get("n_verified"),
"cache_key": entry.get("cache_key"),
"violation_kinds": entry.get("violation_kinds"),
"metaphor_deflection_kind": entry.get("metaphor_deflection_kind"),
"answer_mode": entry.get("answer_mode"),
"sources": entry.get("sources"),
"timings": {
"question_gen_seconds": entry.get("question_gen_seconds"),
"answer_seconds": entry.get("answer_seconds"),
"total_seconds": entry.get("total_seconds"),
},
"bench": "emergent",
"words": entry.get("words"),
"student_error": entry.get("student_error"),
}
with SessionWriter(first_prompt=entry.get("question") or "") as s:
s.system_message(
"bench-emergent cycle: 3-word triangulation",
subtype="init",
aborist_meta={"words": entry.get("words"), "harness_role": "generator"},
)
s.user_message(entry.get("question") or "")
s.assistant_message(
entry.get("answer") or "",
model=model_id,
provider="hermes",
duration_ms=int((entry.get("answer_seconds") or 0) * 1000) or None,
aborist_meta=aborist_meta,
)
def print_pending(log_path: Path) -> int:
"""Print every log entry with `teacher: None` — what fox should
bring to a teacher model for review."""
@ -294,6 +339,14 @@ def main(argv: list[str] | None = None) -> int:
top_k=ns.top_k,
)
append_log(entry, ns.log_path)
# Mirror to unfirehose-compatible journal so the bench cycles
# show up in the unfirehose dashboard alongside Claude Code /
# Fetch sessions. Best-effort — never break the bench on a
# journal write failure.
try:
emit_unfirehose_session(entry, ns.model)
except Exception: # pragma: no cover
pass
# Compact stdout summary so a long sweep is observable.
audit = entry.get("audit_mode") or "?"
ratio = (