russell@unturf.com
9ba06cd809
aborist/sources/grok.py exposes two Source subclasses for ingesting
xAI's user data export:
GrokExportSource walks ttl/<period>/export_data/<user-id>/ and
reads prod-grok-backend.json. Yields one Document
per conversation with:
- URI: grok://conversation/<conversation-id>
- title: the conversation's auto-generated title
- content: full message text in turn order
source_type='grok_export'.
GrokMediaPostsSource same export, but yields per media-generation
post (image/video prompts) under URI
grok://media/<post-id>.
source_type='grok_media'.
Both auto-walk down from the export root so callers can pass the
top-level directory xAI delivered (e.g., ~/Downloads/<user-uuid>/).
Wired into the CLI: aborist ingest --source {grok_export,grok_media}
accepts --path <export-root>. tests/test_grok_source.py covers the
walk + parse + Document shape with a fabricated mini-export fixture.
Once ingested, conversations become normal queryable docs in the
shard cluster — your prior chats become memory the corpus can
consult during RAG.
79 tests passing.
|
||
|---|---|---|
| .. | ||
| __init__.py | ||
| test_concepts.py | ||
| test_distill.py | ||
| test_distill_recursive.py | ||
| test_evict.py | ||
| test_grok_source.py | ||
| test_html_source.py | ||
| test_ingest.py | ||
| test_merkle.py | ||
| test_qa.py | ||
| test_query.py | ||
| test_resume.py | ||
| test_tfidf.py | ||
| test_versioned_ingest.py | ||
| test_wikipedia_old.py | ||