5 commits

Author	SHA1	Message	Date
Russell Ballestrini	52952a104a	modified: CLAUDE.md ... modified: archive.py new file: bootstrap-neopig.sh modified: bootstrap.c modified: serp.py	2026-01-02 11:51:42 -05:00
Russell Ballestrini	eaf21d58d2	modified: bootstrap.c	2026-01-01 16:07:10 -05:00
Russell Ballestrini	17efcf9a27	Archive improvements: venv support, package-only mode, resume fixes ... - bootstrap.c: Extract requirements.txt, create venv, pip install deps - archive.py: Add --package-only flag, use neopig vault directly - archive.py: Resume now loads crawled pages from DB - database.py: Add close() and get_crawled_page_uris() - domain_vault.py: Fix None path handling in url_to_filepath - Makefile: Add test-alpha target for testing archives	2026-01-01 15:57:23 -05:00
Russell Ballestrini	623213f5b6	Self-extracting archives now use bundled neopig/serp.py ... - Removed embedded serve.py (872 lines) from archive.py - bootstrap.c extracts neopig/*.py to /tmp and runs serp.py - serp.py: tarball mode serves media directly from tar.gz - OffsetFile wrapper for reading .run files at correct offset - ArchiveDB: simple SQLite wrapper for archive search (no async deps) - Archives bundle all neopig source files for self-contained operation - --upgrade-neopig flag with progress logging	2025-12-30 07:46:59 -05:00
Russell Ballestrini	416b3cb760	Add self-extracting archives with bundled neopig ... - Bootstrap C program extracts serve.py and runs from tarball - Archives now include neopig source files for self-contained crawling - --upgrade-neopig flag to update neopig in existing archives - html2md.py: smart HTML-to-markdown converter for forums/blogs/Q&A - Fix vault path defaults (data/vault instead of vault) - Streaming tar.gz creation without temp copies - URL rewriting for local media references in archives	2025-12-30 06:55:20 -05:00