Commit graph

17 commits

Author SHA1 Message Date
027f5afc57 Add public domain license and headers to all source files
- Add LICENSE file (public domain, permacomputer values)
- Add license headers to all 27 Python source files
- Restore full about_ch9_p2 with "terminated 🟣" content
- Translate about_ch9_p2 to all 26 languages

The permacomputer is community-owned infrastructure optimized around:
TRUTH, FREEDOM, HARMONY, LOVE
2026-01-08 15:38:28 -05:00
58c93f217a Reorganize project structure: move modules to neopig/, scripts to scripts/, data to data/ 2026-01-06 14:06:43 -05:00
f124828c16 21 side quests scattered through the codebase
1-20 are findable. 21 is the absence itself.

fnord
2026-01-05 21:10:02 -05:00
e9c0cf965f The 5 Adventures: View Source rewards the curious
Setting Orange, 5th of Chaos, YOLD 3192 - Mungday

Adventure #1: View Source on any page
Adventure #2: /fnord - you weren't supposed to see it
Adventure #3: /5 - The Law of Fives is never wrong
Adventure #4: /23 - The 23 enigma
Adventure #5: /kallisti - ΚΑΛΛΙΣΤΙ - to the prettiest one

Hidden throughout:
- HTML comments in templates guide the seeker
- fnord appears where you least expect it
- 23 skidoo in the live feed
- The Sign Maker leaves breadcrumbs

Truth, Freedom, Harmony, Love.
2026-01-05 20:22:34 -05:00
66d2f4d1d9 Port 31337 default, code file display fixes
- Change default port from 8000 to 31337 across all files
- About page: and → &, beast → pig
- Search results: SVG placeholder for code files (shows extension)
- Download button: remove .bin fallback, use original extension
2026-01-02 15:35:39 -05:00
52952a104a modified: CLAUDE.md
modified:   archive.py
	new file:   bootstrap-neopig.sh
	modified:   bootstrap.c
	modified:   serp.py
2026-01-02 11:51:42 -05:00
17efcf9a27 Archive improvements: venv support, package-only mode, resume fixes
- bootstrap.c: Extract requirements.txt, create venv, pip install deps
- archive.py: Add --package-only flag, use neopig vault directly
- archive.py: Resume now loads crawled pages from DB
- database.py: Add close() and get_crawled_page_uris()
- domain_vault.py: Fix None path handling in url_to_filepath
- Makefile: Add test-alpha target for testing archives
2026-01-01 15:57:23 -05:00
b0542fccc6 Improve depth CLI help and add max depth validation
- Clarify depth help: 0=single page, 1=page+links, -1=unlimited
- Reject depth >18, suggest using -1 for unlimited crawling
- Update CLAUDE.md with depth documentation
2025-12-31 15:37:18 -05:00
8d669e206c Comprehensive README rewrite with full API documentation
- README.md: Complete rewrite in About page style with origin story
- Documents all storage systems: filevault, ImageVault, domain_vault
- Full database schema and method reference
- Screenshot capture with all engines and config options
- HTML to Markdown converter with content detection
- CLI reference for neopig.py, archive.py, serp.py
- All API endpoints documented
2025-12-31 15:21:55 -05:00
3952978ba4 Screenshot backfill improvements and oversized image handling
- Filter completed URIs at query level for faster backfill resume
- Split oversized screenshots (>65000px) into multiple JPEG chunks
- Progress bar shows total including skipped, advances on errors
- Suppress PIL DecompressionBombWarning for large screenshots
- Cleaner error logging (truncate verbose wkhtmltoimage output)
- Per-page capture logs moved to debug level
2025-12-30 11:05:10 -05:00
7b4c8b0241 modified: .gitignore
modified:   archive.py
	modified:   html2md.py
	modified:   neopig.py
	modified:   screenshot.py
2025-12-30 09:50:54 -05:00
623213f5b6 Self-extracting archives now use bundled neopig/serp.py
- Removed embedded serve.py (872 lines) from archive.py
- bootstrap.c extracts neopig/*.py to /tmp and runs serp.py
- serp.py: tarball mode serves media directly from tar.gz
- OffsetFile wrapper for reading .run files at correct offset
- ArchiveDB: simple SQLite wrapper for archive search (no async deps)
- Archives bundle all neopig source files for self-contained operation
- --upgrade-neopig flag with progress logging
2025-12-30 07:46:59 -05:00
416b3cb760 Add self-extracting archives with bundled neopig
- Bootstrap C program extracts serve.py and runs from tarball
- Archives now include neopig source files for self-contained crawling
- --upgrade-neopig flag to update neopig in existing archives
- html2md.py: smart HTML-to-markdown converter for forums/blogs/Q&A
- Fix vault path defaults (data/vault instead of vault)
- Streaming tar.gz creation without temp copies
- URL rewriting for local media references in archives
2025-12-30 06:55:20 -05:00
a1a58fbb50 Add backfill-markdown, download button, fix image hydration
- backfill-markdown: regenerate markdown with absolute URLs for a domain
- Download button on /view page with slugified filename from alt/title
- html2text baseurl resolves relative URLs during crawl
- Fallback URL lookup by filename for older crawls
- Video hover-to-play on all video elements
- Avatar CSS float:left for text wrapping
- FileResponse inline disposition (view not download)
- Dynamic screenshot delay based on content length
2025-12-29 15:05:39 -05:00
222c23bb14 modified: CLAUDE.md
modified:   archive.py
	modified:   database.py
	modified:   neopig.py
	modified:   screenshot.py
	modified:   serp.py
2025-12-29 12:13:33 -05:00
36edc3b8dc modified: archive.py
modified:   tests/unit/test_screenshot.py
2025-12-29 11:35:14 -05:00
21e344f675 Add archive.py with --serve flag for live crawl watching
- archive.py: New site archiver wrapper with embedded serve.py
- --serve flag starts SERP server alongside crawl for live viewing
- --fast mode for sites without robots.txt (no crawl delay)
- Symlink-based storage: domain views link to hash vault
- Tarball resolves symlinks to include only domain content
- Fixed bytes_downloaded/bytes_stored accounting for screenshots
- Data directory (data/) for databases and state files
- Standardized on - separator in filenames
2025-12-29 11:26:02 -05:00