Phase 1 of module restructuring as outlined in docs/REFACTOR.md:
New neopig/ package modules:
- live.py: Live media queue (get_live_queue, emit_live_media)
- state.py: AppendOnlyStateLog, state file helpers
- html_utils.py: trim_html_wrapper, extract_meta_from_html
- logging.py: TqdmLoggingHandler, job logging functions
- backfill/: markdown and screenshot backfill operations
Package features:
- Lazy import of NeoPig/main from neopig.py via __getattr__
- Full backwards compatibility with existing imports
- 42 new unit tests for extracted modules
Total: 458 tests passing
- Extract 1100+ lines of translations from serp.py to i18n.py
- Add Georgian (ka) as 27th supported language
- Add 74 about page body text keys (hero, chapters, features, footer)
- Update about.html.j2 to use translation placeholders
- Full Chinese translations for all body text
- English fallback for other 25 languages (can be translated later)
- serp.py reduced from 3291 to 2142 lines
- i18n.py now 3808 lines with 190 keys per language
New files:
- filevault.py: Hash-based file storage with use_pairs option (v1.1.0)
- async_filevault.py: Async wrapper using asyncio.to_thread()
- domain_vault.py: Triple vault system for web archival (HTML, Media, Linkpeek)
- screenshot.py: Async screenshot capture using uri2png
- tests/unit/: Comprehensive test suite (85 tests)
Sync-to-async conversions:
- storage.py: Wrap Path operations in asyncio.to_thread()
- domain_vault.py: Wrap exists(), mkdir(), rglob(), os.walk() in asyncio.to_thread()
- screenshot.py: Wrap read_bytes(), write_bytes(), unlink() in asyncio.to_thread()
All sync filesystem operations now run in thread pool to avoid blocking async loop.
