Commit graph

3 commits

Author SHA1 Message Date
36edc3b8dc modified: archive.py
modified:   tests/unit/test_screenshot.py
2025-12-29 11:35:14 -05:00
1847a2fee6 Add comprehensive unit tests for async modules
Tests for:
- async_web_fetcher: URL handling, media extraction, crawl modes, scoring
- domain_vault: VaultManager, HTML/Media/Linkpeek vaults
- screenshot: ScreenshotCapture configuration and availability
- storage: ImageVault with MD5 deduplication

185 tests total, all passing.
2025-12-22 20:40:58 -05:00
bc11c516b6 Add filevault system with async wrapper and convert sync ops to async
New files:
- filevault.py: Hash-based file storage with use_pairs option (v1.1.0)
- async_filevault.py: Async wrapper using asyncio.to_thread()
- domain_vault.py: Triple vault system for web archival (HTML, Media, Linkpeek)
- screenshot.py: Async screenshot capture using uri2png
- tests/unit/: Comprehensive test suite (85 tests)

Sync-to-async conversions:
- storage.py: Wrap Path operations in asyncio.to_thread()
- domain_vault.py: Wrap exists(), mkdir(), rglob(), os.walk() in asyncio.to_thread()
- screenshot.py: Wrap read_bytes(), write_bytes(), unlink() in asyncio.to_thread()

All sync filesystem operations now run in thread pool to avoid blocking async loop.
2025-12-22 20:18:20 -05:00