Commit graph

6 commits

Author SHA1 Message Date
af50334a7b Fix tests for code media type, add source URL and lore 2026-01-02 13:41:10 -05:00
96a3c9c786 Update tests for filevault 2.0.0 API
- Rewrite test_filevault.py for new API (depth=9, hex pairs, content-addressable)
- Rewrite test_async_filevault.py to import from filevault module
- Fix test_screenshot.py defaults (enabled=True, 1024x768)
- Add backward compatibility exports to filevault.py
- All 196 tests passing
2025-12-31 16:03:43 -05:00
99c836e2c8 Unify filevault: merge storage.py into single sync+async module
- filevault.py: Vault (sync) + AsyncVault (async) in one file
- Delete storage.py (redundant async wrapper)
- Both classes share identical API: store, get, exists, delete, stats
- Add content_hash() and hash_to_path() utility functions
- Update all imports: neopig.py, serp.py, test files
- Update README with unified documentation
- 72 lines saved, cleaner architecture
2025-12-31 15:44:39 -05:00
36edc3b8dc modified: archive.py
modified:   tests/unit/test_screenshot.py
2025-12-29 11:35:14 -05:00
1847a2fee6 Add comprehensive unit tests for async modules
Tests for:
- async_web_fetcher: URL handling, media extraction, crawl modes, scoring
- domain_vault: VaultManager, HTML/Media/Linkpeek vaults
- screenshot: ScreenshotCapture configuration and availability
- storage: ImageVault with MD5 deduplication

185 tests total, all passing.
2025-12-22 20:40:58 -05:00
bc11c516b6 Add filevault system with async wrapper and convert sync ops to async
New files:
- filevault.py: Hash-based file storage with use_pairs option (v1.1.0)
- async_filevault.py: Async wrapper using asyncio.to_thread()
- domain_vault.py: Triple vault system for web archival (HTML, Media, Linkpeek)
- screenshot.py: Async screenshot capture using uri2png
- tests/unit/: Comprehensive test suite (85 tests)

Sync-to-async conversions:
- storage.py: Wrap Path operations in asyncio.to_thread()
- domain_vault.py: Wrap exists(), mkdir(), rglob(), os.walk() in asyncio.to_thread()
- screenshot.py: Wrap read_bytes(), write_bytes(), unlink() in asyncio.to_thread()

All sync filesystem operations now run in thread pool to avoid blocking async loop.
2025-12-22 20:18:20 -05:00