Commit graph

853 commits

Author SHA1 Message Date
01a69db9e7 docs: auto-push when new tests cover new paths and suite is green 2026-02-21 13:57:14 -05:00
2d5a3622a3 bump GIT_HASH to a867602 2026-02-21 13:56:47 -05:00
a8676025e4 fix: update_s3_acls now handles karaoke tracks + add tests
update_s3_acls silently skipped instrumentals/vocals because no s3_key
resolution existed for those file keys. Added generic fallback to
construct s3_path/file_key for any unhandled key.

Added 11 tests: unit (ACL parity across visibility levels, file_keys
membership, update_s3_acls call count), integration (visibility change
propagates to karaoke ACLs), functional (watch JSON and content page
karaoke URL embedding).
2026-02-21 13:56:33 -05:00
f0ca717ff9 fix: restructure ring controls into 2-row grid layout
The controls bar was overflowing — "Next ▶" wrapped vertically. Switch
from single-row grid-auto-flow to a 3-column grid so items split into
two rows: toggles (Fresh/Reverse/Autoplay) on top, action buttons
(Karaoke/Random/Next) below. Reduce gap from 16px to 6×12px.
2026-02-21 12:35:48 -05:00
5c9513daa2 feat: add karaoke toggle to watch mode ring controls
Moves vocal isolation track switching from the pop-out player into the
ring header controls so users can cycle Original/Instrumentals/Vocals
while watching. Preserves playback position on track switch, resets to
Original on SPA navigation, and adds K keyboard shortcut.
2026-02-21 11:12:02 -05:00
68e716ca74 bump GIT_HASH to 94166be 2026-02-19 20:00:36 -05:00
94166be988 fix: sign /upload with empty body — API skips body parsing for streams
The upload endpoint streams the request body in chunks without buffering,
so raw_body stays empty on the server side. The HMAC must sign against an
empty body string to match. _sign_from_disk (which hashed file contents)
produced signatures the server could never verify — every upload got 401.
2026-02-19 20:00:23 -05:00
c3072c2a6e update GIT_HASH to 2e73c09 2026-02-19 19:18:54 -05:00
2e73c09e63 fix: use upload path instead of inline base64 to prevent API OOM
The inline path loaded entire base64 payloads into API memory via
Plug.Parsers, then decoded them again for UploadStore — 4 concurrent
45MB files consumed ~420MB on a 768MB droplet, pushing BEAM into swap
and timing out all RPCs.

Now uses POST /upload (streaming 64KB chunks, constant memory) then
references upload_ids in the execute call. The JSON body drops from
~60MB to ~500 bytes. Zero base64 encoding on the request side.
2026-02-19 19:18:36 -05:00
3fbef65b20 docs: add karaoke pipeline architecture to CLAUDE.md
Disk-backed vocal isolation pipeline, 3.698GB upstream limit,
concurrency, retry, and caller documentation.
2026-02-19 17:56:23 -05:00
046fb1342d update GIT_HASH to 5736ff3 2026-02-19 14:54:57 -05:00
5736ff3330 fix: disk-backed karaoke pipeline to prevent OOM kills
process_karaoke held 4+ copies of every media file in memory
simultaneously (raw bytes, base64, JSON serialization). With 14-16
concurrent workers on large files, memory exploded past 3.2GB RSS.

Replace all in-memory buffers with a disk-backed pipeline:
- Stream S3 download to tmpfile instead of .read()
- Build JSON payload on disk with streaming base64 encoding
- Incremental HMAC-SHA256 signing from disk
- Stream HTTP request body from file, response to file
- Decode artifacts to tmpfiles one at a time for S3 upload

Peak request-side memory drops from ~1GB/worker to ~64KB/worker.
Function signature unchanged — all callers work without modification.
2026-02-19 13:49:45 -05:00
e87bb0b0c3 fix: alembic migration uses table name mps_page_session, not class name
op.add_column takes the actual SQLite table name, not the SQLAlchemy
model class name. PageSession maps to mps_page_session via CLASS_TO_TABLE.
2026-02-19 11:42:18 -05:00
a0d17cbf5f trigger deploy: run pending alembic migration 2026-02-19 10:37:40 -05:00
67e34b08ad fix: create fresh SQLite engine after fork instead of reusing dead fd
The grandchild process closes all inherited fds (3..maxfd) to detach
from uWSGI. session_factory().get_bind() returned a session bound to
the now-dead SQLite fd. Capture the DB URL before forking, create a
fresh engine in the child.
2026-02-19 10:20:38 -05:00
b8955c0459 revert: remove alembic from CI deploy, fix belongs in salt states 2026-02-19 09:28:43 -05:00
b21eb4a1e0 deploy: run alembic as uwsgi user 2026-02-19 09:27:01 -05:00
6e5bf4ad8d deploy: run alembic migrations after highstate 2026-02-19 09:26:05 -05:00
7e620f39f1 uncap concurrency, skip per-item validate_keys in backfill 2026-02-18 19:46:28 -05:00
535e57896b cap backfill concurrency at 2 — 14 simultaneous large payloads crashes API 2026-02-18 19:43:16 -05:00
696a7b853b uncap backfill concurrency — use key's actual limit 2026-02-18 19:30:46 -05:00
e1abca4a66 throttle backfill to 1 worker, increase stderr log to 2000 chars
4 concurrent workers was overwhelming the unsandbox API (429/502).
Cap at 1 worker for now. Also increase stderr truncation from 500
to 2000 chars to see full ffmpeg errors.
2026-02-18 19:16:22 -05:00
2908511a8c double-fork backfill to fully detach from uWSGI worker
Single fork left the child in the uWSGI process group with inherited
HTTP sockets, causing 502s. Double-fork with setsid() and fd cleanup
so the grandchild is fully orphaned to init.
2026-02-18 13:04:46 -05:00
0ad82d5314 add backfill vocal isolation button to shop settings 2026-02-18 11:34:08 -05:00
44e0210d5e extract audio to WAV before running voxsplit
voxsplit only accepts WAV input — use ffmpeg to extract/convert
audio from mp4/mp3/etc before splitting vocals/instrumentals.
2026-02-18 10:18:45 -05:00
20172cffcb log stdout/stderr from unsandbox container on exit_code!=0 2026-02-18 10:14:01 -05:00
707c3f44b0 refactor backfill from daemon thread to forked process
uWSGI kills worker processes after ~60-90s ("NO MERCY!!!"), which
kills all daemon threads including the backfill. Fork a child process
that's independent of the worker — survives parent SIGKILL.

Replace threading.Lock + set guard with fcntl.flock on a lockfile
(/tmp/karaoke_backfill_{shop_id}.lock) for one-per-shop guard that
auto-releases on crash. Also add error logging for unsandbox execute
failures (was previously silent).
2026-02-18 09:48:03 -05:00
b022d67323 analytics: add visual charts to shop and product dashboards
SVG bar charts for 28-day daily views, CSS horizontal bars for
traffic sources and device split, inline mini bars in top products
table. Zero-dep, server-side rendered, theme-aware, no JS.
2026-02-18 09:14:20 -05:00
85dffd0b24 add debug logging to backfill: bucket, key, s3_path, and work item counts 2026-02-18 08:14:48 -05:00
8adf2fadaf analytics: exclude shop owner traffic from sales funnel metrics
add is_owner flag to PageSession — set via data attribute when
the viewer can_edit_shop. beacon stores the flag, view_count
skips increment for owners, and all analytics queries filter
owner sessions out of funnel metrics. owner sessions are still
recorded for separate analysis. existing NULL data treated as
non-owner.
2026-02-18 07:26:03 -05:00
03b098ad3e validate unsandbox keys before each process and backfill
Keys can have tiers changed or expire mid-run. Now:
- process_karaoke validates keys before downloading from S3
- backfill aborts cleanly if keys are invalid instead of defaulting to 1 worker
2026-02-18 07:18:18 -05:00
dd140842ca skip backfill retry when source file missing from S3 (NoSuchKey) 2026-02-18 07:13:28 -05:00
015121b600 include voxsplit.c in package data 2026-02-18 06:52:01 -05:00
8a0eef7b39 keep unsandbox keys on validation failure, let user fix them 2026-02-18 06:49:19 -05:00
16314eeee5 fix backfill: use get_object/put_object instead of download_file/upload_file
boto3 TransferManager spawns its own ThreadPoolExecutor internally,
which crashes with 'cannot schedule new futures after interpreter
shutdown' when uWSGI recycles the worker process.
2026-02-18 06:09:57 -05:00
a9a9ba69b1 add unsandbox key extension link in shop settings 2026-02-18 06:01:50 -05:00
c8b9dd310a validate unsandbox keys on save, reject invalid/expired 2026-02-18 06:01:11 -05:00
be40ab432b vendor voxsplit.c into lib/ — was pointing at local dev path 2026-02-18 00:22:54 -05:00
f77d06a80b vendor unsandbox SDK into lib/un.py 2026-02-17 23:55:59 -05:00
f8ce118305 fix karaoke: lazy-load unsandbox SDK to avoid import failure in CI 2026-02-17 23:51:06 -05:00
1135eb24e4 add karaoke mode: vocal isolation via unsandbox zerotrust containers
Per-shop unsandbox API keys in shop settings. On upload, audio/video
products are processed through voxsplit in a zerotrust container (no
network, destroyed after use) producing both instrumentals and vocals
tracks. Background backfill with ThreadPoolExecutor sized to account
concurrency, exponential backoff retries. Player cycles Original →
Instrumentals → Vocals with toggle button or K shortcut.
2026-02-17 23:46:45 -05:00
6e40886cd7 Fix RGB color filters: use pure CSS filter chains instead of SVG references
Chrome blocks external SVG filter references via CSS url(). Replace with
native CSS filter functions (grayscale + sepia + hue-rotate + saturate)
that work in all browsers. Delete the now-unused SVG file.
2026-02-15 12:25:45 -05:00
9310360903 Fix edit page media previews: add presigned URL generation, type-aware embeds, mock S3 in test 2026-02-15 11:47:27 -05:00
6a536f18c9 Fix RGB color filters: use external SVG file for filter definitions
Inline SVG filter url(#id) fails when the SVG defs are inside the
filtered element (both on <html> and <body>). Data URI SVG filters
are blocked by Chrome. External SVG file at /static/svg/color-filters.svg
avoids both issues — no DOM circular reference, no security block.
Browser fetches the SVG once and caches it.
2026-02-15 11:04:27 -05:00
f0a5a392d0 Fix RGB color filters: apply to body, not html, with inline SVG defs
Data URI SVG filters don't work in Chrome (blocked for security).
Inline url(#id) on <html> fails due to root-element rendering order.
Fix: keep data-color-filter attribute on <html>, apply filter to <body>
via descendant selector. SVG filter defs in <body> with <defs> wrapper
resolve correctly since DOM is parsed before CSS filter is applied.
2026-02-15 10:49:44 -05:00
85b1a2d033 Fix RGB color filters: use data URI SVG instead of inline DOM references
CSS filter: url(#id) on <html> fails because the SVG filter definitions
are children of the element being filtered (circular reference). Grayscale
worked because it uses native CSS filter, not SVG. Now all feColorMatrix
filters are self-contained data URIs in CSS — no DOM SVG block needed.
2026-02-15 09:23:14 -05:00
25b9f53c99 Add random button to watch ring, RGB color filters, move comment email below textarea
- Add shuffle button next to Next in SPA ring navigation
- Expand grayscale mode to full color filter system (off, grayscale, R, G, B, R+G, R+B, G+B)
  using SVG feColorMatrix filters for selective channel isolation
- Move comment email field below textarea, fix email input theming for dark mode
2026-02-15 08:10:15 -05:00
dfb70666b4 Add grayscale mode: CSS filter renders entire shop in black and white
Server renders data-grayscale="on" on <html> so it works without JS.
Toggle in Ribbon & Layout settings, off by default. Original assets
untouched — purely presentational via filter: grayscale(100%).
2026-02-15 05:25:30 -05:00
d07195568b Fix mobile related content showing 42 items instead of 7
The .related-content-overflow display:none rule was being overridden by
.related-content-row display:grid later in the file (equal specificity,
cascade order wins). Bump specificity with .related-content-row.related-content-overflow.
2026-02-14 11:38:32 -05:00
7e4b691882 Trigger salt highstate: add origin.makepostsell.com caddy site block 2026-02-13 19:42:31 -05:00