engineering/make_post_sell
1
0
Fork 0
Code Issues 2 Pull requests Projects Releases 2 Packages Wiki Activity Actions

docs: add MPS-6 through MPS-9 tickets, architecture diagram, update JS docs

Browse source 
- MPS-6: referrer analytics (domain, query, trend line charts)
- MPS-7: sandbox mode creative filter system
- MPS-8: user S3 bucket + artifact storage
- MPS-9: shop S3 mirror bucket
- architecture.md: system diagram, request flow, data pipeline, S3 layout
- JAVASCRIPT.md: add sandbox.js, signals.js, MediaPipe SDK entries
- sandbox-mode.md: mark S3 upload as implemented
- mps-2.md: document referrer_domain + referrer_query columns

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
russell@unturf.com 2026-03-05 06:19:56 -05:00
parent 7a6fadeeaa
commit 32a19a339c
8 changed files with 590 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

21
docs/JAVASCRIPT.md
View file

@ -37,6 +37,26 @@ Located in `make_post_sell/static/js/`:
- Autoplay toggle persisted in localStorage
- Ring state (position, direction, history) persisted in localStorage
### sandbox.js
- Client-side creative filter engine (IIFE, ~700 lines)
- Used by: `base.j2` when shop has `sandbox_mode` enabled
- 32 CSS filter presets across 7 categories (basic, warm, cool, dramatic, color, Instagram-style, SVG)
- 7 adjustment sliders (brightness, contrast, saturation, hue, blur, sepia, grayscale)
- Image export via canvas `toBlob()` (PNG)
- Video export via `MediaRecorder` + `captureStream(30fps)` (WebM)
- On-demand face detection via MediaPipe Face Mesh (468 landmarks, eye glow, face mask overlays)
- Upload-to-bucket: presigned POST to user's S3 bucket via `/u/sandbox/upload`
- localStorage persistence: preset, slider values, panel state, filter mode
- Exposes `window.sandboxReapply()` for watch.js SPA integration
### signals.js
- Anonymous page visit signal collector
- Used by: all product/content pages
- Collects presence (wall_clock, visible, active), scroll (depth, direction changes), and media playback signals
- Sends single JSON beacon (~300 bytes) via `navigator.sendBeacon()` on page unload
- No cookies, no IPs, no fingerprints — purely behavioral signals
- Viewport width sent for device class derivation (server-side)
### comments.js
- AJAX comment submission (progressive enhancement)
- Intercepts comment form POST, submits via `fetch()` with `X-Requested-With: XMLHttpRequest`
@ -138,6 +158,7 @@ Located in `make_post_sell/static/js/`:
| Stripe | `js.stripe.com/v3/` | Card payment form |
| Plausible | Shop-configured domain | Privacy analytics |
| Google Analytics | `googletagmanager.com` | Google analytics |
| MediaPipe | `cdn.jsdelivr.net` | Face detection (sandbox mode, on-demand) |
## CSRF Protection

205
docs/architecture.md Normal file
View file

@ -0,0 +1,205 @@
# MPS Architecture Overview
## System Diagram
```
Internet
|
┌────────┴────────┐
│ Caddy (HTTPS) │
│ reverse proxy │
│ :443 → :6001 │
└────────┬────────┘
|
┌────────┴────────┐
│ uWSGI │
│ 2 proc, 8 thr │
│ reload@512MB │
└────┬───────┬────┘
| |
┌──────────────┘ └──────────────┐
| |
┌────────┴────────┐ ┌────────┴────────┐
│ Pyramid / WSGI │ │ Background │
│ Request Cycle │ │ Threads │
│ │ │ │
│ views/ │ │ S3 mirror │
│ models/ │ │ Karaoke vocal │
│ templates/ │ │ isolation │
│ lib/ │ │ │
└───┬────┬────┬───┘ └────────┬────────┘
| | | |
┌─────────┘ | └─────────┐ |
| | | |
┌────┴────┐ ┌─────┴─────┐ ┌────┴────────┐ ┌───────┴───────┐
│ SQLite │ │ DO Spaces │ │ Payment │ │ DO Spaces │
│ DB │ │ (CDN) │ │ Providers │ │ + User S3 │
│ │ │ │ │ │ │ + Mirror S3 │
│ models │ │ media │ │ Stripe │ │ │
│ sessions│ │ thumbs │ │ PayPal │ │ Presigned │
│ signals │ │ assets │ │ Crypto │ │ URLs only │
└─────────┘ └────────────┘ └──────────────┘ └───────────────┘
```
## Request Flow
```
Browser GET /s/{shop_id}/{slug}
│
├─ Pyramid route dispatch
│ └─ views/content.py or views/product.py
│ ├─ Query product + shop from SQLite
│ ├─ Compute discovery ring related products
│ ├─ Generate presigned URLs for media (15 min TTL)
│ └─ Render Jinja2 template
│
├─ Template layers:
│ ├─ base.j2 (theme, nav, conditional sandbox toolbar)
│ ├─ product.j2 / content.j2 (media, metadata, CTA)
│ ├─ snippets/related_content.j2 (ring sidebar)
│ ├─ snippets/comments.j2 (comment form + list)
│ └─ snippets/analytics.j2 (optional Plausible/GA)
│
├─ Client JS (progressive enhancement):
│ ├─ signals.js → anonymous beacon on unload
│ ├─ watch.js → SPA navigation (if watch_mode_enabled)
│ ├─ sandbox.js → filter toolbar (if sandbox_mode)
│ └─ comments.js → AJAX comment submission
│
└─ Media served from CDN via presigned URLs (never through uwsgi)
```
## Data Collection Pipeline
```
Page visit (browser)
│
│ signals.js collects:
│ - presence (wall_clock, visible, active ms)
│ - scroll (depth, direction changes)
│ - media (play, pause, seek, speed, completion)
│ - viewport width
│
▼
navigator.sendBeacon("/signals/beacon")
│ ~300 bytes JSON, one per page visit
│
▼
views/signals.py
│
├─ classify_referrer(Referer header)
│ → (class, domain, query) tuple
│ Stores: referrer_class, referrer_domain, referrer_query
│
├─ classify_device(viewport_width)
│ → 0=mobile, 1=tablet, 2=desktop
│
├─ Insert mps_page_session row
│
└─ If visible_ms >= 7000:
increment product.view_count
```
## Analytics Pipeline
```
mps_page_session (raw rows)
│
├─ views/analytics.py
│ │
│ ├─ Daily bucketing functions (28-day windows):
│ │ _daily_buckets() → view counts (bar chart)
│ │ _daily_avg_duration() → avg session duration
│ │ _daily_engagement() → engagement ratio
│ │ _daily_bounce_rate() → bounce rate
│ │ _daily_referrer_counts()→ external referrer volume
│ │
│ ├─ Ranked queries:
│ │ _top_referrer_domains() → top external domains
│ │ _top_referrer_queries() → top search engine queries
│ │ _top_search_keywords() → top internal search terms
│ │
│ └─ Aggregate queries:
│ Overview strip (7d)
│ Top products by views (7/14/21d)
│ Ring entry points
│ Engagement/attention leaders
│ Study material / background favorites
│ Traffic sources / device split
│
├─ templates/analytics.j2 (shop-level dashboard)
│ └─ SVG line_chart macro (server-rendered polyline)
│
└─ templates/analytics_product.j2 (per-product dashboard)
└─ SVG line_chart macro
```
## S3 Storage Architecture
```
┌─────────────────────────────────────────────────────────┐
│ MPS Main Bucket │
│ (DigitalOcean Spaces + CDN) │
│ │
│ {shop_id}/products/{product_id}/{filename} │
│ {shop_id}/products/{product_id}/thumb/{filename} │
│ {shop_id}/shop/{logo|banner} │
│ {shop_id}/products/{product_id}/karaoke/{filename} │
└───────────┬─────────────────────────────────┬────────────┘
│ │
Presigned URLs Mirror sync
(15 min TTL) (daemon thread)
│ │
▼ ▼
Browser / CDN ┌────────────────────┐
│ Shop Mirror Bucket │
│ (shop.has_s3_mirror)│
│ │
│ Same key structure │
│ Passive copy │
│ Backfill on setup │
└─────────────────────┘
┌─────────────────────────────────────────────────────────┐
│ User Artifact Bucket │
│ (user.has_s3_bucket) │
│ │
│ sandbox/{user_id}/{timestamp}-{filename} │
│ │
│ Presigned POST from /u/sandbox/upload │
│ Browser uploads directly (never through MPS server) │
│ 50MB max per file │
└─────────────────────────────────────────────────────────┘
```
## Feature Toggle Matrix
| Feature | Model Column | Form Section | Default |
|---------|-------------|--------------|---------|
| Watch mode | `shop.watch_mode_enabled` | `ribbon-settings` | Off |
| Sandbox mode | `shop.sandbox_mode` | `ribbon-settings` | Off |
| Show dates | `shop.show_dates` | `ribbon-settings` | On |
| Grid lanes | `shop.grid_lanes_enabled` | `ribbon-settings` | Off |
| Color filter | `shop.color_filter` | `ribbon-settings` | 0 (none) |
| Comments | `shop.comments_enabled` | `comment-settings` | On |
| Stripe | `shop.stripe_enabled` | `stripe-settings` | On |
| PayPal | `shop.paypal_*` | `paypal-settings` | Off |
| Crypto | `shop.monero_*` / `shop.dogecoin_*` | `crypto-settings` | Off |
| S3 mirror | `shop.mirror_s3_*` | `mirror-settings` | Off |
| Discovery ring | `shop.discovery_ring` | Automatic | Auto-computed |
| Subscriptions | `shop.subscription_*` | `ribbon-settings` | Off |
## Ticket Index
| Ticket | Title | Status |
|--------|-------|--------|
| [MPS-0](tickets/mps-0.md) | AJAX Comment Submission | Complete |
| [MPS-1](tickets/mps-1.md) | YouTube-Style Watch Experience | Complete |
| [MPS-2](tickets/mps-2.md) | Anonymous Signal Gathering & View Count | Complete |
| [MPS-3](tickets/mps-3.md) | Creator Analytics Dashboard | Complete |
| [MPS-4](tickets/mps-4.md) | Eliminate 502s from uWSGI Worker Recycling | Complete |
| [MPS-5](tickets/mps-5.md) | Investigate uWSGI Worker Memory Growth | Open |
| [MPS-6](tickets/mps-6.md) | Referrer Analytics — Domain, Query, Trend Lines | Complete |
| [MPS-7](tickets/mps-7.md) | Sandbox Mode — Creative Filter System | Complete |
| [MPS-8](tickets/mps-8.md) | User S3 Bucket + Artifact Storage | Complete |
| [MPS-9](tickets/mps-9.md) | Shop S3 Mirror Bucket | Complete |

2
docs/sandbox-mode.md
View file

@ -93,7 +93,7 @@ img.product-main
│
├──> Download (a.download = "sandbox-export.png")
│
└──> Upload (presigned POST to S3) [future]
└──> Upload (presigned POST to user's S3 bucket)
```
### Video Export

6
docs/tickets/mps-2.md
View file

@ -68,6 +68,8 @@ Piggyback on existing `timeupdate`, `play`, `pause`, `ended`, `seeked`,
| `is_ring_entry` | Boolean: full page load vs SPA transition | Front door vs ring navigation |
| `ring_position` | Integer: how many SPA transitions from entry | Depth in the ring |
| `referrer_class` | Enum: `direct`, `search`, `social`, `internal`, `unknown` | Where traffic comes from — derived from Referer header domain, NOT the full URL |
| `referrer_domain` | String (128 chars) | Referer hostname (e.g. "www.google.com") — added in MPS-6 |
| `referrer_query` | String (256 chars) | Search engine query parameter — added in MPS-6 |
| `device_class` | Enum: `mobile`, `tablet`, `desktop` | Screen size bucket only, not user agent — derived from viewport width sent in beacon |
`referrer_class` is computed server-side by matching the Referer header domain
@ -114,6 +116,8 @@ media_percent_played Float
is_ring_entry Boolean
ring_position SmallInteger
referrer_class SmallInteger (enum: 0=direct 1=search 2=social 3=internal 4=unknown)
referrer_domain Unicode(128) — extracted domain (e.g. "www.google.com")
referrer_query Unicode(256) — search engine query (e.g. "lo-fi beats")
device_class SmallInteger (enum: 0=mobile 1=tablet 2=desktop)
```
@ -283,7 +287,7 @@ from accidental double-fires, not tracking users across pages.
- Mouse/touch coordinates
- Clicked element identifiers
- Any form input content
- Referrer URLs (only domain classification)
- Referrer URLs (only domain + search query parameter — see MPS-6)
- Cross-page session linking (each page visit is an island)
A viewer could visit every product in a shop and there would be no way to

111
docs/tickets/mps-6.md Normal file
View file

@ -0,0 +1,111 @@
# MPS-6: Referrer Analytics — Domain, Query, and Trend Lines
## Problem
MPS-2 stored only a numeric `referrer_class` (0-4) on each page session. This
tells the creator "42% of traffic is from search" but not *which* search engine,
*which* social platform, or *what keywords* people searched. Creators need
actionable referrer intelligence to know where to spend their marketing energy.
The analytics dashboard also lacked trend visualization — all metrics were
point-in-time tables with no temporal context.
## Solution
### 1. Store referrer domain and search query
Add two columns to `mps_page_session`:
| Column | Type | Example |
|--------|------|---------|
| `referrer_domain` | `Unicode(128)` | `"www.google.com"`, `"twitter.com"` |
| `referrer_query` | `Unicode(256)` | `"lo-fi beats to study to"` |
These are populated by refactoring `classify_referrer()` from returning a single
integer to returning a `(class, domain, query)` tuple. The domain is extracted
from the Referer header URL. The query is parsed from search engine URL
parameters (`q=` for Google/DuckDuckGo/Bing, `p=` for Yahoo).
Privacy: the full Referer URL is never stored. Only the domain and the search
query parameter (if present) are kept. Social and unknown referrers store domain
only.
### 2. Add SVG line chart trend visualization
Server-rendered inline SVG polyline charts for temporal trends. No JavaScript
graph libraries — the charts are computed server-side and rendered as a Jinja2
macro (`line_chart`) that produces `<svg>` elements with `<polyline>` paths.
Five trend lines added to both shop-level and per-product analytics:
| Chart | Y-axis | Color |
|-------|--------|-------|
| Session duration | Avg `wall_clock_ms` (daily) | Blue |
| Engagement | Avg `active_ms / wall_clock_ms` (daily) | Green |
| Bounce rate | % sessions with `visible_ms < 7s` (daily) | Red |
| External referrers | Daily count of non-direct, non-internal sessions | Orange |
Each chart spans 28 days with one data point per day. Zero-days are filled so
the polyline is continuous.
### 3. Add keyword and referrer domain tables
Two new analytics sections:
**Top Referrer Domains** — ranked table of external domains driving traffic,
with session count and percentage bar. Excludes direct and internal traffic.
**Top Search Queries** — ranked table of search engine queries that led to the
shop, extracted from `referrer_query`. Also includes internal shop search
keywords from `mps_shop_search_request`.
### 4. Bucketing functions
Six new query functions in `analytics.py`:
- `_daily_buckets()` — daily view counts (bar chart)
- `_daily_avg_duration()` — daily avg session duration
- `_daily_engagement()` — daily avg engagement ratio
- `_daily_bounce_rate()` — daily bounce rate
- `_daily_referrer_counts()` — daily external referrer sessions
- `_top_referrer_domains()` — top N referrer domains
- `_top_referrer_queries()` — top N search engine queries
- `_top_search_keywords()` — top N internal search keywords
## classify_referrer() Refactor
Before (MPS-2):
```python
def classify_referrer(referrer, request_host):
"""Return 0-4 integer class."""
return 1 # search
```
After (MPS-6):
```python
def classify_referrer(referrer, request_host):
"""Return (class, domain, query) tuple."""
return (1, "www.google.com", "lo-fi beats")
```
Search engine query extraction:
- Google/Bing/DuckDuckGo: `?q=` parameter
- Yahoo: `?p=` parameter
- Other search engines: `?q=` fallback
## Files Changed
| File | Change |
|------|--------|
| `models/page_session.py` | Add `referrer_domain`, `referrer_query` columns |
| `views/signals.py` | Refactor `classify_referrer()` to return tuple; store domain + query |
| `views/analytics.py` | 8 new bucketing/query functions; pass trend data to templates |
| `templates/analytics.j2` | SVG `line_chart` macro; referrer domain table; keyword table |
| `templates/analytics_product.j2` | SVG `line_chart` macro; referrer domain table |
| `scripts/alembic/versions/f3086e09b052_*.py` | Migration: add `referrer_domain`, `referrer_query` |
| `tests/test_models.py` | 12 tests for `classify_referrer()` tuple return |
| `tests/test_functional.py` | Referrer trend chart rendering tests |
## Depends On
MPS-3 (analytics dashboard infrastructure)

63
docs/tickets/mps-7.md Normal file
View file

@ -0,0 +1,63 @@
# MPS-7: Sandbox Mode — Client-Side Creative Filter System
## Problem
Creators want visitors to interact with their media beyond passive viewing.
Existing tools require downloading, editing in external software, and
re-uploading. The friction kills experimentation. Creators need an in-browser
creative toolkit that lets visitors play with filters, export artifacts, and
optionally upload results — all without leaving the shop.
## Solution
A client-side filter engine that runs entirely in the browser. The server stores
a single boolean (`shop.sandbox_mode`). When enabled, every page in the shop
renders a floating toolbar with 32 filter presets, 7 adjustment sliders, SVG
filter effects, canvas export (image + video), and optional face detection via
MediaPipe.
Full architecture and filter reference: [docs/sandbox-mode.md](../sandbox-mode.md)
### Server-side (minimal)
- `sandbox_mode` Boolean column on Shop model
- Toggle in ribbon-settings form section (same form as announcement ribbon,
default theme, grid lanes, show_dates, watch_mode, color_filter)
- `base.j2` conditionally renders toolbar HTML, inline SVG filter definitions,
and loads `sandbox.js` only when `shop.sandbox_mode` is true
- Watch mode SPA integration: `watch.js` calls `window.sandboxReapply()` after
DOM swap to re-apply filters to newly loaded media
### Client-side (sandbox.js, ~700 lines)
- 32 CSS filter presets (4 basic, 4 warm, 4 cool, 4 dramatic, 5 color, 7
Instagram-style, 4 SVG)
- 7 adjustment sliders (brightness, contrast, saturation, hue, blur, sepia,
grayscale)
- Image export via canvas `toBlob()` (requires CORS from CDN)
- Video export via `MediaRecorder` + `captureStream(30fps)`
- Face detection via MediaPipe Face Mesh (4MB WASM, on-demand load)
- localStorage persistence of preset, slider values, panel state
- Upload-to-bucket via presigned POST (see MPS-8)
## Files Changed
| File | Change |
|------|--------|
| `models/shop.py` | `sandbox_mode` Boolean column |
| `views/shop.py` | Toggle handler in `ribbon-settings` form section |
| `templates/shop_settings.j2` | Enable/disable radio buttons |
| `templates/base.j2` | Conditional toolbar HTML + inline SVG defs |
| `static/js/sandbox.js` | Client-side filter engine (IIFE) |
| `static/js/watch.js` | `sandboxReapply()` hook in `updatePageContent()` |
| `static/css/common.css` | Toolbar layout (CSS Grid), mobile responsive |
| `scripts/alembic/versions/a2c13d3117f2_*.py` | Migration: `sandbox_mode` |
| `tests/test_functional.py` | Sandbox toggle, toolbar rendering, script tag tests |
## Depends On
Nothing. Independent feature.
## Blocks
MPS-8 (user S3 bucket for artifact upload)

92
docs/tickets/mps-8.md Normal file
View file

@ -0,0 +1,92 @@
# MPS-8: User S3 Bucket + Artifact Storage
## Problem
Sandbox mode (MPS-7) lets visitors export filtered images and videos, but
exports download to the user's local device. Creators want a way to
automatically upload sandbox artifacts to their own cloud storage — a personal
S3 bucket where filtered content accumulates without manual file management.
## Solution
### User-level S3 credentials
Add S3-compatible storage credentials to the User model (not Shop — a user may
own multiple shops but uses one storage bucket):
| Column | Type | Example |
|--------|------|---------|
| `s3_endpoint` | `Unicode(256)` | `"https://nyc3.digitaloceanspaces.com"` |
| `s3_region` | `Unicode(64)` | `"nyc3"` |
| `s3_bucket` | `Unicode(128)` | `"my-sandbox-exports"` |
| `s3_access_key` | `Unicode(128)` | `"DO00..."` |
| `s3_secret_key` | `Unicode(128)` | `"wJalr..."` |
Property `has_s3_bucket` returns True when endpoint, bucket, access_key, and
secret_key are all non-empty.
### Storage settings form
New section on user settings page (`/u/settings`) with an "Artifact Storage"
form that POSTs to `/u/settings/storage`. Fields: endpoint URL, region
(optional), bucket name, access key, secret key (password field).
### Presigned upload endpoint
`POST /u/sandbox/upload` — accepts `filename` and `content_type` parameters,
generates a presigned POST using the user's stored S3 credentials. Returns
`{presigned: {url, fields}, key}`. The browser uploads the blob directly to the
user's bucket — the file never touches the MPS server.
S3 key format: `sandbox/{user_id}/{timestamp}-{filename}`
Upload limit: 50MB per file (enforced via presigned POST conditions).
### Toolbar integration
When a user has `has_s3_bucket = True`, the sandbox toolbar in `base.j2`
renders with `data-has-bucket="1"`. The `sandbox.js` upload-to-bucket button
appears only when this attribute is present. The upload flow:
1. User exports image/video (canvas toBlob)
2. User clicks "Upload to Bucket"
3. JS POSTs to `/u/sandbox/upload` with filename + content_type
4. Server returns presigned POST URL + fields
5. JS uploads blob directly to user's S3 bucket via FormData POST
### Supported services
Any S3-compatible endpoint: DigitalOcean Spaces, AWS S3, MinIO, Backblaze B2,
Cloudflare R2.
### Security
Credentials stored as plain Unicode columns (same pattern as Stripe/PayPal/Adyen
keys on Shop model). Secret key field uses `type="password"` in the form.
Credentials are only used server-side to generate presigned URLs — they are
never exposed to the browser.
## Shop-Level S3 Mirror (MPS-9)
A separate but related feature: shops can configure their own S3 bucket as a
**mirror** of the MPS main bucket. Every file uploaded to a shop (product files,
thumbnails, shop assets) is automatically copied to the shop's mirror bucket in
a background thread. See [MPS-9](mps-9.md).
## Files Changed
| File | Change |
|------|--------|
| `models/user.py` | S3 credential columns + `has_s3_bucket` property |
| `views/user.py` | Storage settings POST handler + S3 fields in settings dict |
| `views/user_sandbox.py` | Presigned upload endpoint |
| `templates/user_settings.j2` | Artifact Storage form |
| `templates/base.j2` | `data-has-bucket` attribute on sandbox toolbar |
| `static/js/sandbox.js` | Upload-to-bucket button + presigned POST flow |
| `routes.py` | `user_storage_settings`, `user_sandbox_upload` routes |
| `scripts/alembic/versions/f898ba460612_*.py` | Migration: user S3 columns |
| `tests/test_functional.py` | S3 bucket save/clear, presigned upload, toolbar attribute tests |
## Depends On
MPS-7 (sandbox mode — toolbar must exist to add upload button)

92
docs/tickets/mps-9.md Normal file
View file

@ -0,0 +1,92 @@
# MPS-9: Shop S3 Mirror Bucket
## Problem
Creators want a copy of all their shop files in their own S3 bucket — for
backup, CDN flexibility, or migration away from MPS. Currently all files live
exclusively in the MPS DigitalOcean Spaces bucket.
## Solution
### Shop-level mirror credentials
Add S3-compatible mirror credentials to the Shop model:
| Column | Type | Example |
|--------|------|---------|
| `mirror_s3_endpoint` | `Unicode(256)` | `"https://nyc3.digitaloceanspaces.com"` |
| `mirror_s3_region` | `Unicode(64)` | `"nyc3"` |
| `mirror_s3_bucket` | `Unicode(128)` | `"my-shop-mirror"` |
| `mirror_s3_access_key` | `Unicode(128)` | `"DO00..."` |
| `mirror_s3_secret_key` | `Unicode(128)` | `"wJalr..."` |
Property `has_s3_mirror` returns True when all required fields are non-empty.
### Mirror sync engine (`lib/s3_mirror.py`)
Fire-and-forget sync: every file written to the MPS bucket is copied to the
shop's mirror bucket in a daemon thread. The MPS bucket remains the
origin/CDN — the mirror is a passive copy.
Key functions:
- `mirror_key()` — stream-copy a single key (get_object → put_object)
- `mirror_key_async()` — fire-and-forget single key in daemon thread
- `mirror_keys_async()` — fire-and-forget multiple keys in one thread
- `test_mirror_connection()` — validate credentials by listing bucket
- `backfill_mirror_async()` — double-fork detached process that copies all
existing shop files to the mirror (survives uWSGI worker recycling)
Thread safety: ORM objects are not accessed from background threads. All
credentials are captured as plain strings before thread creation. Each thread
creates its own boto3 client.
### Sync hooks
Mirror sync is triggered from:
- **Product upload** (`views/product.py`) — product file + thumbnail
- **Shop asset upload** (`views/shop.py`) — logo, banner
### Mirror settings form
New "Mirror Bucket" section in shop settings (`mirror-settings` form section):
- Endpoint URL, Region, Bucket, Access Key, Secret Key
- "Test Connection" — validates credentials on save
- "Backfill" toggle — triggers `backfill_mirror_async()` to copy all existing
files on first setup
### Backfill architecture
The backfill process needs to survive uWSGI worker recycling (workers get killed
at 512MB RSS). Solution: double-fork to fully detach from uWSGI:
```
Request handler
└─ fork() ─── intermediate child
└─ setsid() + fork() ─── grandchild (fully detached)
└─ fcntl.flock() guard
└─ create own SQLAlchemy engine
└─ list_objects_v2 + mirror_key loop
└─ os._exit(0)
```
One backfill per shop at a time (flock on `/tmp/s3_mirror_backfill_{shop_id}.lock`).
## Files Changed
| File | Change |
|------|--------|
| `models/shop.py` | Mirror S3 credential columns + `has_s3_mirror` property |
| `lib/s3_mirror.py` | Mirror engine: sync, async, test, backfill |
| `views/shop.py` | `mirror-settings` form handler + sync hooks |
| `views/product.py` | Mirror sync hooks on product upload |
| `templates/shop_settings.j2` | Mirror Bucket settings form |
| `routes.py` | No new routes (uses existing shop settings POST) |
| `scripts/alembic/versions/6b516114c393_*.py` | Migration: shop mirror S3 columns |
| `tests/test_functional.py` | Mirror settings save, connection test, backfill |
## Depends On
Nothing. Independent feature (but complements MPS-8 user S3 bucket).