# MPS Architecture Overview ## System Diagram ``` Internet | ┌────────┴────────┐ │ Caddy (HTTPS) │ │ reverse proxy │ │ :443 → :6001 │ └────────┬────────┘ | ┌────────┴────────┐ │ uWSGI │ │ 2 proc, 8 thr │ │ reload@512MB │ └────┬───────┬────┘ | | ┌──────────────┘ └──────────────┐ | | ┌────────┴────────┐ ┌────────┴────────┐ │ Pyramid / WSGI │ │ Background │ │ Request Cycle │ │ Threads │ │ │ │ │ │ views/ │ │ S3 mirror │ │ models/ │ │ Karaoke vocal │ │ templates/ │ │ isolation │ │ lib/ │ │ │ └───┬────┬────┬───┘ └────────┬────────┘ | | | | ┌─────────┘ | └─────────┐ | | | | | ┌────┴────┐ ┌─────┴─────┐ ┌────┴────────┐ ┌───────┴───────┐ │ SQLite │ │ DO Spaces │ │ Payment │ │ DO Spaces │ │ DB │ │ (CDN) │ │ Providers │ │ + User S3 │ │ │ │ │ │ │ │ + Mirror S3 │ │ models │ │ media │ │ Stripe │ │ │ │ sessions│ │ thumbs │ │ PayPal │ │ Presigned │ │ signals │ │ assets │ │ Crypto │ │ URLs only │ │ │ │ │ │ Gift Cards │ │ │ └─────────┘ └────────────┘ └──────────────┘ └───────────────┘ ``` ## Request Flow ``` Browser GET /s/{shop_id}/{slug} │ ├─ Pyramid route dispatch │ └─ views/content.py or views/product.py │ ├─ Query product + shop from SQLite │ ├─ Compute discovery ring related products │ ├─ Generate presigned URLs for media (15 min TTL) │ └─ Render Jinja2 template │ ├─ Template layers: │ ├─ base.j2 (theme, nav, conditional sandbox toolbar) │ ├─ product.j2 / content.j2 (media, metadata, CTA) │ ├─ snippets/related_content.j2 (ring sidebar) │ ├─ snippets/comments.j2 (comment form + list) │ └─ snippets/analytics.j2 (optional Plausible/GA) │ ├─ Client JS (progressive enhancement): │ ├─ signals.js → anonymous beacon on unload │ ├─ watch.js → SPA navigation (if watch_mode_enabled) │ ├─ sandbox.js → filter toolbar (if sandbox_mode) │ └─ comments.js → AJAX comment submission │ └─ Media served from CDN via presigned URLs (never through uwsgi) ``` ## Data Collection Pipeline ``` Page visit (browser) │ │ signals.js collects: │ - presence (wall_clock, visible, active ms) │ - scroll (depth, direction changes) │ - media (play, pause, seek, speed, completion) │ - viewport width │ ▼ navigator.sendBeacon("/signals/beacon") │ ~300 bytes JSON, one per page visit │ ▼ views/signals.py │ ├─ classify_referrer(Referer header) │ → (class, domain, query) tuple │ Stores: referrer_class, referrer_domain, referrer_query │ ├─ classify_device(viewport_width) │ → 0=mobile, 1=tablet, 2=desktop │ ├─ Insert mps_page_session row │ └─ If visible_ms >= 7000: increment product.view_count ``` ## Analytics Pipeline ``` mps_page_session (raw rows) │ ├─ views/analytics.py │ │ │ ├─ Daily bucketing functions (28-day windows): │ │ _daily_buckets() → view counts (bar chart) │ │ _daily_avg_duration() → avg session duration │ │ _daily_engagement() → engagement ratio │ │ _daily_bounce_rate() → bounce rate │ │ _daily_referrer_counts()→ external referrer volume │ │ │ ├─ Ranked queries: │ │ _top_referrer_domains() → top external domains │ │ _top_referrer_queries() → top search engine queries │ │ _top_search_keywords() → top internal search terms │ │ │ └─ Aggregate queries: │ Overview strip (7d) │ Top products by views (7/14/21d) │ Ring entry points │ Engagement/attention leaders │ Study material / background favorites │ Traffic sources / device split │ ├─ templates/analytics.j2 (shop-level dashboard) │ └─ SVG line_chart macro (server-rendered polyline) │ └─ templates/analytics_product.j2 (per-product dashboard) └─ SVG line_chart macro ``` ## S3 Storage Architecture ``` ┌─────────────────────────────────────────────────────────┐ │ MPS Main Bucket │ │ (DigitalOcean Spaces + CDN) │ │ │ │ {shop_id}/products/{product_id}/{filename} │ │ {shop_id}/products/{product_id}/thumb/{filename} │ │ {shop_id}/shop/{logo|banner} │ │ {shop_id}/products/{product_id}/karaoke/{filename} │ └───────────┬─────────────────────────────────┬────────────┘ │ │ Presigned URLs Mirror sync (15 min TTL) (daemon thread) │ │ ▼ ▼ Browser / CDN ┌────────────────────┐ │ Shop Mirror Bucket │ │ (shop.has_s3_mirror)│ │ │ │ Same key structure │ │ Passive copy │ │ Backfill on setup │ └─────────────────────┘ ┌─────────────────────────────────────────────────────────┐ │ Shop Primary Bucket (BYOB) │ │ (shop.has_primary_s3 — MPS-16) │ │ │ │ When enabled, REPLACES MPS Main Bucket for this shop: │ │ - All presigned URLs use shop's S3 client │ │ - All CDN URLs use shop's cdn_endpoint │ │ - request.shop_uploads_client / shop_bucket_name / │ │ shop_cdn_endpoint fall back to MPS default when off │ │ │ │ Configured via bucket-settings form section │ └─────────────────────────────────────────────────────────┘ ┌─────────────────────────────────────────────────────────┐ │ User Artifact Bucket │ │ (user.has_s3_bucket) │ │ │ │ sandbox/{user_id}/{timestamp}-{filename} │ │ │ │ Presigned POST from /u/sandbox/upload │ │ Browser uploads directly (never through MPS server) │ │ 50MB max per file │ └─────────────────────────────────────────────────────────┘ ``` ## Feature Toggle Matrix | Feature | Model Column | Form Section | Default | |---------|-------------|--------------|---------| | Watch mode | `shop.watch_mode_enabled` | `ribbon-settings` | Off | | Sandbox mode | `shop.sandbox_mode` | `ribbon-settings` | Off | | Show dates | `shop.show_dates` | `ribbon-settings` | On | | Grid lanes | `shop.grid_lanes_enabled` | `ribbon-settings` | Off | | Color filter | `shop.color_filter` | `ribbon-settings` | 0 (none) | | Comments | `shop.comments_enabled` | `comment-settings` | On | | Stripe | `shop.stripe_enabled` | `stripe-settings` | On | | PayPal | `shop.paypal_*` | `paypal-settings` | Off | | Crypto | `shop.monero_*` / `shop.dogecoin_*` | `crypto-settings` | Off | | Gift cards | `shop.gift_card_enabled` | `gift-card-settings` | Off | | S3 mirror | `shop.mirror_s3_*` | `mirror-settings` | Off | | BYOB (primary S3) | `shop.primary_s3_*` | `bucket-settings` | Off | | Environment | `shop.environment` | `environment-settings` | 0 (production) | | Trial | `shop.trial_started_timestamp` | Auto on creation | 21 days | | Discovery ring | `shop.discovery_ring` | Automatic | Auto-computed | | Subscriptions | `shop.subscription_*` | `ribbon-settings` | Off | ## Ticket Index | Ticket | Title | Status | |--------|-------|--------| | [MPS-0](tickets/mps-0.md) | AJAX Comment Submission | Complete | | [MPS-1](tickets/mps-1.md) | YouTube-Style Watch Experience | Complete | | [MPS-2](tickets/mps-2.md) | Anonymous Signal Gathering & View Count | Complete | | [MPS-3](tickets/mps-3.md) | Creator Analytics Dashboard | Complete | | [MPS-4](tickets/mps-4.md) | Eliminate 502s from uWSGI Worker Recycling | Complete | | [MPS-5](tickets/mps-5.md) | Investigate uWSGI Worker Memory Growth | Open | | [MPS-6](tickets/mps-6.md) | Referrer Analytics — Domain, Query, Trend Lines | Complete | | [MPS-7](tickets/mps-7.md) | Sandbox Mode — Creative Filter System | Complete | | [MPS-8](tickets/mps-8.md) | User S3 Bucket + Artifact Storage | Complete | | [MPS-9](tickets/mps-9.md) | Shop S3 Mirror Bucket | Complete | | [MPS-10](tickets/mps-10.md) | Gift Card — Models & Migration | Complete | | [MPS-11](tickets/mps-11.md) | Gift Card — Purchase Flow | Complete | | [MPS-12](tickets/mps-12.md) | Gift Card — Redemption at Checkout | Complete | | [MPS-13](tickets/mps-13.md) | Gift Card — Shop Admin & Settings | Complete | | [MPS-14](tickets/mps-14.md) | Shop Environment — Dev & Stage Shops | Complete | | [MPS-15](tickets/mps-15.md) | 21-Day Free Trial | Complete | | [MPS-16](tickets/mps-16.md) | Bring Your Own Bucket (BYOB) | Complete | ## Related Docs | Doc | Purpose | |-----|---------| | [Design System](design-system.md) | Design tokens, CSS architecture, component library | | [JavaScript](JAVASCRIPT.md) | Client-side JS architecture | | [Sandbox Mode](sandbox-mode.md) | Creative filter system | | [Testing Performance](testing-performance.md) | Test suite optimization |