# MPS Architecture Overview

## System Diagram

```
                                    Internet
                                       |
                              ┌────────┴────────┐
                              │  Caddy (HTTPS)   │
                              │  reverse proxy   │
                              │  :443 → :6001    │
                              └────────┬────────┘
                                       |
                              ┌────────┴────────┐
                              │   uWSGI          │
                              │   2 proc, 8 thr  │
                              │   reload@512MB   │
                              └────┬───────┬────┘
                                   |       |
                    ┌──────────────┘       └──────────────┐
                    |                                      |
           ┌────────┴────────┐                   ┌────────┴────────┐
           │  Pyramid / WSGI  │                   │   Background    │
           │  Request Cycle   │                   │   Threads       │
           │                  │                   │                 │
           │  views/          │                   │  S3 mirror      │
           │  models/         │                   │  Karaoke vocal  │
           │  templates/      │                   │  isolation      │
           │  lib/            │                   │                 │
           └───┬────┬────┬───┘                   └────────┬────────┘
               |    |    |                                 |
     ┌─────────┘    |    └─────────┐                      |
     |              |              |                      |
┌────┴────┐   ┌─────┴─────┐  ┌────┴────────┐    ┌───────┴───────┐
│ SQLite  │   │  DO Spaces │  │  Payment     │    │  DO Spaces    │
│ DB      │   │  (CDN)     │  │  Providers   │    │  + User S3    │
│         │   │            │  │              │    │  + Mirror S3  │
│ models  │   │  media     │  │  Stripe      │    │               │
│ sessions│   │  thumbs    │  │  PayPal      │    │  Presigned    │
│ signals │   │  assets    │  │  Crypto      │    │  URLs only    │
│         │   │            │  │  Gift Cards  │    │               │
└─────────┘   └────────────┘  └──────────────┘    └───────────────┘
```

## Request Flow

```
Browser GET /s/{shop_id}/{slug}
  │
  ├─ Pyramid route dispatch
  │   └─ views/content.py or views/product.py
  │       ├─ Query product + shop from SQLite
  │       ├─ Compute discovery ring related products
  │       ├─ Generate presigned URLs for media (15 min TTL)
  │       └─ Render Jinja2 template
  │
  ├─ Template layers:
  │   ├─ base.j2 (theme, nav, conditional sandbox toolbar)
  │   ├─ product.j2 / content.j2 (media, metadata, CTA)
  │   ├─ snippets/related_content.j2 (ring sidebar)
  │   ├─ snippets/comments.j2 (comment form + list)
  │   └─ snippets/analytics.j2 (optional Plausible/GA)
  │
  ├─ Client JS (progressive enhancement):
  │   ├─ signals.js → anonymous beacon on unload
  │   ├─ watch.js → SPA navigation (if watch_mode_enabled)
  │   ├─ sandbox.js → filter toolbar (if sandbox_mode)
  │   └─ comments.js → AJAX comment submission
  │
  └─ Media served from CDN via presigned URLs (never through uwsgi)
```

## Data Collection Pipeline

```
Page visit (browser)
  │
  │ signals.js collects:
  │  - presence (wall_clock, visible, active ms)
  │  - scroll (depth, direction changes)
  │  - media (play, pause, seek, speed, completion)
  │  - viewport width
  │
  ▼
navigator.sendBeacon("/signals/beacon")
  │  ~300 bytes JSON, one per page visit
  │
  ▼
views/signals.py
  │
  ├─ classify_referrer(Referer header)
  │   → (class, domain, query) tuple
  │   Stores: referrer_class, referrer_domain, referrer_query
  │
  ├─ classify_device(viewport_width)
  │   → 0=mobile, 1=tablet, 2=desktop
  │
  ├─ Insert mps_page_session row
  │
  └─ If visible_ms >= 7000:
      increment product.view_count
```

## Analytics Pipeline

```
mps_page_session (raw rows)
  │
  ├─ views/analytics.py
  │   │
  │   ├─ Daily bucketing functions (28-day windows):
  │   │   _daily_buckets()        → view counts (bar chart)
  │   │   _daily_avg_duration()   → avg session duration
  │   │   _daily_engagement()     → engagement ratio
  │   │   _daily_bounce_rate()    → bounce rate
  │   │   _daily_referrer_counts()→ external referrer volume
  │   │
  │   ├─ Ranked queries:
  │   │   _top_referrer_domains() → top external domains
  │   │   _top_referrer_queries() → top search engine queries
  │   │   _top_search_keywords()  → top internal search terms
  │   │
  │   └─ Aggregate queries:
  │       Overview strip (7d)
  │       Top products by views (7/14/21d)
  │       Ring entry points
  │       Engagement/attention leaders
  │       Study material / background favorites
  │       Traffic sources / device split
  │
  ├─ templates/analytics.j2 (shop-level dashboard)
  │   └─ SVG line_chart macro (server-rendered polyline)
  │
  └─ templates/analytics_product.j2 (per-product dashboard)
      └─ SVG line_chart macro
```

## S3 Storage Architecture

```
┌─────────────────────────────────────────────────────────┐
│                    MPS Main Bucket                        │
│            (DigitalOcean Spaces + CDN)                   │
│                                                          │
│  {shop_id}/products/{product_id}/{filename}              │
│  {shop_id}/products/{product_id}/thumb/{filename}        │
│  {shop_id}/shop/{logo|banner}                            │
│  {shop_id}/products/{product_id}/karaoke/{filename}      │
└───────────┬─────────────────────────────────┬────────────┘
            │                                 │
     Presigned URLs                    Mirror sync
     (15 min TTL)                   (daemon thread)
            │                                 │
            ▼                                 ▼
    Browser / CDN                 ┌────────────────────┐
                                  │  Shop Mirror Bucket │
                                  │  (shop.has_s3_mirror)│
                                  │                     │
                                  │  Same key structure  │
                                  │  Passive copy        │
                                  │  Backfill on setup   │
                                  └─────────────────────┘

┌─────────────────────────────────────────────────────────┐
│              Shop Primary Bucket (BYOB)                  │
│            (shop.has_primary_s3 — MPS-16)                │
│                                                          │
│  When enabled, REPLACES MPS Main Bucket for this shop:   │
│  - All presigned URLs use shop's S3 client               │
│  - All CDN URLs use shop's cdn_endpoint                  │
│  - request.shop_uploads_client / shop_bucket_name /      │
│    shop_cdn_endpoint fall back to MPS default when off   │
│                                                          │
│  Configured via bucket-settings form section             │
└─────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────┐
│                  User Artifact Bucket                     │
│               (user.has_s3_bucket)                        │
│                                                          │
│  sandbox/{user_id}/{timestamp}-{filename}                │
│                                                          │
│  Presigned POST from /u/sandbox/upload                   │
│  Browser uploads directly (never through MPS server)     │
│  50MB max per file                                       │
└─────────────────────────────────────────────────────────┘
```

## Feature Toggle Matrix

| Feature | Model Column | Form Section | Default |
|---------|-------------|--------------|---------|
| Watch mode | `shop.watch_mode_enabled` | `ribbon-settings` | Off |
| Sandbox mode | `shop.sandbox_mode` | `ribbon-settings` | Off |
| Show dates | `shop.show_dates` | `ribbon-settings` | On |
| Grid lanes | `shop.grid_lanes_enabled` | `ribbon-settings` | Off |
| Color filter | `shop.color_filter` | `ribbon-settings` | 0 (none) |
| Comments | `shop.comments_enabled` | `comment-settings` | On |
| Stripe | `shop.stripe_enabled` | `stripe-settings` | On |
| PayPal | `shop.paypal_*` | `paypal-settings` | Off |
| Crypto | `shop.monero_*` / `shop.dogecoin_*` | `crypto-settings` | Off |
| Gift cards | `shop.gift_card_enabled` | `gift-card-settings` | Off |
| S3 mirror | `shop.mirror_s3_*` | `mirror-settings` | Off |
| BYOB (primary S3) | `shop.primary_s3_*` | `bucket-settings` | Off |
| Environment | `shop.environment` | `environment-settings` | 0 (production) |
| Trial | `shop.trial_started_timestamp` | Auto on creation | 21 days |
| Discovery ring | `shop.discovery_ring` | Automatic | Auto-computed |
| Subscriptions | `shop.subscription_*` | `ribbon-settings` | Off |
| Karaoke (vocal isolation) | `shop.unsandbox_*_key` | `unsandbox-settings` | Off |

## Ticket Index

| Ticket | Title | Status |
|--------|-------|--------|
| [MPS-0](tickets/mps-0.md) | AJAX Comment Submission | Complete |
| [MPS-1](tickets/mps-1.md) | YouTube-Style Watch Experience | Complete |
| [MPS-2](tickets/mps-2.md) | Anonymous Signal Gathering & View Count | Complete |
| [MPS-3](tickets/mps-3.md) | Creator Analytics Dashboard | Complete |
| [MPS-4](tickets/mps-4.md) | Eliminate 502s from uWSGI Worker Recycling | Complete |
| [MPS-5](tickets/mps-5.md) | Investigate uWSGI Worker Memory Growth | Open |
| [MPS-6](tickets/mps-6.md) | Referrer Analytics — Domain, Query, Trend Lines | Complete |
| [MPS-7](tickets/mps-7.md) | Sandbox Mode — Creative Filter System | Complete |
| [MPS-8](tickets/mps-8.md) | User S3 Bucket + Artifact Storage | Complete |
| [MPS-9](tickets/mps-9.md) | Shop S3 Mirror Bucket | Complete |
| [MPS-10](tickets/mps-10.md) | Gift Card — Models & Migration | Complete |
| [MPS-11](tickets/mps-11.md) | Gift Card — Purchase Flow | Complete |
| [MPS-12](tickets/mps-12.md) | Gift Card — Redemption at Checkout | Complete |
| [MPS-13](tickets/mps-13.md) | Gift Card — Shop Admin & Settings | Complete |
| [MPS-14](tickets/mps-14.md) | Shop Environment — Dev & Stage Shops | Complete |
| [MPS-15](tickets/mps-15.md) | 21-Day Free Trial | Complete |
| [MPS-16](tickets/mps-16.md) | Bring Your Own Bucket (BYOB) | Complete |
| [MPS-17](tickets/mps-17.md) | REST API v1 — HMAC-signed product/content + upload | Open |
| [MPS-18](tickets/mps-18.md) | Karaoke Mode — Diagnose & Fix Vocal Isolation | Open (Broken in prod) |
| [MPS-19](tickets/mps-19.md) | BitTorrent / Magnet Link — Diagnose & Fix Distribution | Open (Broken in prod) |
| [MPS-20](tickets/mps-20.md) | Auction House Mode (eBay-style Bidding) | Complete |
| [MPS-21](tickets/mps-21.md) | Make-an-Offer Mode | Complete |
| [MPS-22](tickets/mps-22.md) | Kill-Switch Feature Flags — Karaoke + Torrent Off by Default | Complete |
| [MPS-23](tickets/mps-23.md) | Consolidated Transactional Sender Identity + Shop Contact Email | Open |

## Related Docs

| Doc | Purpose |
|-----|---------|
| [Design System](design-system.md) | Design tokens, CSS architecture, component library |
| [JavaScript](JAVASCRIPT.md) | Client-side JS architecture |
| [Karaoke Pipeline](karaoke-pipeline.md) | Vocal isolation: pipeline, on-demand, streaming architecture |
| [Sandbox Mode](sandbox-mode.md) | Creative filter system |
| [Auction House](auction-house.md) | MPS-20: state machine, bidding logic, soft-close, proxy |
| [Make-an-Offer](make-offer.md) | MPS-21: state machine, counter rounds, auto-accept/decline |
| [Testing Performance](testing-performance.md) | Test suite optimization |
