fix: MPS-24 Phase 2.8 — bulk tagger AJAX tag-focus + real drag-to-reorder

Operator (printableprompts.com, 481 products) reported the bulk tagger
'still refreshing the whole screen' and 'dragging tags doesn't work'
after 2.7. Two real defects the 2.7 static audit missed:

1. Tag-focus was a full-page navigation: clicking a tag chip is
   <a href=?focus=slug>, and the view loaded+rendered ALL products on
   EVERY GET. On a 481-product catalog every tag click reloaded a
   multi-MB page. The forms were AJAX; the dominant workflow was not.
2. Drag-to-reorder never existed: shop_tags.j2 shipped draggable=true +
   a handle + help text, but tag_bulk.js had ZERO drag handlers.

Fix:
- shop.py:shop_tags — all_products loads only when focus_tag or
  show_suggestions (bare GET is light). New AJAX branch: is_ajax +
  ?focus=slug -> JSON {focus, products:[{id,title,url,attached}]}.
- shop_tags.j2 — stable [data-focus-section] (always in DOM, hidden
  until focused); ?focus= chips carry data-tag-focus-link. No-JS
  unchanged (real navigation, server renders the section).
- tag_bulk.js — wireFocusLinks() intercepts chip clicks, fetchFocus()
  + renderFocus() swap the list in place, active-chip + history
  pushState/popstate, real-navigation fallback. wireDragAndDrop()
  HTML5 DnD -> persistOrder() POSTs action=set_order&tag_slugs=…
  (view already supported it) + re-syncs up/down disabled states.
  .tag-list-dragging CSS added.
- Tests: TestProductTagsSpa +4 (ajax focus json, unknown-slug null,
  set_order persists positions, bare GET no catalog). 1128 passed.

Docs: mps-24.md Phase 2.8, architecture.md, design-system.md, CLAUDE.md.
Deferred: AJAX 'Suggest categories' link (occasional click, not hot path).

2026-05-16 08:31:03 -04:00

16 KiB

Raw Permalink Blame History

MPS Architecture Overview

System Diagram

                                    Internet
                                       |
                              ┌────────┴────────┐
                              │  Caddy (HTTPS)   │
                              │  reverse proxy   │
                              │  :443 → :6001    │
                              └────────┬────────┘
                                       |
                              ┌────────┴────────┐
                              │   uWSGI          │
                              │   2 proc, 8 thr  │
                              │   reload@512MB   │
                              └────┬───────┬────┘
                                   |       |
                    ┌──────────────┘       └──────────────┐
                    |                                      |
           ┌────────┴────────┐                   ┌────────┴────────┐
           │  Pyramid / WSGI  │                   │   Background    │
           │  Request Cycle   │                   │   Threads       │
           │                  │                   │                 │
           │  views/          │                   │  S3 mirror      │
           │  models/         │                   │  Karaoke vocal  │
           │  templates/      │                   │  isolation      │
           │  lib/            │                   │                 │
           └───┬────┬────┬───┘                   └────────┬────────┘
               |    |    |                                 |
     ┌─────────┘    |    └─────────┐                      |
     |              |              |                      |
┌────┴────┐   ┌─────┴─────┐  ┌────┴────────┐    ┌───────┴───────┐
│ SQLite  │   │  DO Spaces │  │  Payment     │    │  DO Spaces    │
│ DB      │   │  (CDN)     │  │  Providers   │    │  + User S3    │
│         │   │            │  │              │    │  + Mirror S3  │
│ models  │   │  media     │  │  Stripe      │    │               │
│ sessions│   │  thumbs    │  │  PayPal      │    │  Presigned    │
│ signals │   │  assets    │  │  Crypto      │    │  URLs only    │
│         │   │            │  │  Gift Cards  │    │               │
└─────────┘   └────────────┘  └──────────────┘    └───────────────┘

Request Flow

Browser GET /s/{shop_id}/{slug}
  │
  ├─ Pyramid route dispatch
  │   └─ views/content.py or views/product.py
  │       ├─ Query product + shop from SQLite
  │       ├─ Compute discovery ring related products
  │       ├─ Generate presigned URLs for media (15 min TTL)
  │       └─ Render Jinja2 template
  │
  ├─ Template layers:
  │   ├─ base.j2 (theme, nav, conditional sandbox toolbar)
  │   ├─ product.j2 / content.j2 (media, metadata, CTA)
  │   ├─ snippets/related_content.j2 (ring sidebar)
  │   ├─ snippets/comments.j2 (comment form + list)
  │   └─ snippets/analytics.j2 (optional Plausible/GA)
  │
  ├─ Client JS (progressive enhancement):
  │   ├─ signals.js → anonymous beacon on unload
  │   ├─ watch.js → SPA navigation (if watch_mode_enabled)
  │   ├─ sandbox.js → filter toolbar (if sandbox_mode)
  │   └─ comments.js → AJAX comment submission
  │
  └─ Media served from CDN via presigned URLs (never through uwsgi)

Data Collection Pipeline

Page visit (browser)
  │
  │ signals.js collects:
  │  - presence (wall_clock, visible, active ms)
  │  - scroll (depth, direction changes)
  │  - media (play, pause, seek, speed, completion)
  │  - viewport width
  │
  ▼
navigator.sendBeacon("/signals/beacon")
  │  ~300 bytes JSON, one per page visit
  │
  ▼
views/signals.py
  │
  ├─ classify_referrer(Referer header)
  │   → (class, domain, query) tuple
  │   Stores: referrer_class, referrer_domain, referrer_query
  │
  ├─ classify_device(viewport_width)
  │   → 0=mobile, 1=tablet, 2=desktop
  │
  ├─ Insert mps_page_session row
  │
  └─ If visible_ms >= 7000:
      increment product.view_count

Analytics Pipeline

mps_page_session (raw rows)
  │
  ├─ views/analytics.py
  │   │
  │   ├─ Daily bucketing functions (28-day windows):
  │   │   _daily_buckets()        → view counts (bar chart)
  │   │   _daily_avg_duration()   → avg session duration
  │   │   _daily_engagement()     → engagement ratio
  │   │   _daily_bounce_rate()    → bounce rate
  │   │   _daily_referrer_counts()→ external referrer volume
  │   │
  │   ├─ Ranked queries:
  │   │   _top_referrer_domains() → top external domains
  │   │   _top_referrer_queries() → top search engine queries
  │   │   _top_search_keywords()  → top internal search terms
  │   │
  │   └─ Aggregate queries:
  │       Overview strip (7d)
  │       Top products by views (7/14/21d)
  │       Ring entry points
  │       Engagement/attention leaders
  │       Study material / background favorites
  │       Traffic sources / device split
  │
  ├─ templates/analytics.j2 (shop-level dashboard)
  │   └─ SVG line_chart macro (server-rendered polyline)
  │
  └─ templates/analytics_product.j2 (per-product dashboard)
      └─ SVG line_chart macro

S3 Storage Architecture

┌─────────────────────────────────────────────────────────┐
│                    MPS Main Bucket                        │
│            (DigitalOcean Spaces + CDN)                   │
│                                                          │
│  {shop_id}/products/{product_id}/{filename}              │
│  {shop_id}/products/{product_id}/thumb/{filename}        │
│  {shop_id}/shop/{logo|banner}                            │
│  {shop_id}/products/{product_id}/karaoke/{filename}      │
└───────────┬─────────────────────────────────┬────────────┘
            │                                 │
     Presigned URLs                    Mirror sync
     (15 min TTL)                   (daemon thread)
            │                                 │
            ▼                                 ▼
    Browser / CDN                 ┌────────────────────┐
                                  │  Shop Mirror Bucket │
                                  │  (shop.has_s3_mirror)│
                                  │                     │
                                  │  Same key structure  │
                                  │  Passive copy        │
                                  │  Backfill on setup   │
                                  └─────────────────────┘

┌─────────────────────────────────────────────────────────┐
│              Shop Primary Bucket (BYOB)                  │
│            (shop.has_primary_s3 — MPS-16)                │
│                                                          │
│  When enabled, REPLACES MPS Main Bucket for this shop:   │
│  - All presigned URLs use shop's S3 client               │
│  - All CDN URLs use shop's cdn_endpoint                  │
│  - request.shop_uploads_client / shop_bucket_name /      │
│    shop_cdn_endpoint fall back to MPS default when off   │
│                                                          │
│  Configured via bucket-settings form section             │
└─────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────┐
│                  User Artifact Bucket                     │
│               (user.has_s3_bucket)                        │
│                                                          │
│  sandbox/{user_id}/{timestamp}-{filename}                │
│                                                          │
│  Presigned POST from /u/sandbox/upload                   │
│  Browser uploads directly (never through MPS server)     │
│  50MB max per file                                       │
└─────────────────────────────────────────────────────────┘

Feature Toggle Matrix

Feature	Model Column	Form Section	Default
Watch mode	`shop.watch_mode_enabled`	`ribbon-settings`	Off
Sandbox mode	`shop.sandbox_mode`	`ribbon-settings`	Off
Show price history (MPS-24 Phase 2.5)	`shop.show_price_history`	`ribbon-settings`	Off
Show dates	`shop.show_dates`	`ribbon-settings`	On
Grid lanes	`shop.grid_lanes_enabled`	`ribbon-settings`	Off
Color filter	`shop.color_filter`	`ribbon-settings`	0 (none)
Comments	`shop.comments_enabled`	`comment-settings`	On
Stripe	`shop.stripe_enabled`	`stripe-settings`	On
PayPal	`shop.paypal_*`	`paypal-settings`	Off
Crypto	`shop.monero_` / `shop.dogecoin_`	`crypto-settings`	Off
Gift cards	`shop.gift_card_enabled`	`gift-card-settings`	Off
S3 mirror	`shop.mirror_s3_*`	`mirror-settings`	Off
BYOB (primary S3)	`shop.primary_s3_*`	`bucket-settings`	Off
Environment	`shop.environment`	`environment-settings`	0 (production)
Trial	`shop.trial_started_timestamp`	Auto on creation	21 days
Discovery ring	`shop.discovery_ring`	Automatic	Auto-computed
Subscriptions	`shop.subscription_*`	`ribbon-settings`	Off
Karaoke (vocal isolation)	`shop.unsandbox_*_key`	`unsandbox-settings`	Off
Make-an-offer	`shop.offer_enabled` + `product.allow_offers`	`offer-settings`	Off
Offer expiration (hours)	`shop.offer_expiration_hours`	`offer-settings`	48
Offer pay window post-accept (hours)	`shop.offer_acceptance_payment_hours`	`offer-settings`	24
In-app notifications	(always on, no toggle — every txn email also drops a row)	n/a	On
Home page layout (MPS-24)	`shop.home_layout` (0=flat / 1=chips / 2=lanes)	`home-layout-settings`	0 (flat)
Tag chip / lane caps (MPS-24)	`shop.home_layout_tag_limit` / `shop.home_layout_per_lane_limit`	`home-layout-settings`	8 / 10
Featured products strip (MPS-24)	`shop.featured_product_ids_json`	`home-layout-settings`	empty
Product tags (MPS-24)	`Tag` + `ProductTag` association	product edit + `/s/{id}/tags` bulk editor	—
Tag auto-suggest (MPS-24 Phase 2)	`lib/tag_suggest.py` over `Product.title` + `Product.description`	`?show_suggestions=1` on `/s/{id}/tags` + `scripts/backfill_tags.py`	Never auto-applies
Tag-detail facet sidebar (MPS-24 Phase 2.6)	`shop_tag.j2` + `views/shop.py` `_price_range_from_request` / `_filter_by_price_range`	Sort + price range + categories list, GET form, sidebar ≥800px, top chip strip <800px	Always on for tag detail pages
6-sentence SERP excerpt (MPS-24 Phase 2.6)	`Product.excerpt_sentences(n=6, max_chars=1500)` via shared `_strip_markdown()` helper	`shop_tag.j2` row description	Always on
Facet nav on shop home (MPS-24 Phase 2.6b)	`_facet_nav.j2` macros + `home.j2` / `shop.j2` wrappers	Desktop sidebar + mobile `<details>` accordion when `shop.home_layout >= 1`	Opt-in via home_layout
Mobile SERP rows under each lane (MPS-24 Phase 2.6b)	`home.j2` / `shop.j2` `.tag-lane-rows` markup + CSS visibility swap	Horizontal tiles ≥800px; vertical SERP rows with 6-sentence excerpts <800px	On for layout 2
Per-product SPA tag chips (MPS-24 Phase 2.7)	`product_tags` view (`/p/{id}/tags`) + `static/js/product_tags.js` + `views/__init__.py:is_ajax()`	Chip add/remove on product edit, AJAX persist, no full page reload; no-JS keeps the comma `tags` field on the main form	Always on (JS-enhanced; no-JS fallback)
Bulk tagger AJAX focus + DnD (MPS-24 Phase 2.8)	`shop.py:shop_tags` AJAX `?focus=` branch + `tag_bulk.js` `wireFocusLinks`/`wireDragAndDrop`	Tag chip click swaps product list in place (no reload); HTML5 drag-to-reorder POSTs `set_order`; bare GET no longer loads the whole catalog	Always on (JS-enhanced; no-JS = real navigation)

Ticket Index

Ticket	Title	Status
MPS-0	AJAX Comment Submission	Complete
MPS-1	YouTube-Style Watch Experience	Complete
MPS-2	Anonymous Signal Gathering & View Count	Complete
MPS-3	Creator Analytics Dashboard	Complete
MPS-4	Eliminate 502s from uWSGI Worker Recycling	Complete
MPS-5	Investigate uWSGI Worker Memory Growth	Open
MPS-6	Referrer Analytics — Domain, Query, Trend Lines	Complete
MPS-7	Sandbox Mode — Creative Filter System	Complete
MPS-8	User S3 Bucket + Artifact Storage	Complete
MPS-9	Shop S3 Mirror Bucket	Complete
MPS-10	Gift Card — Models & Migration	Complete
MPS-11	Gift Card — Purchase Flow	Complete
MPS-12	Gift Card — Redemption at Checkout	Complete
MPS-13	Gift Card — Shop Admin & Settings	Complete
MPS-14	Shop Environment — Dev & Stage Shops	Complete
MPS-15	21-Day Free Trial	Complete
MPS-16	Bring Your Own Bucket (BYOB)	Complete
MPS-17	REST API v1 — HMAC-signed product/content + upload	Open
MPS-18	Karaoke Mode — Diagnose & Fix Vocal Isolation	Open (Broken in prod)
MPS-19	BitTorrent / Magnet Link — Diagnose & Fix Distribution	Open (Broken in prod)
MPS-20	Auction House Mode (eBay-style Bidding)	Complete
MPS-21	Make-an-Offer Mode	Complete
MPS-22	Kill-Switch Feature Flags — Karaoke + Torrent Off by Default	Complete
MPS-23	Consolidated Transactional Sender Identity + Shop Contact Email	Open
MPS-24	Shop home page overhaul + product categorization (tags + chips + lanes + auto-suggest + per-product SPA tag chips + bulk-tagger AJAX/DnD)	In progress (Phases 1 + 2 + 2.6 + 2.7 + 2.8 landed)

Doc	Purpose
Design System	Design tokens, CSS architecture, component library
Notifications	In-app notification table, kinds, and per-transition wiring
Auction House	MPS-20 — bidding, soft-close, payment deadline
Make Offer	MPS-21 — offer state machine, counter rounds, expiry
JavaScript	Client-side JS architecture
Karaoke Pipeline	Vocal isolation: pipeline, on-demand, streaming architecture
Sandbox Mode	Creative filter system
Auction House	MPS-20: state machine, bidding logic, soft-close, proxy
Make-an-Offer	MPS-21: state machine, counter rounds, auto-accept/decline
Testing Performance	Test suite optimization

16 KiB Raw Permalink Blame History

MPS Architecture Overview

System Diagram

Request Flow

Data Collection Pipeline

Analytics Pipeline

S3 Storage Architecture

Feature Toggle Matrix

Ticket Index

Related Docs

16 KiB

Raw Permalink Blame History