make_post_sell/docs/tickets/mps-19.md

MPS-19: BitTorrent / Magnet Link Mode — Diagnose & Fix Distribution

Status

BROKEN in production. Schema and code paths exist:

• mps_shop.torrent_enabled (Boolean) — shop-level toggle
• mps_product.torrent_opt_in (Boolean) — per-product opt-in
• mps_product.torrent_magnet_link + torrent_file_url columns
• make_post_sell/lib/torrent.py — bundle + .torrent generation pipeline
• Migrations 0ee5654cfe7d_* and c0236e351476_torrent_distribution_support.py
• Backfill route /s/{shop_id}/torrent-backfill-status

What does not work end-to-end: opting a public product in does not yield a populated magnet link or a bundle.torrent reachable on the CDN. Listeners never see a magnet button. Files never seed.

Problem

The torrent path has more moving parts than karaoke. Each is a candidate break point:

1. Opt-in only fires for visibility=1 (public). Off for unlisted/private.
2. Bundle assembly downloads source + thumbnails + description.md from S3 into a temp dir — a single missing fetch leaves an incomplete bundle
3. torf.Torrent builds multi-file .torrent with web seeds (BEP 19) and trackers (BEP 12) — wrong web-seed URL = no seeders forever
4. .torrent upload back to S3 at {s3_path}/bundle.torrent — must use shop-aware client, not MPS default
5. Magnet link derived from info hash + display name + trackers + web seeds
6. DB write of torrent_magnet_link + torrent_file_url is the user-visible commit — if it fails silently, every step before it was wasted
7. Free content vs paid product bundles differ — paid never includes the paid file (preview only). A copy-paste defect here = piracy via opt-in

Diagnosis Required

Same fail-loud principle as MPS-18. Add an MpsTorrentRun row per attempt.

Probe	Expected	Actual
Shop torrent_enabled=True	yes	?
Product torrent_opt_in=True AND visibility=1	yes	?
Bundle dir contains source + thumbs + description.md	full set	?
.torrent bytes generated by torf.Torrent	non-zero	?
.torrent uploaded to shop bucket at {s3_path}/bundle.torrent	object exists	?
Web seed URL in .torrent matches CDN URL of source file	exact match	?
torrent_magnet_link column populated with magnet:?xt=urn:btih:...	string	?
torrent_file_url column populated with CDN URL of .torrent	string	?
Watch JSON / content / product templates surface magnet button	rendered	?
Magnet link opens in BitTorrent client and pulls bundle	swarm fetches bundle	?

Proposal

1. Persist run state — MpsTorrentRun table

Column	Type	Purpose
id	UUID	row id
shop_id	UUID FK	shop
product_id	UUID FK	product
started_timestamp	int (ms)	run start
finished_timestamp	int (ms) nullable	run end
status	int	0=running 1=success 2=failed 3=cancelled
step	str	last completed step
error_message	UnicodeText nullable	failure detail
info_hash	str(40) nullable	hex info hash on success
bundle_bytes	int nullable	total bundle size

2. Verify paid-product safety

Add explicit unit test: paid product (is_sellable=True) bundle includes preview.{ext} and never the source file. Bundle assembly must read product.is_sellable and pick the right path. Single boolean defect here ships the paid file as freely-seedable. Test guards this.

3. Web seed URL correctness

The web seed URL inside .torrent is the CDN URL (public, no presigning). For shops using BYOB, that's request.shop_cdn_endpoint, not the MPS default CDN. Confirm lib/torrent.py uses the shop-aware endpoint when building the torrent metadata.

4. Backfill UX

Existing /s/{shop_id}/torrent-backfill-status is a status endpoint. Add a backfill button to settings that POSTs form_section=backfill-torrent into views/shop.py. Concurrency-bound. Idempotent (skip products with populated torrent_magnet_link + fresh mtime).

5. UI surface

Magnet button on:

• templates/product.j2 — under download/CTA area
• templates/content.j2 — same position
• templates/snippets/related_content.j2 — small magnet icon next to free items
• Watch mode (watch.js updatePageContent, watch.py JSON)

Button visibility rules:

Visibility	Sellable	torrent_opt_in	Show magnet?
public (1)	False (free)	True	yes
public (1)	True (paid)	True	yes — bundle is preview only
unlisted (2)	any	any	no
private (0)	any	any	no

6. Tracker / web-seed strategy

Default DEFAULT_TRACKERS list in lib/torrent.py is acceptable but hardcoded — make it shop-configurable via a new column mps_shop.torrent_trackers (Text, newline-separated). NULL = use defaults. Sites that want their own tracker in front (private tracker, CDN-friendly tracker) can override.

Go-to-Market

Surface	Action
docs/torrent-distribution.md	New: end-to-end pipeline doc with dot diagram (mirror karaoke-pipeline.md)
docs/architecture.md	Add torrent run table + opt-in matrix to feature toggle matrix
/styleguide	Magnet button component + torrent health card
~/git/www.makepostsell.com/index.html	Feature card: "BitTorrent distribution. Web seeds. No friction. Free your bandwidth bill."
~/git/www.makepostsell.com/pricing.html	Note torrent included on plans that include uploads
Shop settings UI	Torrent health card + tracker config + backfill button

Files

File	Change
make_post_sell/models/torrent_run.py	New: MpsTorrentRun
make_post_sell/models/__init__.py	Import
make_post_sell/models/meta.py	Register
make_post_sell/models/shop.py	Add torrent_trackers (Text, nullable)
make_post_sell/lib/torrent.py	Persist run rows; shop-aware CDN; paid/free bundle branching with explicit tests
make_post_sell/views/shop.py	backfill-torrent form section handler; torrent health endpoint
make_post_sell/views/product.py	Trigger torrent generation on opt-in flip + visibility=public
make_post_sell/views/watch.py	Surface torrent_magnet_link + torrent_file_url in watch JSON
make_post_sell/routes.py	Add /s/{shop_id}/torrent-health
make_post_sell/templates/shop_settings.j2	Torrent health card + tracker config + backfill button
make_post_sell/templates/product.j2	Magnet button (server-rendered)
make_post_sell/templates/content.j2	Magnet button (server-rendered)
make_post_sell/templates/snippets/related_content.j2	Small magnet icon
make_post_sell/static/js/watch.js	updatePageContent swaps magnet href on SPA nav
make_post_sell/scripts/alembic/versions/XXXX_torrent_runs_and_trackers.py	Migration
make_post_sell/tests/test_models.py	MpsTorrentRun + visibility/opt-in matrix
make_post_sell/tests/test_integration.py	End-to-end bundle generation; paid vs free; BYOB
make_post_sell/tests/test_functional.py	Magnet button render rules; backfill POST; health endpoint auth

Tests

Unit (test_models.py)

• Product.show_magnet_button matrix — 8 combinations (visibility × sellable × opt_in)
• Product.torrent_bundle_paths returns preview path for paid, source for free
• MpsTorrentRun status helpers
• Shop.effective_trackers — falls back to DEFAULT_TRACKERS when column NULL

Integration (test_integration.py)

• Free content opt-in: bundle includes source + thumbs + description.md
• Paid product opt-in: bundle includes preview + thumbs + description.md, never source
• Web seed URL inside .torrent matches shop's CDN endpoint (BYOB-aware)
• Magnet link round-trip: torf parses our generated magnet and returns same info hash
• Backfill skips products with populated torrent_magnet_link + fresh mtime
• Failed run leaves status=failed + populated error_message; does not corrupt product columns

Functional (test_functional.py)

• Toggling torrent_opt_in on a public product fires generation (run row created)
• Toggling on private/unlisted product does not fire
• Magnet button renders on public product, not on private/unlisted
• Backfill button POST creates run rows for eligible products only
• Anon GET /s/{shop_id}/torrent-health → 403; owner → 200 JSON
• Watch JSON includes torrent_magnet_link + torrent_file_url when populated

Verification

1. source vars.sh && make test — all pass
2. Local dev: enable torrent_enabled on shop, opt-in a public free product, confirm bundle.torrent lands in bucket and magnet link populates
3. Open magnet link in a real BitTorrent client, watch it pull bundle from web seeds (no peers needed for first download)
4. Repeat for paid product — confirm bundle has preview file, not source
5. Force a failure (revoke S3 creds mid-run) — run row reflects failure
6. Push → CI green → deploy → bump GIT_HASH