make_post_sell/docs/tickets/mps-24.md

MPS-24: Shop home page overhaul + product categorization

Status

PHASES 1 + 2 SHIPPED (2026-05-15). Tag model + chip strip + sectioned-lanes layout + bulk tagger live behind an opt-in home_layout selector (default 0 = flat = unchanged). Phase 2 adds a title-plus-description auto-tagger surfaced as one-click cluster apply in the bulk tagger UI + scripts/backfill_tags.py CLI. Phase 3 (uncloseai-backed ML categorization behind a kill switch) follows under this same ticket per CLAUDE.md "One Feature, One Ticket".

Background: operator feedback on shop.printableprompts.com flagged our default home page as the reason for considering a move to Shopify. We needed an opt-in home-page layout overhaul and a way to surface natural product categories so shoppers can browse a 481-item shop without scrolling a flat list.

Problem

shop.printableprompts.com is a digital-printables shop with 481 products, all K-1 classroom materials. Crawled 2026-05-15 from https://shop.printableprompts.com/sitemap.xml:

• 481 product pages, 2 shop pages, no tag/category pages (none exist).
• Natural groupings are obvious from titles alone: Math (Addition to 10, Counting to 100), Seasonal/Holiday (Valentine's Day, St. Patrick's Day, Christmas), Literacy (Little Red Hen, Frog and Toad), Science (Life Cycle of a Butterfly, Solar Eclipse), Novel Studies (Stone Fox, Chocolate Touch), Procedural Writing (How to Build a Snowman), Thematic Units (Thanksgiving Writing).
• Our home today renders a flat chronological grid with no way to filter, group, or jump to a topic.

Why an operator would reach for Shopify

Shopify shops get collections (operator-defined groups), automated collections (rule-based — "all products with tag X"), a sectioned home page template, collection lanes on home, a faceted product index, and tag-based search. None of that exists in MPS.

Current MPS home page

• Route: home / shop / shop_slug → views/shop.py:212 (home) + views/shop.py:226 (shop).
• Template: templates/home.j2 (shared by site root + merchant shop).
• Data: get_products_from_a_shop(shop, visibility=1) at models/product.py:730 — single query, ordered by updated_timestamp DESC, no grouping, no filtering, no pagination.
• Visible features: optional sales-stats banner, flat .serp grid, optional subscription CTA. That's it.

What we already have (do not rebuild)

Capability	Where	Notes
Visibility (public/private/unlisted)	Product.visibility (product.py:130)	Already filters home grid
Digital vs physical	Product.is_physical (product.py:157)	Binary, not a category
Sellable vs content	Product.is_sellable (product.py:154)	Blog post vs product
Pricing modes	Product.pricing_mode (product.py:137)	Fixed/auction/offer combos
Grid lanes (masonry)	Shop.grid_lanes_enabled (shop.py:71)	Layout polish only
Watch mode SPA	Shop.watch_mode_enabled (shop.py:150)	Sticky media SPA
Discovery ring (circular order)	Shop.json_discovery_ring (shop.py:216)	For watch mode SPA, not home
Full-text title search	views/shop.py:275 → get_products_by_keywords (product.py:740)	Title ilike, no tags, no description
Sandbox mode (creative filters)	Shop.sandbox_mode (shop.py:166)	Image filters — not categorization

What we do NOT have

• No Tag model. No Collection model. No Category table. No tag-style fields on Product. No tag-aware search.
• No LLM or embeddings infrastructure inside MPS (karaoke is audio ML routed to unsandbox; lib/sentiment.py is a rule-based comment scorer).
• No featured-item or hero columns on Shop.
• No browse routes beyond /search?keywords=. No /tag/X, /collection/X, /category/X.

Goals — fewest clicks to a purchase

Our checkout flows are done; what's missing is navigation into our catalog. Every design decision below optimises for: shopper lands → finds a relevant product → opens it → buys. Each extra click, extra page, or extra scroll between "land" and "open" is friction we cut.

1. Opt-in. Default home_layout = 0 (flat) keeps every existing shop pixel-identical.
2. A flipped-on shop with no tagging effort still produces a usable, grouped home page within minutes of opt-in — backfill must work without operator hand-labeling 481 products.
3. Shopper sees the grouping on land, not behind a click. Categories live above the fold; chip click filters in place (no page reload, no extra page).
4. Operator can correct mistakes — auto-grouping is never final state.
5. No new external dependencies on first ship. ML-assisted tagging stays pluggable, off-by-default, last phase.

Two orthogonal design dimensions

We have two independent decisions that combine to form the overhaul. Treating them as one decision is what makes the design feel huge — splitting them lets us ship Phase 1 in a week.

Dimension A — How products get grouped (categorization mechanism)

Option	Approach	Operator effort	Quality on day 1	Infra cost	Reversible?
A1 Manual tags	Operator types comma-separated tags per product (or via bulk admin)	High (481 products × a few seconds)	Perfect — operator picks	Tiny — Tag + product_tag table	Trivial
A2 Manual collections	Operator creates named collections, assigns products	Medium-High	Perfect	Small — Collection + collection_product table	Trivial
A3 Auto-tag from title keywords	Deterministic rules: tokenize title, strip stopwords, group by shared stems, emit top-N tags	Zero on backfill, low on new uploads (operator confirms suggested tag)	Decent — works very well for printableprompts because titles are descriptive	Tiny — pure Python, no external deps	Trivial
A4 Auto-tag via embeddings + clustering	Embed each product title+description, cluster via K-means or HDBSCAN, label clusters by centroid keyword	Zero	Better than A3 on shops with cryptic titles	Moderate — embedding lib + model file (~100MB) or external API	Trivial (re-cluster)
A5 Auto-tag via LLM	One-shot call per product: "pick 1–3 categories from this taxonomy"	Zero	Best	High — needs LLM API, retry/timeout/cost handling, kill-switch	Trivial (re-run)
A6 Hybrid (A3 or A5 → operator approve)	Auto-suggest tags on product edit form; operator one-click accepts	Zero baseline + low correction	Best — operator owns final state	Same as picked auto-method	Trivial

Reads from CLAUDE.md — "MPS uses one warm sending identity / one source of truth / one place per fact" — argues we should pick one storage for groupings and let mechanisms write into it. That storage is a Tag table. A1/A3/A5 all write tags. A2 (collections) is a different primitive that we may want on top of tags (a curated subset).

Dimension B — How groups render on home (layout)

Option	Layout	Shopper benefit	Implementation
B1 Sectioned home (Shopify-style lanes)	One horizontal lane per category, products scroll horizontally within each lane; lanes stacked vertically	Browse by topic at-a-glance, see ≤10 per category	New template; loop tags → query per tag (capped)
B2 Filter chips above flat grid	Existing grid stays; chip row at top (Math · Seasonal · Literacy · ...); clicking a chip filters the grid in place (no reload — JS optional)	Lightest visual change; preserves chronological signal	Existing template + chip strip + JS data-tag filter
B3 Featured + flat grid	Operator picks ≤6 "featured" products shown as large cards; rest of catalog underneath in current flat grid	No categorization needed; operator merchandises	New featured_product_ids JSON column; small template addition
B4 Sidebar nav	Left rail with category list; main pane shows filtered grid	Familiar pattern; bad on mobile (we have no sidebar pattern today)	Bigger template lift; mobile collapse
B5 Tag cloud + grid	Cloud at top sized by tag popularity, grid below	Discovery-flavoured; less directed than chips	Similar to B2 but visual variant
B6 Search-first	Big search bar hero, popular searches/tags chips under it, grid below	Best for shops with a known-item search pattern	Promote existing /search UI; needs popular-search data we already log in ShopSearchRequest

A and B compose. E.g. A6 + B1 = "auto-suggest tags with operator approval, rendered as sectioned lanes." A1 + B2 = "manual tags, filter chips." Both ship.

Phased implementation (all in this ticket)

Per CLAUDE.md "Ticket Scoping" — one feature, one ticket. Phases below land incrementally but live under one MPS-24 thread.

Phase 1 — Foundation: tags + filter chips on flat grid (A1 + B2)

Smallest ship that solves the printableprompts feedback. Cuts shopper clicks from "scroll 481 items" to "click chip, scan ~50, click product."

• New Tag model (id, shop_id, name, slug, created_timestamp).
• New product_tag association (composite PK product_id + tag_id).
• New Product.tags relationship (collection, not lazy=dynamic — small N per product).
• New Shop.home_layout Integer column, default 0:
  • 0 = flat (current behavior, unchanged)
  • 1 = filter chips on flat grid
  • 2 = sectioned lanes (Phase 2)
• New form section home-layout-settings in views/shop.py + shop_settings.j2.
• Tag editor: comma-separated input on product edit form (product_edit.j2) — splits, slugifies, upserts Tag rows scoped to shop.
• Bulk tag editor: small admin page at /s/{shop_id}/tags listing tags + product counts, click a tag → list of products with checkboxes to add/remove. (Avoids forcing operator into product-by-product.)
• Home template: if home_layout == 1, render chip strip from shop.tags_by_popularity() (top N, capped); chip click adds ?tag=<slug> to URL; server filters grid; JS enhancement does it in-place (zero navigation cost when JS is available).
• Tag detail route: /s/{shop_id}/tag/{slug} for crawlers + no-JS users (capability-driven presentation per CLAUDE.md).
• Filter chips also added to /search results so shopper can refine by tag after a keyword query (/search?keywords=X&tag=Y).

Phase 2 — Sectioned lanes shipped in Phase 1; auto-tag from title + description

Phase 1 already shipped sectioned-lane layout (home_layout == 2) — we brought it forward because rendering the lanes was a one-line template branch on top of the chip work. What remains for Phase 2 is the deterministic title + description auto-tagger so an operator with 481 untagged products gets a working categorization in one click.

Inputs:

• Product.title — full token weight × 3 (short, decisive, intentional).
• Product.description — raw markdown stripped of formatting, tokenised, weight × 1, capped at the first ~100 unique tokens per product so long blog posts don't drown short product copy.

Pipeline:

1. Tokenize title + description → lowercased words ≥ 3 chars.
2. Drop platform-default English stopwords + per-shop tag_stopwords_json overrides. For printableprompts that adds write, room, activity, the, etc.
3. Stem with a simple suffix-strip (no Porter port, no new dep) — seasonal/seasons/season → season.
4. Build per-stem product sets across the catalog.
5. Drop stems whose slug already exists as a shop tag (we don't re-suggest already-applied categories).
6. Keep stems carried by ≥ 2 products; rank by product count desc.
7. For each candidate stem, label = most frequent original word for that stem (so valentin displays as Valentine's, not valentin).

Surface:

• New "Suggest categories from titles + descriptions" button on /s/{shop_id}/tags. Renders a "Suggested categories" well listing each candidate cluster — label, sample product titles, product count.
• One-click apply per cluster — creates the tag + bulk-attaches every product in the cluster, all under one form POST.
• One-click dismiss per cluster — adds the stem's label to tag_stopwords_json so it never resurfaces.
• Standalone CLI scripts/backfill_tags.py --shop=<id> [--dry-run] for larger shops that prefer a terminal preview.

A6 hybrid — suggest, never auto-commit. Cluster output is rendered to the operator; nothing writes Tag / ProductTag rows until the operator clicks Apply.

No ML, no external deps. Pure Python over Product.title + Product.description. O(N × tokens) over a shop's catalog.

Phase 3 — ML-assisted categorization via uncloseai (A5)

Optional, off-by-default kill-switch (mirrors MPS-22 pattern): app.features.ml_categorization.enabled default False.

• Per-product call to uncloseai.com OpenAI-compatible endpoint we already operate — "pick 1–3 from this taxonomy (provided)."
• Cheaper than vendor LLMs because we run the endpoint ourselves.
• Same approve-don't-commit UX as Phase 2 — operator owns final state.
• Backfill script scripts/ml_tag_suggest.py runs over a shop's catalog, writes suggestions to a new tag_suggestion table (not product_tag), surfaces them in the bulk tagger for one-click accept.
• We do not ship embedding-clustering (A4) — A5 is cheaper to operate given our existing uncloseai infrastructure, and the operator-approval UX is identical so we don't need both.

Phase 3 lands behind the kill-switch even when shipped. Operator opt-in required.

Shop setting toggle (the operator-facing surface)

New form section home-layout-settings, added to the existing 19 sections in views/shop.py. UI lives in shop_settings.j2 alongside ribbon-settings.

Columns added to Shop

Column	Type	Default	Purpose
home_layout	Integer	0	0=flat, 1=filter_chips, 2=sectioned_lanes
home_layout_tag_limit	Integer	8	Max chips / lanes to show on home
home_layout_per_lane_limit	Integer	10	Max products per lane (B1)
featured_product_ids_json	UnicodeText	""	JSON list of UUIDs for optional B3 hero strip; nullable, opt-in

All server_default per CLAUDE.md SQLite migration rule.

Form UI (operator's view)

A single select for home_layout with previewable explanations:

• Flat grid (default) — every product, newest first. Same as today.
• Filter chips on flat grid — flat grid with a clickable category strip on top. Categories come from product tags.
• Sectioned by category — separate lanes per category, like a magazine rack. Best for shops with 50+ products in 4+ categories.

Plus three numeric fields (tag limit, per-lane limit, featured strip on/off). Plus a "Featured products" picker (Phase 1 ships the column + form, the rich picker is Phase 2).

Decisions (resolved at draft time — flag in review if fox disagrees)

1. Many tags per product, not single category. Matches printableprompts — a product can be both math and valentines.
2. Tags scoped per shop, not platform-wide. Avoids collision between unrelated shops (a music shop's blues ≠ a gardening shop's blues).
3. Tag bulk editor reachable from /actions/view as a new .mps-button in action-button-grid.
4. Mobile: chip strip horizontally scrolls; sectioned lanes stack as single-column below 800px (existing mobile reorder pattern in CLAUDE.md).
5. Watch mode uses discovery_ring once a shopper enters it — sectioned home is entry-page only, no SPA JSON shape change.
6. /search?keywords=X&tag=Y — tag filter on search results in Phase 1.
7. Stopwords: per-shop tag_stopwords_json override on top of a platform-wide default list.
8. Phase 2 + 3 are suggest-then-approve only — never auto-commit tags.
9. Naming: Tag not Category — tags are many-per-product and flat; categories would imply a tree we are not building.

Implementation (Phase 1 — shipped 2026-05-15)

File	Change
models/tag.py (new)	Tag model: id, shop_id, name, slug, created_timestamp; unique (shop_id, slug); helpers get_or_create_tag, tags_by_popularity
models/product_tag.py (new)	ProductTag many-to-many association with (product_id, tag_id) unique constraint
models/product.py	Add tags association_proxy
models/shop.py	Add home_layout, home_layout_tag_limit, home_layout_per_lane_limit, featured_product_ids_json, tag_stopwords_json columns + is_home_flat/is_home_chips/is_home_lanes/home_layout_label/featured_product_ids/tag_stopwords helpers + tags relationship
models/meta.py	Register Tag / ProductTag in CLASS_TO_TABLE
models/__init__.py	Import tag + product_tag modules
scripts/alembic/versions/882d68db47fa_mps_24_*.py (new)	Idempotent migration: creates mps_tag + mps_product_tag + 5 mps_shop columns; guards via _table_exists / _column_exists (CLAUDE.md pattern)
routes.py	Add shop_tags + shop_tag_detail before shop_slug catch-all
views/shop.py	_build_home_layout_context() helper; home-layout-settings form_section handler; shop_tag_detail + shop_tags (bulk tagger) views; tag filter param on home / shop / search views
views/product.py	Tag handling on product edit POST — comma-separated slugify + diff
templates/home.j2	Branch on shop.home_layout for chip strip / sectioned lanes / flat grid
templates/shop.j2	Same branching (used by /s/{id}/{slug})
templates/shop_settings.j2	New home-layout-settings section
templates/product_edit.j2	Comma-separated tag input
templates/shop_tag.j2 (new)	Tag detail page (works without JS)
templates/shop_tags.j2 (new)	Bulk tagger UI: list tags + apply/remove per product
templates/actions_view.j2	Add Tags shortcut to operator action grid
templates/styleguide.j2	Live tag-chip + tag-lane examples under #cards
static/css/common.css	.tag-chip-strip / .tag-chip / .tag-chip-active / .tag-lane / .tag-list / .tag-product-list styles — tokens only, Grid only
static/js/tag_filter.js (new)	Progressive enhancement: in-place chip filter via data-tag-slugs; falls back to server ?tag=
tests/test_models.py	TestShopHomeLayout + TestTagModel — 14 unit tests
tests/test_functional.py	TestHomeLayoutAndTags — 10 functional tests (settings save, tag editor, attach/detach, chip filter, tag detail)
docs/architecture.md	Add MPS-24 to feature matrix + ticket index
docs/design-system.md	Document MPS-24 chip + lane component classes
CLAUDE.md	New "Shop Home Layout + Tags" section + "Ticket Scoping — One Feature, One Ticket" rule
~/git/www.makepostsell.com/index.html + pricing.html	"Categorized Home Page" feature card + pricing list entry

Phase 2 — shipped 2026-05-15

File	Change
lib/tag_suggest.py (new)	Pure-function clusterer: tokenize, simple_stem, stem_bag, suggest_clusters over title (weight 3) + description (weight 1, capped at 100 unique tokens)
scripts/backfill_tags.py (new)	CLI: --shop=<id> previews suggestions; --apply creates tags + attaches products
views/shop.py	shop_tags view gained action=suggest, action=apply_suggestion, action=dismiss_suggestion; ?show_suggestions=1 triggers cluster compute
templates/shop_tags.j2	"Suggest categories from titles + descriptions" button + suggestions well with one-click Apply / Dismiss per cluster
static/css/common.css	.tag-suggest-list / .tag-suggest-item / .tag-suggest-actions styles
tests/test_models.py	TestTagSuggestPureFunctions — 11 unit tests over tokenize / stem / cluster
tests/test_functional.py	test_suggest_clusters_renders_candidates, test_apply_suggestion_creates_tag_and_attaches_products, test_dismiss_suggestion_adds_to_stopwords, test_apply_suggestion_rejects_empty_input

Phase 2.1 — shop-vocabulary filter + top-N bump (shipped 2026-05-15)

Initial Phase 2 deploy surfaced the wrong candidates on shop.printableprompts.com: Students, Resource, Activities, Writing, Practice (each in 30–53% of products). These are shop vocabulary — words that describe the whole shop, not categories within it. A stem in 53% of products tells a shopper almost nothing about which subset a product belongs to. Fix:

• max_share filter in lib/tag_suggest.py:suggest_clusters — default 0.4 drops any stem in more than 40% of products as shop vocabulary. Returns a (clusters, filtered_count) tuple so callers can show "auto-dropped N common words."
• top_n default 20 → 50 so the long tail of niche categories surfaces on a 481-product catalog. Backfill CLI default also bumped.
• URL knobs on /s/{shop_id}/tags: ?max_share=0.3 (stricter), ?max_share=1 (disable), ?top_n=200 (show more). No DB column — power users tune in the browser.
• Template note under the suggestions well reports how many stems got filtered as shop vocabulary plus the tuning hints.
• CLI flag --max-share=0.4 on scripts/backfill_tags.py.
• Tests: test_suggest_clusters_filters_shop_vocabulary + test_suggest_clusters_max_share_one_disables_filter. Existing TestTagSuggestPureFunctions tests pass max_share=1.0 (their tiny fixtures would otherwise be penalised for being small).

Tests (Phase 1)

Unit (test_models.py)

• Tag create/slugify/uniqueness-per-shop
• Product.tags collection add/remove
• Shop.home_layout defaults to 0; integer round-trip 0/1/2
• Shop.tags_by_popularity() returns shop-scoped tag list ordered by count
• Featured product ids JSON parse + roundtrip

Integration (test_integration.py)

• Operator saves tags on product edit → product_tag row written; comma split handles whitespace, dedupes, slugifies
• Bulk tagger add/remove flow
• home-layout-settings form_section save persists all four columns

Functional (test_functional.py)

• Shop home with home_layout=0 renders .serp flat grid, no chip strip
• Shop home with home_layout=1 renders chip strip + filterable grid
• ?tag=<slug> filters the grid server-side
• Tag detail page renders products with that tag only
• Bulk tagger page loads, POSTs persist
• Mobile chip strip horizontally scrolls (CSS check — render at <800px viewport via testbench)

Verification

1. source vars.sh && make test — all pass
2. Local: make serve, create a shop with 10 fake products, opt into home_layout=1, tag products math / seasonal, verify chip filter works
3. Local: opt out (home_layout=0), verify identical to current behavior
4. Push → CI green → Salt highstate → verify on my.makepostsell.com
5. Send shop link to printableprompts operator for feedback; if positive, plan Phase 2 (auto-tagger) as MPS-25

Out of scope (genuinely separate tickets later)

• Tag-aware search ranking (Phase 1 adds a tag filter to /search; tuning rank weights for tag matches vs title matches is its own ticket once we have shopper data).
• Faceted filtering (price range + tag + type combined) — wait for shopper signal after Phase 1.
• Cross-shop tag discovery (browse all shops by tag) — privacy question, defer.
• Tag-based RSS / sitemap segmentation — defer until tags exist for a few weeks and the segmentation use case is concrete.
• Tag tree / nested categories — explicitly not in scope, see decision #9.

References

• shop.printableprompts.com crawled 2026-05-15 via sitemap (481 products, obvious natural categories surfaced from titles)
• views/shop.py:212 (home), :226 (shop), :275 (search)
• models/product.py:730 (get_products_from_a_shop), :740 (get_products_by_keywords)
• templates/home.j2 (current flat grid)
• CLAUDE.md "Feature Kill Switches" pattern (MPS-22) — model for shop toggle
• CLAUDE.md "Capability-Driven Presentation" — tag detail page works without JS