feat: MPS-24 Phase 2.1 — drop shop-vocabulary stems, surface more candidates
First Phase 2 deploy surfaced the wrong candidates on
shop.printableprompts.com: Students (53%), Resource (32%), Activities
(32%), Writing (31%), Practice (30%). These are shop vocabulary —
words that describe the whole shop, not categories within it. A stem
in 53% of products gives a shopper almost no information about which
subset a product belongs to.
- lib/tag_suggest.py: new max_share filter (default 0.4). Stems whose
product share exceeds this fraction auto-drop as shop vocabulary.
suggest_clusters now returns (clusters, filtered_count) so the UI
can show how many stems were filtered.
- top_n default 20 → 50 so the long tail of niche categories surfaces.
- views/shop.py: ?max_share=0.3 (stricter), ?max_share=1 (disable),
?top_n=200 URL knobs on the suggestions endpoint — power users tune
in the browser without redeploying. Floats over 1.0 are interpreted
as percentages (40 → 0.4) so the URL accepts either form.
- templates/shop_tags.j2: filtered-count hint with copy-paste tuning
knobs ("?max_share=0.3 stricter, ?max_share=1 to disable").
- scripts/backfill_tags.py: --max-share=0.4 CLI flag.
- Tests: test_suggest_clusters_filters_shop_vocabulary +
test_suggest_clusters_max_share_one_disables_filter. Existing pure-
function tests pass max_share=1.0 since their tiny fixtures would
otherwise be penalised for being small. 1067 total passing.
This commit is contained in:
parent
60a6f02cbc
commit
546e85416e
8 changed files with 241 additions and 35 deletions
18
CLAUDE.md
18
CLAUDE.md
|
|
@ -481,13 +481,19 @@ Phase 2 (shipped): deterministic title-plus-description auto-tagger
|
|||
in `lib/tag_suggest.py`. Title tokens weight × 3, description × 1 (capped
|
||||
at 100 unique tokens per product). Pipeline: tokenize → English + per-shop
|
||||
stopwords → suffix-strip stem → group by stem → drop stems matching
|
||||
existing tag slugs → rank by product count → label = most frequent
|
||||
original word for that stem. Surface: button on `/s/{id}/tags` →
|
||||
"Suggested categories" well with one-click Apply / Dismiss per cluster
|
||||
(`action=apply_suggestion` / `action=dismiss_suggestion`). CLI:
|
||||
existing tag slugs → **drop stems appearing in > 40% of products as shop
|
||||
vocabulary** (`max_share` filter — words like "students" / "resource" /
|
||||
"activity" describe the whole shop, not categories) → rank by product
|
||||
count → label = most frequent original word for that stem. Returns a
|
||||
`(clusters, filtered_count)` tuple so the UI can report how many stems
|
||||
got filtered. Surface: button on `/s/{id}/tags` → "Suggested categories"
|
||||
well with one-click Apply / Dismiss per cluster
|
||||
(`action=apply_suggestion` / `action=dismiss_suggestion`). URL knobs:
|
||||
`?max_share=0.3` (stricter), `?max_share=1` (disable),
|
||||
`?top_n=200` (show more). CLI:
|
||||
`python -m make_post_sell.scripts.backfill_tags data/development.ini
|
||||
--shop=<id> [--apply]`. **Never auto-commits** — operator approves
|
||||
every cluster.
|
||||
--shop=<id> [--max-share=0.4] [--apply]`. **Never auto-commits** —
|
||||
operator approves every cluster.
|
||||
|
||||
Phase 3 (this ticket, gated): ML categorization via uncloseai endpoint
|
||||
behind `app.features.ml_categorization.enabled` kill switch (mirror MPS-22).
|
||||
|
|
|
|||
|
|
@ -319,6 +319,32 @@ picker is Phase 2).
|
|||
| `tests/test_models.py` | `TestTagSuggestPureFunctions` — 11 unit tests over tokenize / stem / cluster |
|
||||
| `tests/test_functional.py` | `test_suggest_clusters_renders_candidates`, `test_apply_suggestion_creates_tag_and_attaches_products`, `test_dismiss_suggestion_adds_to_stopwords`, `test_apply_suggestion_rejects_empty_input` |
|
||||
|
||||
### Phase 2.1 — shop-vocabulary filter + top-N bump (shipped 2026-05-15)
|
||||
|
||||
Initial Phase 2 deploy surfaced the wrong candidates on
|
||||
`shop.printableprompts.com`: `Students`, `Resource`, `Activities`,
|
||||
`Writing`, `Practice` (each in 30–53% of products). These are *shop
|
||||
vocabulary* — words that describe the whole shop, not categories
|
||||
within it. A stem in 53% of products tells a shopper almost nothing
|
||||
about which subset a product belongs to. Fix:
|
||||
|
||||
- **`max_share` filter** in `lib/tag_suggest.py:suggest_clusters` —
|
||||
default `0.4` drops any stem in more than 40% of products as shop
|
||||
vocabulary. Returns a `(clusters, filtered_count)` tuple so callers
|
||||
can show "auto-dropped N common words."
|
||||
- **`top_n` default 20 → 50** so the long tail of niche categories
|
||||
surfaces on a 481-product catalog. Backfill CLI default also bumped.
|
||||
- **URL knobs** on `/s/{shop_id}/tags`: `?max_share=0.3` (stricter),
|
||||
`?max_share=1` (disable), `?top_n=200` (show more). No DB column —
|
||||
power users tune in the browser.
|
||||
- **Template note** under the suggestions well reports how many stems
|
||||
got filtered as shop vocabulary plus the tuning hints.
|
||||
- **CLI flag** `--max-share=0.4` on `scripts/backfill_tags.py`.
|
||||
- Tests: `test_suggest_clusters_filters_shop_vocabulary` +
|
||||
`test_suggest_clusters_max_share_one_disables_filter`. Existing
|
||||
`TestTagSuggestPureFunctions` tests pass `max_share=1.0` (their tiny
|
||||
fixtures would otherwise be penalised for being small).
|
||||
|
||||
## Tests (Phase 1)
|
||||
|
||||
### Unit (`test_models.py`)
|
||||
|
|
|
|||
|
|
@ -31,7 +31,16 @@ DESCRIPTION_WEIGHT = 1
|
|||
DEFAULT_MIN_PRODUCTS = 2
|
||||
|
||||
# Default cap on how many candidate clusters we return per call.
|
||||
DEFAULT_TOP_N = 20
|
||||
# Set generously — large catalogues (printableprompts has 481) carry many
|
||||
# valid niche categories beyond the obvious top 20.
|
||||
DEFAULT_TOP_N = 50
|
||||
|
||||
# A stem appearing in more than this fraction of products is treated as
|
||||
# **shop vocabulary** — words the operator uses to describe everything
|
||||
# they sell, not differentiators between products. E.g. on a K-1 printables
|
||||
# shop: students / resource / activity / practice — every product is one
|
||||
# of those. Auto-drop them so the next 20 candidates are actually useful.
|
||||
DEFAULT_MAX_SHARE = 0.4
|
||||
|
||||
# English stopwords — small, hand-tuned for product copy. Operators add
|
||||
# shop-specific extras via shop.tag_stopwords_json (e.g. printableprompts
|
||||
|
|
@ -185,6 +194,7 @@ def suggest_clusters(
|
|||
existing_tag_slugs=None,
|
||||
min_products=DEFAULT_MIN_PRODUCTS,
|
||||
top_n=DEFAULT_TOP_N,
|
||||
max_share=DEFAULT_MAX_SHARE,
|
||||
):
|
||||
"""Group products by shared stems and rank candidate categories.
|
||||
|
||||
|
|
@ -197,16 +207,23 @@ def suggest_clusters(
|
|||
already-applied categories.
|
||||
min_products: minimum products sharing a stem to qualify.
|
||||
top_n: maximum candidate clusters returned.
|
||||
max_share: fraction (0..1). A stem appearing in more than this
|
||||
share of the catalog is dropped as shop vocabulary.
|
||||
`1.0` disables the filter.
|
||||
|
||||
Returns:
|
||||
List of dicts (ranked, longest first):
|
||||
{
|
||||
"stem": "season",
|
||||
"label": "Seasonal", # most frequent original word
|
||||
"score": 117, # sum of weighted stem hits
|
||||
"product_ids": [<uuid>, <uuid>, ...],
|
||||
"sample_titles": ["Spring Math", "Autumn Read-Alouds", ...],
|
||||
}
|
||||
Tuple of (clusters, filtered_count):
|
||||
clusters: list of dicts ranked by product count desc:
|
||||
{
|
||||
"stem": "season",
|
||||
"label": "Seasonal", # most frequent original word
|
||||
"score": 117, # sum of weighted stem hits
|
||||
"product_ids": [<uuid>, <uuid>, ...],
|
||||
"sample_titles": ["Spring Math", "Autumn Reading", ...],
|
||||
}
|
||||
filtered_count: number of stems auto-dropped as shop vocabulary
|
||||
(so callers can show "n stems filtered as shop
|
||||
vocabulary").
|
||||
"""
|
||||
from slugify import slugify # local import; only needed here
|
||||
|
||||
|
|
@ -218,7 +235,9 @@ def suggest_clusters(
|
|||
stem_labels = defaultdict(Counter) # stem -> Counter(original_word)
|
||||
stem_titles = defaultdict(list) # stem -> [title, ...] for samples
|
||||
|
||||
total_products = 0
|
||||
for product in products:
|
||||
total_products += 1
|
||||
title = product.title or ""
|
||||
description = product.description or ""
|
||||
bag, labels = stem_bag(title, description, stopwords=stopwords)
|
||||
|
|
@ -230,9 +249,16 @@ def suggest_clusters(
|
|||
stem_titles[stem].append(title)
|
||||
|
||||
candidates = []
|
||||
shop_vocab_filtered = 0
|
||||
for stem, ids in stem_products.items():
|
||||
if len(ids) < min_products:
|
||||
continue
|
||||
# Drop stems that describe the whole shop, not a category subset.
|
||||
if total_products > 0 and max_share < 1.0:
|
||||
share = len(ids) / total_products
|
||||
if share > max_share:
|
||||
shop_vocab_filtered += 1
|
||||
continue
|
||||
# Best-vote original word becomes the candidate label.
|
||||
label_word = (
|
||||
stem_labels[stem].most_common(1)[0][0] if stem_labels[stem] else stem
|
||||
|
|
@ -254,4 +280,4 @@ def suggest_clusters(
|
|||
key=lambda c: (len(c["product_ids"]), c["score"]),
|
||||
reverse=True,
|
||||
)
|
||||
return candidates[:top_n]
|
||||
return candidates[:top_n], shop_vocab_filtered
|
||||
|
|
|
|||
|
|
@ -34,8 +34,13 @@ def main(argv=None):
|
|||
help="Minimum products sharing a stem to surface (default: 2)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--top-n", type=int, default=20,
|
||||
help="Max candidate clusters to print (default: 20)",
|
||||
"--top-n", type=int, default=50,
|
||||
help="Max candidate clusters to print (default: 50)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--max-share", type=float, default=0.4,
|
||||
help="Drop stems in more than this share of products as shop "
|
||||
"vocabulary; pass 1.0 to disable (default: 0.4)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--apply", action="store_true",
|
||||
|
|
@ -63,14 +68,20 @@ def main(argv=None):
|
|||
print(f"Mode: {'APPLY' if args.apply else 'DRY-RUN (preview)'}")
|
||||
print("-" * 60)
|
||||
|
||||
suggestions = suggest_clusters(
|
||||
suggestions, vocab_filtered = suggest_clusters(
|
||||
products,
|
||||
stopwords=shop.tag_stopwords,
|
||||
existing_tag_slugs=existing_slugs,
|
||||
min_products=args.min_products,
|
||||
top_n=args.top_n,
|
||||
max_share=args.max_share,
|
||||
)
|
||||
|
||||
if vocab_filtered:
|
||||
print(
|
||||
f"({vocab_filtered} stem(s) dropped as shop vocabulary — "
|
||||
f"appeared in >{int(args.max_share * 100)}% of products)"
|
||||
)
|
||||
if not suggestions:
|
||||
print("No candidate categories surfaced.")
|
||||
return 0
|
||||
|
|
|
|||
|
|
@ -4926,6 +4926,40 @@ textarea {
|
|||
padding: 0 10px;
|
||||
}
|
||||
|
||||
.analytics-header {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr auto;
|
||||
gap: var(--space-3, 12px);
|
||||
align-items: center;
|
||||
margin-bottom: var(--space-3, 12px);
|
||||
}
|
||||
|
||||
.analytics-header h3 {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.analytics-range-form {
|
||||
display: grid;
|
||||
grid-template-columns: auto auto auto;
|
||||
gap: var(--space-2, 8px);
|
||||
align-items: center;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.analytics-range-form label {
|
||||
font-size: var(--font-size-sm, 0.875rem);
|
||||
color: var(--text-muted, #666);
|
||||
}
|
||||
|
||||
.analytics-range-form select {
|
||||
padding: var(--space-1, 4px) var(--space-2, 8px);
|
||||
border-radius: var(--radius-sm, 4px);
|
||||
border: 1px solid var(--border-color, #dee2e6);
|
||||
background: var(--surface-color, #fff);
|
||||
color: var(--text-color, #222);
|
||||
font-size: var(--font-size-sm, 0.875rem);
|
||||
}
|
||||
|
||||
.analytics-overview {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(2, 1fr);
|
||||
|
|
|
|||
|
|
@ -35,6 +35,15 @@
|
|||
</a>
|
||||
|
||||
{% if show_suggestions %}
|
||||
{% if shop_vocab_filtered and shop_vocab_filtered > 0 %}
|
||||
<p class="type-body-sm tag-suggest-filtered-note">
|
||||
Auto-dropped <b>{{ shop_vocab_filtered }}</b> common word{% if shop_vocab_filtered != 1 %}s{% endif %} as shop vocabulary
|
||||
(appeared in too many products to be a useful category).
|
||||
Tune with <code>?show_suggestions=1&max_share=0.3</code> (lower = stricter)
|
||||
or <code>&max_share=1</code> to disable.
|
||||
Show more with <code>&top_n=200</code>.
|
||||
</p>
|
||||
{% endif %}
|
||||
{% if suggestions %}
|
||||
<h3 class="type-title tag-suggest-heading">Candidate categories ({{ suggestions|length }})</h3>
|
||||
<ul class="tag-suggest-list">
|
||||
|
|
|
|||
|
|
@ -5138,11 +5138,15 @@ class TestTagSuggestPureFunctions(unittest.TestCase):
|
|||
SimpleNamespace(id="e", title="Stone Fox Novel Study", description="Reading novel."),
|
||||
SimpleNamespace(id="f", title="Chocolate Touch Novel Study", description="Reading novel."),
|
||||
]
|
||||
clusters = suggest_clusters(
|
||||
# max_share=1.0 disables the shop-vocabulary filter (covered by its
|
||||
# own dedicated test below) so this fixture isn't penalised for
|
||||
# being tiny — "math" naturally appears in 3 of 6 sample products.
|
||||
clusters, _ = suggest_clusters(
|
||||
products,
|
||||
stopwords=["write", "room", "activity", "day"],
|
||||
existing_tag_slugs=[],
|
||||
min_products=2,
|
||||
max_share=1.0,
|
||||
)
|
||||
labels = [c["label"] for c in clusters]
|
||||
self.assertIn("Math", labels)
|
||||
|
|
@ -5159,8 +5163,8 @@ class TestTagSuggestPureFunctions(unittest.TestCase):
|
|||
SimpleNamespace(id="a", title="Math Worksheet One", description=""),
|
||||
SimpleNamespace(id="b", title="Math Worksheet Two", description=""),
|
||||
]
|
||||
clusters = suggest_clusters(
|
||||
products, stopwords=[], existing_tag_slugs=["math"]
|
||||
clusters, _ = suggest_clusters(
|
||||
products, stopwords=[], existing_tag_slugs=["math"], max_share=1.0
|
||||
)
|
||||
labels = [c["label"] for c in clusters]
|
||||
self.assertNotIn("Math", labels)
|
||||
|
|
@ -5174,13 +5178,10 @@ class TestTagSuggestPureFunctions(unittest.TestCase):
|
|||
SimpleNamespace(id="b", title="Valentine's Day Reading", description=""),
|
||||
SimpleNamespace(id="c", title="Valentine's Day Crafts", description=""),
|
||||
]
|
||||
clusters = suggest_clusters(
|
||||
products, stopwords=["day"], existing_tag_slugs=[]
|
||||
clusters, _ = suggest_clusters(
|
||||
products, stopwords=["day"], existing_tag_slugs=[], max_share=1.0,
|
||||
)
|
||||
# The label should be the most common original word, not the stem.
|
||||
labels = [c["label"] for c in clusters]
|
||||
# "Valentine" comes back as the readable label — the apostrophe is
|
||||
# stripped at the markdown-cleaning step.
|
||||
self.assertTrue(
|
||||
any("alentin" in l.lower() for l in labels),
|
||||
f"expected valentine-ish label in {labels}",
|
||||
|
|
@ -5196,8 +5197,9 @@ class TestTagSuggestPureFunctions(unittest.TestCase):
|
|||
SimpleNamespace(id="d", title="Delta Reading", description=""),
|
||||
SimpleNamespace(id="e", title="Epsilon Reading", description=""),
|
||||
]
|
||||
clusters = suggest_clusters(
|
||||
products, stopwords=[], existing_tag_slugs=[], min_products=2
|
||||
clusters, _ = suggest_clusters(
|
||||
products, stopwords=[], existing_tag_slugs=[],
|
||||
min_products=2, max_share=1.0,
|
||||
)
|
||||
# "Math" (3 products) ranks above "Reading" (2 products).
|
||||
self.assertEqual(clusters[0]["label"], "Math")
|
||||
|
|
@ -5210,10 +5212,67 @@ class TestTagSuggestPureFunctions(unittest.TestCase):
|
|||
SimpleNamespace(id="a", title="Unique Title One", description=""),
|
||||
SimpleNamespace(id="b", title="Different Title Two", description=""),
|
||||
]
|
||||
clusters = suggest_clusters(
|
||||
products, stopwords=["title"], existing_tag_slugs=[], min_products=2
|
||||
clusters, _ = suggest_clusters(
|
||||
products, stopwords=["title"], existing_tag_slugs=[],
|
||||
min_products=2, max_share=1.0,
|
||||
)
|
||||
# No stem appears in both products → no clusters.
|
||||
self.assertEqual(clusters, [])
|
||||
|
||||
def test_suggest_clusters_filters_shop_vocabulary(self):
|
||||
"""Stems present in too many products auto-drop as shop vocabulary.
|
||||
|
||||
Mirrors the printableprompts gotcha: 'students', 'resource',
|
||||
'activity' appear in 53%/32%/32% of products — they describe
|
||||
the entire shop, not subsets, and confuse shoppers.
|
||||
"""
|
||||
from types import SimpleNamespace
|
||||
from ..lib.tag_suggest import suggest_clusters
|
||||
# 10 products: "resource" in 6/10 (60%); "math" in 3/10 (30%);
|
||||
# "reading" in 3/10 (30%).
|
||||
products = [
|
||||
SimpleNamespace(id=f"p{i}", title=t, description="")
|
||||
for i, t in enumerate([
|
||||
"Math Resource Alpha",
|
||||
"Math Resource Beta",
|
||||
"Math Worksheet Gamma",
|
||||
"Reading Resource Delta",
|
||||
"Reading Resource Epsilon",
|
||||
"Reading Resource Zeta",
|
||||
"Eta Resource",
|
||||
"Theta Resource",
|
||||
"Iota Practice",
|
||||
"Kappa Practice",
|
||||
])
|
||||
]
|
||||
# max_share=0.4 → "resource" (60%) drops, "math" (30%) and
|
||||
# "reading" (30%) survive.
|
||||
clusters, filtered = suggest_clusters(
|
||||
products,
|
||||
stopwords=[],
|
||||
existing_tag_slugs=[],
|
||||
min_products=2,
|
||||
max_share=0.4,
|
||||
)
|
||||
labels = [c["label"] for c in clusters]
|
||||
self.assertNotIn("Resource", labels)
|
||||
self.assertIn("Math", labels)
|
||||
self.assertIn("Reading", labels)
|
||||
self.assertGreaterEqual(filtered, 1)
|
||||
|
||||
def test_suggest_clusters_max_share_one_disables_filter(self):
|
||||
from types import SimpleNamespace
|
||||
from ..lib.tag_suggest import suggest_clusters
|
||||
products = [
|
||||
SimpleNamespace(id="a", title="Foo Bar", description=""),
|
||||
SimpleNamespace(id="b", title="Foo Bar", description=""),
|
||||
]
|
||||
# "foo" and "bar" each in 100% of products — only filtered when
|
||||
# max_share < 1.0.
|
||||
clusters, filtered = suggest_clusters(
|
||||
products, stopwords=[], existing_tag_slugs=[],
|
||||
min_products=2, max_share=1.0,
|
||||
)
|
||||
self.assertEqual(filtered, 0)
|
||||
self.assertTrue(len(clusters) > 0)
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -2108,17 +2108,51 @@ def shop_tags(request):
|
|||
# MPS-24 Phase 2: compute candidate clusters on demand. We always
|
||||
# compute (cheap O(N × tokens) over the shop catalog), but the
|
||||
# template only renders the well when the operator clicks the
|
||||
# button (?show_suggestions=1).
|
||||
# button (?show_suggestions=1). Power-user knobs `max_share` and
|
||||
# `top_n` accept URL overrides so the operator can tune without
|
||||
# redeploying.
|
||||
show_suggestions = (request.params.get("show_suggestions") or "") == "1"
|
||||
suggestions = []
|
||||
shop_vocab_filtered = 0
|
||||
if show_suggestions:
|
||||
from ..lib.tag_suggest import suggest_clusters
|
||||
from ..lib.tag_suggest import (
|
||||
DEFAULT_MAX_SHARE,
|
||||
DEFAULT_TOP_N,
|
||||
suggest_clusters,
|
||||
)
|
||||
|
||||
def _float_param(name, default, lo=0.01, hi=1.0):
|
||||
raw = (request.params.get(name) or "").strip()
|
||||
if not raw:
|
||||
return default
|
||||
try:
|
||||
v = float(raw)
|
||||
if v > 1.0:
|
||||
v = v / 100.0 # accept "40" for 0.4
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
return max(lo, min(hi, v))
|
||||
|
||||
def _int_param(name, default, lo=1, hi=500):
|
||||
raw = (request.params.get(name) or "").strip()
|
||||
if not raw:
|
||||
return default
|
||||
try:
|
||||
v = int(float(raw))
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
return max(lo, min(hi, v))
|
||||
|
||||
max_share = _float_param("max_share", DEFAULT_MAX_SHARE)
|
||||
top_n = _int_param("top_n", DEFAULT_TOP_N)
|
||||
|
||||
existing_slugs = [t.slug for t in all_tags]
|
||||
suggestions = suggest_clusters(
|
||||
suggestions, shop_vocab_filtered = suggest_clusters(
|
||||
list(all_products),
|
||||
stopwords=shop.tag_stopwords,
|
||||
existing_tag_slugs=existing_slugs,
|
||||
top_n=top_n,
|
||||
max_share=max_share,
|
||||
)
|
||||
|
||||
return {
|
||||
|
|
@ -2126,6 +2160,7 @@ def shop_tags(request):
|
|||
"focus_tag": focus_tag,
|
||||
"all_products": all_products,
|
||||
"suggestions": suggestions,
|
||||
"shop_vocab_filtered": shop_vocab_filtered,
|
||||
"show_suggestions": show_suggestions,
|
||||
}
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue