feat: MPS-24 Phase 2.1 — drop shop-vocabulary stems, surface more candidates
First Phase 2 deploy surfaced the wrong candidates on
shop.printableprompts.com: Students (53%), Resource (32%), Activities
(32%), Writing (31%), Practice (30%). These are shop vocabulary —
words that describe the whole shop, not categories within it. A stem
in 53% of products gives a shopper almost no information about which
subset a product belongs to.
- lib/tag_suggest.py: new max_share filter (default 0.4). Stems whose
product share exceeds this fraction auto-drop as shop vocabulary.
suggest_clusters now returns (clusters, filtered_count) so the UI
can show how many stems were filtered.
- top_n default 20 → 50 so the long tail of niche categories surfaces.
- views/shop.py: ?max_share=0.3 (stricter), ?max_share=1 (disable),
?top_n=200 URL knobs on the suggestions endpoint — power users tune
in the browser without redeploying. Floats over 1.0 are interpreted
as percentages (40 → 0.4) so the URL accepts either form.
- templates/shop_tags.j2: filtered-count hint with copy-paste tuning
knobs ("?max_share=0.3 stricter, ?max_share=1 to disable").
- scripts/backfill_tags.py: --max-share=0.4 CLI flag.
- Tests: test_suggest_clusters_filters_shop_vocabulary +
test_suggest_clusters_max_share_one_disables_filter. Existing pure-
function tests pass max_share=1.0 since their tiny fixtures would
otherwise be penalised for being small. 1067 total passing.
This commit is contained in:
parent
60a6f02cbc
commit
546e85416e
8 changed files with 241 additions and 35 deletions
|
|
@ -319,6 +319,32 @@ picker is Phase 2).
|
|||
| `tests/test_models.py` | `TestTagSuggestPureFunctions` — 11 unit tests over tokenize / stem / cluster |
|
||||
| `tests/test_functional.py` | `test_suggest_clusters_renders_candidates`, `test_apply_suggestion_creates_tag_and_attaches_products`, `test_dismiss_suggestion_adds_to_stopwords`, `test_apply_suggestion_rejects_empty_input` |
|
||||
|
||||
### Phase 2.1 — shop-vocabulary filter + top-N bump (shipped 2026-05-15)
|
||||
|
||||
Initial Phase 2 deploy surfaced the wrong candidates on
|
||||
`shop.printableprompts.com`: `Students`, `Resource`, `Activities`,
|
||||
`Writing`, `Practice` (each in 30–53% of products). These are *shop
|
||||
vocabulary* — words that describe the whole shop, not categories
|
||||
within it. A stem in 53% of products tells a shopper almost nothing
|
||||
about which subset a product belongs to. Fix:
|
||||
|
||||
- **`max_share` filter** in `lib/tag_suggest.py:suggest_clusters` —
|
||||
default `0.4` drops any stem in more than 40% of products as shop
|
||||
vocabulary. Returns a `(clusters, filtered_count)` tuple so callers
|
||||
can show "auto-dropped N common words."
|
||||
- **`top_n` default 20 → 50** so the long tail of niche categories
|
||||
surfaces on a 481-product catalog. Backfill CLI default also bumped.
|
||||
- **URL knobs** on `/s/{shop_id}/tags`: `?max_share=0.3` (stricter),
|
||||
`?max_share=1` (disable), `?top_n=200` (show more). No DB column —
|
||||
power users tune in the browser.
|
||||
- **Template note** under the suggestions well reports how many stems
|
||||
got filtered as shop vocabulary plus the tuning hints.
|
||||
- **CLI flag** `--max-share=0.4` on `scripts/backfill_tags.py`.
|
||||
- Tests: `test_suggest_clusters_filters_shop_vocabulary` +
|
||||
`test_suggest_clusters_max_share_one_disables_filter`. Existing
|
||||
`TestTagSuggestPureFunctions` tests pass `max_share=1.0` (their tiny
|
||||
fixtures would otherwise be penalised for being small).
|
||||
|
||||
## Tests (Phase 1)
|
||||
|
||||
### Unit (`test_models.py`)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue