fix: MPS-24 auto-suggest — keep umbrella unigrams (surface 'holiday' standalone)

Operator: sees 'December Holiday' but not 'Holiday' on its own as a
recommended tag.

Bigram supersession dropped a unigram when the UNION of all bigrams
containing it covered >=0.8 of its products. 'holiday' spans
'december holiday' + 'winter holiday' + 'christmas holiday' -> union
covered it -> 'holiday' hidden as redundant. Changed: supersede only
when a SINGLE bigram covers >=0.8 (a true fragment, e.g.
'write'->'write room'). An umbrella unigram covered only by the union
of DISTINCT bigrams is now KEPT as its own category.

Also answered operator Q in docs: the stem engine NEVER auto-applies
to new products — suggest-then-approve only (/s/{id}/tags button or
operator-run backfill_tags CLI). Adding a product does not
auto-categorize it.

Tests: test_suggest_clusters_multi_bigram_supersedes_unigram renamed
to ..._keeps_umbrella_unigram_over_multi_bigrams (behaviour
intentionally flipped per operator); +..._surfaces_holiday_with_phrase_bigrams;
single-dominant-bigram supersession test unchanged + still green.
1140 passed. Docs: mps-24.md Phase 2.8m.

docs/tickets/mps-24.md

@@ -363,6 +363,24 @@ Tests (`test_functional.py::TestProductTagsSpa`):
 Deferred (occasional click, not the hot path): AJAX-ifying the
 "Suggest categories" link — still a full navigation by design.
 
+**Phase 2.8m — supersession keeps umbrella unigrams** (shipped
+2026-05-17): operator saw "December Holiday" but not "Holiday" on its
+own. Bigram supersession dropped a unigram when the *union* of all
+bigrams containing it covered ≥0.8 of its products — so "holiday"
+(spanning "december holiday" + "winter holiday" + "christmas holiday")
+was hidden as redundant. Changed to supersede only when a **single**
+bigram covers ≥0.8 (true fragment, e.g. "write"→"write room"); an
+umbrella unigram covered only by the union of DISTINCT bigrams is now
+KEPT as its own category. Tests:
+`test_suggest_clusters_keeps_umbrella_unigram_over_multi_bigrams`
+(was `_multi_bigram_supersedes_unigram`, behaviour intentionally
+flipped), `test_suggest_clusters_surfaces_holiday_with_phrase_bigrams`;
+the single-dominant-bigram test still passes unchanged. Also clarified
+(operator Q): the stem engine **never auto-applies** to new products
+— it's suggest-then-approve only (`/s/{id}/tags` button or the
+operator-run `backfill_tags` CLI); adding a product does not
+auto-categorize it.
+
 **Phase 2.8l — SERP right rail (featured / random) + smaller thumbs**
 (shipped 2026-05-16): operator wanted smaller SERP thumbnails and a
 right column. Thumb column shrunk (140/200/260/320 → 88/110/130/150

make_post_sell/lib/tag_suggest.py

@@ -449,13 +449,19 @@ def suggest_clusters(
         reverse=True,
     )
 
-    # Bigram supersession: a unigram drops if a bigram (or collectively
-    # the union of bigrams) containing it covers ≥ SUPERSESSION_THRESHOLD
-    # of the unigram's product set. The operator wants a single phrase
-    # row ("Write Room") instead of three near-duplicates
-    # ("Write Room" + "Write" + "Room"), and a single "Day" row should
-    # drop when "Valentine's Day" + "Patrick's Day" + … collectively
-    # cover most of its products.
+    # Bigram supersession: a unigram drops ONLY when a *single* bigram
+    # already covers ≥ SUPERSESSION_THRESHOLD of its product set — i.e.
+    # the unigram is a fragment of one phrase ("write" → "write room";
+    # the operator wants one "Write Room" row, not "Write Room" + "Write"
+    # + "Room").
+    #
+    # An umbrella unigram covered only by the UNION of several DISTINCT
+    # bigrams (e.g. "holiday" → "december holiday" + "winter holiday" +
+    # "christmas holiday"; "day" → "valentine day" + "patrick day") is a
+    # real standalone category and is KEPT. (Operator: they want
+    # "holiday" recommended on its own even though "December Holiday"
+    # also surfaces.) Previously the union was superseded too, which
+    # hid these broad categories.
     SUPERSESSION_THRESHOLD = 0.8
     bigram_components = defaultdict(list)
     for c in candidates:
@@ -470,11 +476,12 @@ def suggest_clusters(
         c_ids = set(c["product_ids"])
         if not c_ids:
             continue
-        covered = set()
+        best_single = 0.0
         for big in bigram_components.get(c["stem"], []):
-            covered |= set(big["product_ids"])
-        overlap = len(c_ids & covered) / len(c_ids)
-        if overlap >= SUPERSESSION_THRESHOLD:
+            overlap = len(c_ids & set(big["product_ids"])) / len(c_ids)
+            if overlap > best_single:
+                best_single = overlap
+        if best_single >= SUPERSESSION_THRESHOLD:
             superseded.add(c["stem"])
             filtered_count += 1
 

make_post_sell/tests/test_models.py

@@ -5570,12 +5570,14 @@ class TestTagSuggestPureFunctions(unittest.TestCase):
             f"expected an apostrophe label in {labels}",
         )
 
-    def test_suggest_clusters_multi_bigram_supersedes_unigram(self):
-        """A unigram drops when the UNION of bigrams covering it
-        collectively exceeds the supersession threshold. Mirrors the
-        printableprompts gotcha: 'Day' shouldn't surface when
-        'Valentine's Day' + 'Patrick's Day' + 'Christmas' bigrams
-        already cover all its products."""
+    def test_suggest_clusters_keeps_umbrella_unigram_over_multi_bigrams(self):
+        """An umbrella unigram is KEPT when it's covered only by the
+        UNION of several DISTINCT bigrams (no single bigram dominates).
+        Operator: they want 'holiday' recommended on its own even
+        though 'December Holiday' etc. also surface. Here 'Day' spans
+        'Valentine Day' + 'Patrick Day' (each ~50%, neither ≥0.8), so
+        'Day' survives as its own broad category. (A single dominant
+        bigram still supersedes — see the test above.)"""
         from types import SimpleNamespace
         from ..lib.tag_suggest import suggest_clusters
         # 4 products: two carry "Valentine's Day", two carry
@@ -5600,7 +5602,36 @@ class TestTagSuggestPureFunctions(unittest.TestCase):
         # Bigrams survive
         self.assertTrue(any("alentin" in l.lower() for l in labels))
         self.assertTrue(any("atrick" in l.lower() for l in labels))
-        # "Day" unigram drops — collectively covered by both bigrams
-        self.assertNotIn("Day", labels)
+        # "Day" umbrella unigram SURVIVES — only the UNION of the two
+        # distinct bigrams covers it; no single bigram reaches the
+        # supersession threshold, so it stays a standalone category.
+        self.assertIn("Day", labels)
+
+    def test_suggest_clusters_surfaces_holiday_with_phrase_bigrams(self):
+        """Operator's exact case: 'December Holiday' / 'Winter Holiday' /
+        'Christmas Holiday' bigrams surface AND 'Holiday' is also
+        recommended on its own (umbrella unigram across distinct
+        bigrams — no single bigram dominates ≥0.8)."""
+        from types import SimpleNamespace
+        from ..lib.tag_suggest import suggest_clusters
+        products = [
+            SimpleNamespace(id="a", title="December Holiday Math", description=""),
+            SimpleNamespace(id="b", title="December Holiday Reading", description=""),
+            SimpleNamespace(id="c", title="Winter Holiday Crafts", description=""),
+            SimpleNamespace(id="d", title="Christmas Holiday Activities", description=""),
+        ]
+        clusters, _ = suggest_clusters(
+            products, stopwords=[], existing_tag_slugs=[],
+            min_products=2, max_share=1.0, min_title_share=0.0,
+            bigrams=True,
+        )
+        labels = [c["label"].lower() for c in clusters]
+        # 'holiday' on its own is recommended...
+        self.assertIn("holiday", labels)
+        # ...alongside at least one of the phrase bigrams.
+        self.assertTrue(
+            any("holiday" in l and l != "holiday" for l in labels),
+            f"expected a *_holiday bigram too, got {labels}",
+        )