feat: MPS-24 — auto-hydrate products into existing tags on create/edit
Operator: new products / edited descriptions should auto-file into the shop's existing categories without manual tagging. - lib/tag_suggest.py:auto_hydrate_tags(dbsession, product) — stem-match title+description against the shop's EXISTING tag names (reuses tokenize/simple_stem; every stem of the tag name must be in the product stem set, so unigram 'Holiday' and phrase 'First Grade' both work). ADDITIVE (never removes), IDEMPOTENT, never CREATES tags (inventing categories stays suggest-then-approve). - views/product.py:_auto_hydrate_and_flash wired into product_new (create), product_edit_description (markup desc editor), and product_edit (when title/desc changed, AFTER the explicit comma-tag sync so it's purely additive). - Tests: TestAutoHydrateTags (3, integration) + test_new_product_auto_hydrates_existing_tag / test_edit_description_auto_hydrates_existing_tag (functional). 1145 passed. - Docs: CLAUDE.md (auto-apply-existing vs never-auto-CREATE distinction), mps-24.md Phase 2.8n.
This commit is contained in:
parent
60a93de763
commit
2bc1e20958
6 changed files with 253 additions and 2 deletions
19
CLAUDE.md
19
CLAUDE.md
|
|
@ -611,8 +611,23 @@ URL knobs: `?max_share=0.3` (stricter shop-vocab cut), `?max_share=1`
|
|||
default 500, clamped ≤5000).
|
||||
CLI: `python -m make_post_sell.scripts.backfill_tags
|
||||
data/development.ini --shop=<id> [--max-share=0.4]
|
||||
[--min-title-share=0.3] [--no-bigrams] [--apply]`. **Never
|
||||
auto-commits** — operator approves every cluster.
|
||||
[--min-title-share=0.3] [--no-bigrams] [--apply]`. The suggest engine
|
||||
(creating NEW categories) **never auto-commits** — operator approves
|
||||
every cluster.
|
||||
|
||||
**Auto-hydrate (Phase 2.8n) — distinct from the suggest engine.**
|
||||
`lib/tag_suggest.py:auto_hydrate_tags(dbsession, product)` auto-files
|
||||
a product into the shop's **EXISTING** tags by stem-matching its
|
||||
title+description (same `tokenize`/`simple_stem` as the suggest
|
||||
engine; a tag matches when every stem of its name is in the product's
|
||||
stem set — unigram "Holiday" and phrase "First Grade" both work).
|
||||
Called automatically from `views/product.py` on **product create**
|
||||
(`product_new`) and on **title/description change** (`product_edit`,
|
||||
`product_edit_description`) via `_auto_hydrate_and_flash`. Contract:
|
||||
**additive** (never removes), **idempotent**, **never CREATES tags**
|
||||
(inventing categories stays suggest-then-approve). This is the one
|
||||
place existing-tag application is automatic — it files products into
|
||||
categories the operator already defined; it does not invent taxonomy.
|
||||
|
||||
Phase 3 (this ticket, gated): ML categorization via uncloseai endpoint
|
||||
behind `app.features.ml_categorization.enabled` kill switch (mirror MPS-22).
|
||||
|
|
|
|||
|
|
@ -363,6 +363,25 @@ Tests (`test_functional.py::TestProductTagsSpa`):
|
|||
Deferred (occasional click, not the hot path): AJAX-ifying the
|
||||
"Suggest categories" link — still a full navigation by design.
|
||||
|
||||
**Phase 2.8n — auto-hydrate products into existing tags** (shipped
|
||||
2026-05-17): operator wants new/edited products auto-filed into
|
||||
existing categories without manual tagging.
|
||||
`lib/tag_suggest.py:auto_hydrate_tags` stem-matches a product's
|
||||
title+description against the shop's EXISTING tag names (reuses
|
||||
`tokenize`/`simple_stem`; every stem of the tag name must appear in
|
||||
the product stem set — unigram + phrase tags both work). Wired via
|
||||
`views/product.py:_auto_hydrate_and_flash` into `product_new`
|
||||
(create), `product_edit_description` (markup desc editor) and
|
||||
`product_edit` (when title/desc changed, AFTER the explicit comma-tag
|
||||
sync so it's additive). Contract: additive (never removes),
|
||||
idempotent, never CREATES tags — inventing categories stays
|
||||
suggest-then-approve; this only files into operator-defined ones.
|
||||
Tests: `TestAutoHydrateTags` (3, test_integration) +
|
||||
`test_new_product_auto_hydrates_existing_tag` /
|
||||
`test_edit_description_auto_hydrates_existing_tag` (test_functional).
|
||||
Note: shares the suggest engine's stemmer, so its known traits apply
|
||||
(e.g. "studies"→"stud" ≠ "study"→"study"); consistent by design.
|
||||
|
||||
**Phase 2.8m — supersession keeps umbrella unigrams** (shipped
|
||||
2026-05-17): operator saw "December Holiday" but not "Holiday" on its
|
||||
own. Bigram supersession dropped a unigram when the *union* of all
|
||||
|
|
|
|||
|
|
@ -487,3 +487,59 @@ def suggest_clusters(
|
|||
|
||||
candidates = [c for c in candidates if c["stem"] not in superseded]
|
||||
return candidates[:top_n], filtered_count
|
||||
|
||||
|
||||
def auto_hydrate_tags(dbsession, product):
|
||||
"""Auto-file a product into the shop's EXISTING tags by its content.
|
||||
|
||||
Called when a product is created or its title/description changes.
|
||||
A tag matches when EVERY stem of its name appears in the product's
|
||||
title+description stem set, so unigram tags ("Holiday") and phrase
|
||||
tags ("First Grade", "Novel Studies") both work. Reuses the same
|
||||
tokenize/simple_stem as the suggest engine for consistency.
|
||||
|
||||
Contract (deliberately conservative):
|
||||
* ADDITIVE only — never removes a tag (operator/explicit tags and
|
||||
prior auto-tags are preserved).
|
||||
* IDEMPOTENT — skips tags already on the product.
|
||||
* Never CREATES tags — inventing categories stays
|
||||
suggest-then-approve. This only files products into categories
|
||||
the operator already defined.
|
||||
|
||||
Returns the list of Tag rows newly attached (for flash/logging).
|
||||
"""
|
||||
shop = product.shop
|
||||
if shop is None:
|
||||
return []
|
||||
from ..models.tag import tags_by_popularity
|
||||
|
||||
tags = tags_by_popularity(dbsession, shop)
|
||||
if not tags:
|
||||
return []
|
||||
|
||||
stopwords = list(shop.tag_stopwords)
|
||||
prod_tokens = tokenize(product.title or "", stopwords=stopwords)
|
||||
prod_tokens += tokenize(
|
||||
product.description or "",
|
||||
stopwords=stopwords,
|
||||
cap=DESCRIPTION_TOKEN_CAP,
|
||||
)
|
||||
prod_stems = {simple_stem(t) for t in prod_tokens}
|
||||
if not prod_stems:
|
||||
return []
|
||||
|
||||
existing_ids = {t.id for t in product.tags}
|
||||
attached = []
|
||||
for tag in tags:
|
||||
if tag.id in existing_ids:
|
||||
continue
|
||||
name_tokens = tokenize(tag.name or "", stopwords=stopwords)
|
||||
if not name_tokens:
|
||||
# Tag name is all stopwords/punctuation — nothing safe to
|
||||
# match on; skip rather than mis-file.
|
||||
continue
|
||||
name_stems = {simple_stem(t) for t in name_tokens}
|
||||
if name_stems <= prod_stems:
|
||||
product.tags.append(tag)
|
||||
attached.append(tag)
|
||||
return attached
|
||||
|
|
|
|||
|
|
@ -6892,6 +6892,51 @@ class TestProductTagsSpa(_AuthenticatedBase):
|
|||
self.assertEqual(res.json["status"], "ok")
|
||||
self.assertEqual(res.json["tag"]["slug"], "geometry")
|
||||
|
||||
def test_new_product_auto_hydrates_existing_tag(self):
|
||||
"""MPS-24: creating a product auto-files it into the shop's
|
||||
EXISTING tags by title/description content (no operator action,
|
||||
never creates tags)."""
|
||||
shop = self._create_shop_helper(user_creds=self.user1_creds)
|
||||
shop_id = shop.id
|
||||
# Operator-defined category.
|
||||
self.testapp.post(
|
||||
f"/s/{shop_id}/tags",
|
||||
{"action": "create", "name": "Holiday"},
|
||||
headers={"X-Requested-With": "XMLHttpRequest"}, status=200,
|
||||
)
|
||||
# New content product whose title carries the category word.
|
||||
params = dict(self.product1_params)
|
||||
params["title"] = "Winter Holiday Pack"
|
||||
params.pop("is_sellable", None) # content → is_ready
|
||||
self.testapp.post(f"/p/new?shop_id={shop_id}", params)
|
||||
from ..models.product import get_all_products
|
||||
self.dbsession.expire_all()
|
||||
product = get_all_products(self.dbsession).all()[-1]
|
||||
slugs = sorted(t.slug for t in product.tags)
|
||||
self.assertIn("holiday", slugs)
|
||||
|
||||
def test_edit_description_auto_hydrates_existing_tag(self):
|
||||
"""Editing a product's description auto-files it into matching
|
||||
EXISTING tags (the bulk-edit form path)."""
|
||||
shop_id, product_id = self._make_product()
|
||||
self.testapp.post(
|
||||
f"/s/{shop_id}/tags",
|
||||
{"action": "create", "name": "Robotics"},
|
||||
headers={"X-Requested-With": "XMLHttpRequest"}, status=200,
|
||||
)
|
||||
# Description now mentions the category word.
|
||||
self.testapp.post(
|
||||
f"/p/{product_id}/edit",
|
||||
{
|
||||
"title": "Algebra Workbook",
|
||||
"description": "A robotics enrichment unit.",
|
||||
"price": "50.00",
|
||||
"visibility": "1",
|
||||
"tags": "",
|
||||
},
|
||||
)
|
||||
self.assertIn("robotics", self._product_tag_slugs(product_id))
|
||||
|
||||
# ── Phase 2.8: AJAX tag-focus + drag set_order + page weight ──────
|
||||
|
||||
def test_ajax_focus_returns_product_list_json(self):
|
||||
|
|
|
|||
|
|
@ -5644,3 +5644,87 @@ class TestProductTagAddRemoveIntegration(DatabaseIntegrationTests):
|
|||
# Nothing removed — Seasonal still applied.
|
||||
self.assertEqual([t.slug for t in product.tags], ["seasonal"])
|
||||
transaction.commit()
|
||||
|
||||
|
||||
class TestAutoHydrateTags(DatabaseIntegrationTests):
|
||||
"""MPS-24: auto_hydrate_tags files a product into the shop's
|
||||
EXISTING tags by title/description content. Additive, idempotent,
|
||||
never creates tags."""
|
||||
|
||||
def _shop(self):
|
||||
get_or_create_user_by_email(self.dbsession, "hydrate@example.com")
|
||||
shop = Shop(
|
||||
name="Hydrate Shop",
|
||||
phone_number="555-555-5555",
|
||||
billing_address="123 Test St",
|
||||
description="A test shop",
|
||||
)
|
||||
shop.domain_name = "hydrate.test"
|
||||
self.dbsession.add(shop)
|
||||
self.dbsession.flush()
|
||||
return shop
|
||||
|
||||
def test_attaches_existing_tag_by_title_and_description(self):
|
||||
from ..models.tag import get_or_create_tag
|
||||
from ..lib.tag_suggest import auto_hydrate_tags
|
||||
shop = self._shop()
|
||||
get_or_create_tag(self.dbsession, shop, "Holiday") # unigram, title
|
||||
get_or_create_tag(self.dbsession, shop, "First Grade") # phrase, desc
|
||||
get_or_create_tag(self.dbsession, shop, "Robotics") # unrelated
|
||||
p = Product(title="Winter Holiday Pack",
|
||||
description="Great first grade practice activities.")
|
||||
p.shop = shop
|
||||
self.dbsession.add(p)
|
||||
self.dbsession.flush()
|
||||
|
||||
attached = auto_hydrate_tags(self.dbsession, p)
|
||||
slugs = sorted(t.slug for t in p.tags)
|
||||
# "holiday" (title unigram) + "first grade" (desc phrase: both
|
||||
# stems present). Same stemmer as the suggest engine.
|
||||
self.assertIn("holiday", slugs)
|
||||
self.assertIn("first-grade", slugs)
|
||||
# Unrelated tag NOT attached.
|
||||
self.assertNotIn("robotics", slugs)
|
||||
self.assertEqual(
|
||||
sorted(t.slug for t in attached),
|
||||
["first-grade", "holiday"],
|
||||
)
|
||||
transaction.commit()
|
||||
|
||||
def test_idempotent_and_additive_never_creates(self):
|
||||
from ..models.tag import get_or_create_tag, tags_by_popularity
|
||||
from ..lib.tag_suggest import auto_hydrate_tags
|
||||
shop = self._shop()
|
||||
get_or_create_tag(self.dbsession, shop, "Math")
|
||||
manual = get_or_create_tag(self.dbsession, shop, "Manual Pick")
|
||||
p = Product(title="Math Worksheet", description="x")
|
||||
p.shop = shop
|
||||
p.tags.append(manual) # operator-applied, must be preserved
|
||||
self.dbsession.add(p)
|
||||
self.dbsession.flush()
|
||||
tag_count_before = len(tags_by_popularity(self.dbsession, shop))
|
||||
|
||||
first = auto_hydrate_tags(self.dbsession, p)
|
||||
self.assertEqual([t.slug for t in first], ["math"])
|
||||
# Second pass attaches nothing new (idempotent).
|
||||
second = auto_hydrate_tags(self.dbsession, p)
|
||||
self.assertEqual(second, [])
|
||||
# Manual tag preserved (additive, never removes).
|
||||
slugs = sorted(t.slug for t in p.tags)
|
||||
self.assertIn("manual-pick", slugs)
|
||||
self.assertIn("math", slugs)
|
||||
# No NEW tags were created by hydration.
|
||||
self.assertEqual(
|
||||
len(tags_by_popularity(self.dbsession, shop)), tag_count_before
|
||||
)
|
||||
transaction.commit()
|
||||
|
||||
def test_no_tags_no_match_is_safe(self):
|
||||
from ..lib.tag_suggest import auto_hydrate_tags
|
||||
shop = self._shop()
|
||||
p = Product(title="Untagged Thing", description="nothing matches")
|
||||
p.shop = shop
|
||||
self.dbsession.add(p)
|
||||
self.dbsession.flush()
|
||||
self.assertEqual(auto_hydrate_tags(self.dbsession, p), [])
|
||||
transaction.commit()
|
||||
|
|
|
|||
|
|
@ -25,6 +25,25 @@ def checkbox_to_bool(checkbox):
|
|||
return checkbox == "on"
|
||||
|
||||
|
||||
def _auto_hydrate_and_flash(request, product):
|
||||
"""Auto-file a product into the shop's EXISTING tags by its title +
|
||||
description content, and flash what was applied. Additive + never
|
||||
creates tags — see lib/tag_suggest.auto_hydrate_tags. Called on
|
||||
product create and on title/description change."""
|
||||
from ..lib.tag_suggest import auto_hydrate_tags
|
||||
|
||||
hydrated = auto_hydrate_tags(request.dbsession, product)
|
||||
if hydrated:
|
||||
names = ", ".join(t.name for t in hydrated)
|
||||
request.session.flash((
|
||||
"Auto-filed into "
|
||||
f"{len(hydrated)} existing "
|
||||
f"categor{'y' if len(hydrated) == 1 else 'ies'}: {names}.",
|
||||
"success",
|
||||
))
|
||||
return hydrated
|
||||
|
||||
|
||||
@view_config(route_name="product", renderer="product.j2")
|
||||
@view_config(route_name="product_slug", renderer="product.j2")
|
||||
def product(request):
|
||||
|
|
@ -197,6 +216,7 @@ def product_new(request):
|
|||
|
||||
request.dbsession.add(product)
|
||||
request.dbsession.flush()
|
||||
_auto_hydrate_and_flash(request, product)
|
||||
if request.shop.watch_mode_enabled:
|
||||
reforge_discovery_ring_async(
|
||||
request.shop.id, request.registry["dbsession_factory"]
|
||||
|
|
@ -236,6 +256,7 @@ def product_edit_description(request):
|
|||
request.session.flash(msg)
|
||||
request.dbsession.add(product)
|
||||
request.dbsession.flush()
|
||||
_auto_hydrate_and_flash(request, product)
|
||||
if product.shop.watch_mode_enabled:
|
||||
reforge_discovery_ring_async(
|
||||
product.shop.id, request.registry["dbsession_factory"]
|
||||
|
|
@ -283,13 +304,17 @@ def product_edit(request):
|
|||
s3_webhook_bucket = request.params.get("bucket")
|
||||
s3_webhook_etag = request.params.get("etag")
|
||||
|
||||
title_or_desc_changed = False
|
||||
|
||||
if title != product.title:
|
||||
product_modified = True
|
||||
title_or_desc_changed = True
|
||||
product.title = title
|
||||
request.session.flash(("You updated the product's title.", "success"))
|
||||
|
||||
if description != product.description:
|
||||
product_modified = True
|
||||
title_or_desc_changed = True
|
||||
product.set_description(description)
|
||||
request.session.flash(("You updated the product's description.", "success"))
|
||||
|
||||
|
|
@ -345,6 +370,13 @@ def product_edit(request):
|
|||
("You updated the product's tags.", "success")
|
||||
)
|
||||
|
||||
# MPS-24: when the title/description changed, auto-file the product
|
||||
# into the shop's EXISTING categories by content. Runs AFTER the
|
||||
# explicit comma-tags sync above so it's purely additive (it never
|
||||
# strips operator tags and never creates categories).
|
||||
if title_or_desc_changed:
|
||||
_auto_hydrate_and_flash(request, product)
|
||||
|
||||
# MPS-20 + MPS-21: pricing_mode + allow_offers (per-product overrides).
|
||||
# Gated on the pricing_mode field being present — it only appears on
|
||||
# the product-title-and-description edit form, so other forms on the
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue