Fix SQLAlchemy deprecation in async reforge, add playback progress restore, add signal gathering tickets

This commit is contained in:
russell@unturf.com 2026-02-09 13:52:34 -05:00
parent 7b7a2d1edb
commit 3ba8802d9a
13 changed files with 496 additions and 2 deletions

View file

@ -0,0 +1,17 @@
# Build engagement signal weighting system
Implement weighted engagement scoring where different signals carry different algorithmic weight:
| Signal | Weight |
|--------|--------|
| Purchase | 42x |
| Share to friend | 69x |
| Replay/re-listen | 21x |
| Save/bookmark | 14x |
| Positive comment | 7x |
| Neutral comment | 1x |
| Quick bounce (<7s) | 0x (null) |
| Negative comment | 0x (null — algo learns nothing from hate) |
| Report | -7x (only negative signal) |
Critical: negative engagement is a NULL signal, not positive. The algorithm literally cannot optimize for outrage. Create an EngagementScore model/table to track and aggregate these weighted signals per product.

View file

@ -0,0 +1,13 @@
# Implement Truth Fidelity Score (7-point)
Create a 7-point Truth Fidelity Score for products, computed from:
1. Semantic similarity between description text and actual content
2. Thumbnail-to-content visual coherence
3. Title keyword overlap with content tags
4. Buyer satisfaction rate post-purchase
5. Refund/dispute ratio (inverted)
6. Repeat-buyer rate from same creator
7. Organic share rate
Each scored 0-1, summed to 7 max. Creators above 4.2 get "Truth Verified" status. Store as a model field, recalculate periodically.

View file

@ -0,0 +1,12 @@
# Add sentiment analysis to comments
Integrate sentiment analysis for comments to support the Love pillar:
- Classify comments as positive/neutral/negative
- Products with >7:1 positive:negative ratio get "Love Verified" badge
- Weight thoughtful 42+ word comments 14x more than emoji-only or single-word reactions
- Feed sentiment scores into the engagement weighting system (positive comments = 7x, neutral = 1x, negative = 0x)
- Use a lightweight local model or rule-based approach to avoid external API dependency
- Store sentiment score per comment in DB
Blocked by: engagement signal weighting system.

View file

@ -0,0 +1,11 @@
# Implement Love Gifts (voluntary tipping system)
Add voluntary tip/gift system beyond purchases:
- Love Gifts: Tips from 1-420 units of currency
- No platform cut on the first $0.69 (pure creator love)
- UI: Gift button on product pages and in watch mode
- Integrate with existing payment infrastructure (Stripe, PayPal, crypto)
- Track gifts in DB for engagement scoring (counts as strong positive signal)
- Update watch.js SPA to include gift button in updatePageContent()
- Server-rendered gift button for non-JS users

View file

@ -0,0 +1,11 @@
# Implement Love Chains (single-level referral credits)
When a buyer shares a product and someone else buys through that share:
- Original buyer gets 7% "love commission" as store credit
- Single-level only (NOT MLM) — no chains of chains
- Capped at 42 referrals per product per user
- Generate unique share links per buyer per product
- Track referral source on purchase
- Credit applied as store credit for future purchases
- Dashboard for buyers to see their love chain impact

View file

@ -0,0 +1,11 @@
# Implement Love Letters (private buyer-to-creator messages)
Add private appreciation messages from buyer to creator, unlocked after purchase:
- Only available to verified buyers of a product
- Private 1-to-1, not public comments
- Creator can optionally publish as testimonials (with buyer permission)
- Simple message form on post-purchase page and product page (for past buyers)
- Store in DB with buyer_id, creator_id, product_id, message, published flag
- Creator dashboard to view and manage love letters
- Permission toggle for buyers to allow/deny publication

View file

@ -0,0 +1,9 @@
# Build transparent pricing history display
Show price history for products to enforce Truth:
- Display 14-day, 21-day, and 42-day price history on product pages
- Track all price changes in a price_history table (product_id, price, timestamp)
- No dark patterns — "was $420, now $7!" must be verifiable from history
- Simple visual: small text or expandable section showing price timeline
- If product has digital scarcity limits, enforce and display actual remaining count

View file

@ -0,0 +1,12 @@
# Create creator Pillars Score analytics dashboard
Real-time analytics dashboard for creators showing their four-pillar scores:
- Truth Score: content-signal alignment, refund rate, repeat-buyer rate
- Harmony Score: collaboration rate, cross-pollination, ecosystem contribution
- Freedom Score: format diversity, audience diversity
- Love Score: sentiment ratio, gift rate, love letter count, share rate
Each pillar scored and displayed visually. Show trends over 7/14/21/42 day windows. Include "Wand Mode" preview: suggestions like "Add a 21s hook to boost discovery?" based on current metrics. Actionable insights, not vanity metrics.
Blocked by: Truth Fidelity Score, engagement signal weighting, comment sentiment analysis.

290
docs/tickets/mps-2.md Normal file
View file

@ -0,0 +1,290 @@
# MPS-2: Anonymous Signal Gathering & View Count
## Philosophy
Collect every honest signal we can about how people experience content —
without recording who they are. No cookies, no IPs, no fingerprints, no user
agent strings. A viewer's identity is never captured. Only the aggregate truth
of what happened on the page.
The raw signals feed four derived scores: **Engagement**, **Attention**,
**Learning**, and **Passive Consumption**. These scores are computed from simple
math now and can be refined with machine learning later — the raw data is the
same either way.
A **view** counts after 7 seconds of wall-clock time on the page with the tab
visible. This respects all four pillars: truthful (they were really here), free
(no judgment on how they consume), harmonious (same bar for a 30-second clip
and a 2-hour film), and loving (assumes good intent from the visitor).
## What We Collect
All signals are gathered in JavaScript memory during a page session and sent as
a single JSON summary via `navigator.sendBeacon()` on page unload. One beacon
per page visit. Nothing streams, nothing phones home mid-session.
### Presence Signals
| Signal | How | Why |
|--------|-----|-----|
| `wall_clock_ms` | `Date.now()` delta from page load to unload | Total time on page |
| `visible_ms` | Page Visibility API (`visibilitychange`) — accumulate time while `visible` | Tab in foreground vs backgrounded |
| `active_ms` | Accumulate time while mouse/touch/keyboard events fire (debounce 2s idle gap) | Human is present and interacting |
| `idle_ms` | `wall_clock_ms - active_ms` | Derived, not tracked separately |
**Mouse/touch activity**: Listen for `mousemove`, `touchstart`, `keydown`
debounced to a boolean "active" flag that goes idle after 2 seconds of silence.
We track **whether** the user is active, never **where** on the page. No
coordinates, no element targets.
### Scroll Signals
| Signal | How | Why |
|--------|-----|-----|
| `scroll_depth_max` | `window.scrollY / document.body.scrollHeight` — track max (0.0-1.0) | How far down the page they went |
| `scroll_direction_changes` | Count sign changes in scroll delta | Reading (few reversals) vs scanning (many) |
### Media Playback Signals
Piggyback on existing `timeupdate`, `play`, `pause`, `ended`, `seeked`,
`ratechange` events already wired in watch.js.
| Signal | How | Why |
|--------|-----|-----|
| `media_duration_ms` | `media.duration * 1000` on `loadedmetadata` | Length of the content |
| `media_play_ms` | Accumulate time between `play` and `pause`/`ended` events | Actual playback time |
| `media_play_count` | Count `play` events | Replays = love |
| `media_pause_count` | Count `pause` events (user-initiated, not ended) | Interaction with content |
| `media_seek_count` | Count `seeked` events | Skipping around (study or skip?) |
| `media_seek_back_count` | Count seeks where `currentTime` decreased | Rewinding = learning |
| `media_speed` | Last value from `ratechange` event, default 1.0 | 2x = efficient, 0.5x = studying |
| `media_completed` | Boolean: did `ended` event fire at least once | Watched/listened to the end |
| `media_percent_played` | `media_play_ms / media_duration_ms` (can exceed 1.0 on replay) | Completion without judging speed |
### Context Signals (server-side, from the HTTP request)
| Signal | How | Why |
|--------|-----|-----|
| `is_ring_entry` | Boolean: full page load vs SPA transition | Front door vs ring navigation |
| `ring_position` | Integer: how many SPA transitions from entry | Depth in the ring |
| `referrer_class` | Enum: `direct`, `search`, `social`, `internal`, `unknown` | Where traffic comes from — derived from Referer header domain, NOT the full URL |
| `device_class` | Enum: `mobile`, `tablet`, `desktop` | Screen size bucket only, not user agent — derived from viewport width sent in beacon |
`referrer_class` is computed server-side by matching the Referer header domain
against known patterns (google/bing/ddg → search, twitter/facebook/reddit →
social, same domain → internal, everything else → unknown). The raw Referer URL
is never stored.
`device_class` is derived from viewport width sent in the beacon payload:
≤768 → mobile, ≤1024 → tablet, else desktop. No user agent parsing.
## Storage
### Per-session table: `mps_page_session`
One row per page visit. This is the raw truth.
```
id UUID PK
shop_id FK → mps_shop
product_id FK → mps_product
created_timestamp BigInteger (ms) — when beacon was received
-- presence
wall_clock_ms Integer
visible_ms Integer
active_ms Integer
-- scroll
scroll_depth_max Float (0.0-1.0)
scroll_direction_changes Integer
-- media
media_duration_ms Integer (NULL if no media)
media_play_ms Integer (NULL if no media)
media_play_count SmallInteger
media_pause_count SmallInteger
media_seek_count SmallInteger
media_seek_back_count SmallInteger
media_speed Float
media_completed Boolean
media_percent_played Float
-- context
is_ring_entry Boolean
ring_position SmallInteger
referrer_class SmallInteger (enum: 0=direct 1=search 2=social 3=internal 4=unknown)
device_class SmallInteger (enum: 0=mobile 1=tablet 2=desktop)
```
No foreign key to any user/account table. No IP column. No session cookie
column. The row is an anonymous fact about what happened, nothing more.
### Aggregate columns on `mps_product`
Updated asynchronously (on beacon receipt or periodic batch):
- `view_count` (BigInteger, server_default="0") — incremented when a session
hits the 7-second visible threshold
### Retention
Keep individual `mps_page_session` rows for 90 days. After that, aggregate
into `mps_product_daily_stats` (one row per product per day with averages and
sums) and delete the raw sessions. The daily rollup lives forever.
## Derived Scores
Computed from raw signals, either on-read (in Python properties) or in the
daily rollup. These are the creator-facing metrics.
### Engagement (0.0 1.0)
```
engagement = active_ms / wall_clock_ms
```
High = clicking, scrolling, pausing, seeking — actively interacting with the
content. Low = tab open but idle. This is the "lean-in" metric.
### Attention (0.0 1.0)
```
attention = visible_ms / wall_clock_ms
```
High = tab in foreground the entire time. Low = they switched tabs. Combined
with engagement: high attention + low engagement = focused passive watching.
High attention + high engagement = deeply studying.
### Learning Signal (heuristic score)
Indicators of active study, not passive consumption:
- `media_seek_back_count > 0` — they rewound to re-hear something
- `media_speed < 1.0` — slowed down to absorb
- `scroll_direction_changes > 3` — re-reading sections
- `media_pause_count > 2` — pausing to think or take notes
- `engagement > 0.7` — high interaction throughout
Score: count of true indicators (0-5). A session with 3+ is "learning mode."
### Passive Consumption (heuristic score)
Indicators of lean-back enjoyment:
- `attention > 0.7` — tab is in foreground
- `engagement < 0.3` — but not interacting much
- `media_percent_played > 0.69` — played most/all of the content
- `media_play_count == 1` — straight through, no replays
- `media_pause_count == 0` — never paused
Score: count of true indicators (0-5). A session with 3+ is "passive
consumption" — and this is a GOOD signal. It means the content is worth
playing through without interruption.
### Machine Learning Hook
The raw `mps_page_session` rows are feature vectors. Every field is a number.
When ML is added later, it can:
- Cluster sessions into behavioral profiles beyond the 4 heuristic scores
- Detect anomalies (bot patterns: zero scroll + zero active + high play count)
- Predict which products will retain based on early session patterns
- Surface "hidden gems" — products with high learning/attention scores but
low view counts
The collection schema doesn't change. ML consumes the same rows and produces
richer derived scores that feed into dashboards. The raw truth stays raw.
## View Count Display
Show the human-readable view count on product and content pages, next to the
upload date under the title h1:
```
Track Title
uploaded to ShopName
Created 3 days ago · 420 views
```
### Formatting
- 0: don't show view count at all (no "0 views")
- 1: "1 view"
- 2-999: exact number ("420 views")
- 1,000-999,999: "1.2K views"
- 1,000,000+: "1.8M views"
### Three-layer update (SPA compatibility)
1. **Templates** (`content.j2`, `product.j2`): render in `.product-meta-dates`
2. **watch.py** (`watch_json`): include `view_count` and `human_view_count`
3. **watch.js** (`updatePageContent`): update the view count span during SPA nav
## JavaScript Collector (`signals.js`)
New file. Lightweight, no dependencies. Loaded on all product/content pages.
```
Page load:
→ Record start time
→ Attach debounced mousemove/touchstart/keydown → active flag
→ Attach visibilitychange → accumulate visible_ms
→ Attach scroll → track depth + direction changes
→ Attach media events (play/pause/ended/seeked/ratechange)
→ Generate random session token (in-memory only, never persisted)
Page unload (beforeunload / visibilitychange → hidden):
→ Bundle all signals into JSON
→ POST via navigator.sendBeacon('/signals/beacon')
→ Fallback: fetch() with keepalive: true
```
The beacon payload is ~300 bytes of JSON. One request per page visit. No
polling, no websockets, no streaming.
## Server Endpoint
`POST /signals/beacon` — accepts JSON, validates fields, computes
`referrer_class` from the Referer header, inserts one `mps_page_session` row.
If `visible_ms >= 7000`, also increments `product.view_count`. Returns 204.
Rate limiting: max 1 beacon per product per client per 60 seconds, enforced
by a short-lived server-side cache keyed on `(session_token, product_id)`.
The session token is random per page load — it prevents double-counting
from accidental double-fires, not tracking users across pages.
## Files Changed
| File | Change |
|------|--------|
| `models/page_session.py` | New model: `MpsPageSession` |
| `models/product.py` | Add `view_count` column, `human_view_count` property |
| `models/__init__.py` | Import new model |
| `views/signals.py` | New: beacon endpoint |
| `views/watch.py` | Include `view_count` / `human_view_count` in JSON |
| `views/product.py` | Pass view count to template context |
| `views/content.py` | Pass view count to template context |
| `static/js/signals.js` | New: client-side signal collector |
| `templates/content.j2` | Show view count in `.product-meta-dates` |
| `templates/product.j2` | Show view count in `.product-meta-dates` |
| `templates/snippets/base_styles.j2` | Load signals.js on product pages |
| `static/js/watch.js` | Update view count in `updatePageContent()` |
| `scripts/alembic/versions/` | Migration: `mps_page_session` table + `view_count` column |
| `tests/test_functional.py` | Beacon endpoint tests, view count display tests |
| `tests/test_models.py` | `human_view_count` formatting tests |
## What This Does NOT Collect
- IP addresses
- User agent strings (device class is viewport width only)
- Cookies or persistent identifiers
- Mouse/touch coordinates
- Clicked element identifiers
- Any form input content
- Referrer URLs (only domain classification)
- Cross-page session linking (each page visit is an island)
A viewer could visit every product in a shop and there would be no way to
connect those visits to the same person. That is by design.

51
docs/tickets/mps-3.md Normal file
View file

@ -0,0 +1,51 @@
# MPS-3: Creator Analytics Dashboard
## Problem
MPS-2 collects anonymous signals and derives Engagement, Attention, Learning,
and Passive Consumption scores. Creators need somewhere to see this data and
act on it.
## Solution
Add a simple analytics page accessible from the shop dashboard. No graphs
library — clean server-rendered HTML with CSS grid tables. Machine learning
refinements to the derived scores can enhance this page later without changing
its structure.
### Content
- **Top products by views** — ranked list, last 7 / 14 / 21 days
- **Ring entry points** — which products are "front doors" (top 7 by `is_ring_entry` count)
- **Average ring depth** — how far viewers ride the ring (mean `ring_position`)
- **Daily view totals** — last 21 days, per product and shop-wide
- **Engagement leaders** — products with highest average engagement score
- **Attention holders** — products with highest average attention score
- **Study material** — products with highest learning signal (people rewind, slow down, re-read)
- **Background favorites** — products with highest passive consumption score (lean-back plays)
- **Traffic sources** — breakdown by `referrer_class` (direct / search / social / internal)
- **Device split** — mobile vs tablet vs desktop percentages
### Access
New route `/shop/analytics` — only visible to shop owner/mods. Link from shop
settings or dashboard nav.
### Privacy
All data is anonymous aggregates computed from `mps_page_session` rows. No
individual viewer data exists to display even if someone wanted to.
## Files Changed
| File | Change |
|------|--------|
| `views/shop.py` | New `analytics` view with aggregate queries |
| `templates/analytics.j2` | New template |
| `static/css/common.css` | Analytics table styles |
| `templates/snippets/shop_nav.j2` | Link to analytics (if exists) |
| `tests/test_functional.py` | Analytics page access tests |
## Depends On
MPS-2 (signal gathering + `mps_page_session` table)

View file

@ -1 +1 @@
6eb4b3c
7b7a2d1

View file

@ -716,7 +716,7 @@ def reforge_discovery_ring_async(shop_id, session_factory):
from sqlalchemy.orm import Session
session = Session(bind=session_factory().get_bind())
try:
shop = session.query(Shop).get(shop_id)
shop = session.get(Shop, shop_id)
if shop is None:
return
ring = compute_discovery_ring(shop)

View file

@ -21,6 +21,9 @@
var currentProductId = null;
var urlRefreshTimer = null;
var URL_REFRESH_MS = 7 * 60 * 1000; // 7 minutes
var PROGRESS_KEY = 'mps_watch_progress';
var PROGRESS_SAVE_INTERVAL = 7000;
var lastProgressSave = 0;
// --- Ring state (persisted in localStorage) ---
var ringProductIds = [];
@ -174,6 +177,12 @@
// --- Autoplay with sound (for video/audio) ---
if (activeMedia) {
// Restore saved progress on reload (seek + volume fade-in)
activeMedia.addEventListener('loadeddata', function onInitLoad() {
activeMedia.removeEventListener('loadeddata', onInitLoad);
restoreProgress();
});
var playPromise = activeMedia.play();
if (playPromise !== undefined) {
playPromise.catch(function() {
@ -1104,8 +1113,50 @@
});
}
// --- Playback progress: save to localStorage, restore on reload ---
function saveProgress() {
if (!activeMedia || !currentProductId) return;
if (!activeMedia.duration || !isFinite(activeMedia.duration)) return;
try {
localStorage.setItem(PROGRESS_KEY, JSON.stringify({
id: currentProductId,
time: activeMedia.currentTime,
ts: Date.now()
}));
} catch(e) {}
}
function clearProgress() {
try { localStorage.removeItem(PROGRESS_KEY); } catch(e) {}
}
function restoreProgress() {
if (!activeMedia || !currentProductId) return;
try {
var raw = localStorage.getItem(PROGRESS_KEY);
if (!raw) return;
var saved = JSON.parse(raw);
// Only restore if same product and saved within last 24 hours
if (saved.id !== currentProductId) return;
if (Date.now() - saved.ts > 24 * 60 * 60 * 1000) { clearProgress(); return; }
if (!saved.time || saved.time < 3) return;
activeMedia.currentTime = saved.time;
// Fade volume up from 0 over 2 seconds
activeMedia.volume = 0;
var fadeSteps = 40;
var fadeStep = 0;
var fadeInterval = setInterval(function() {
fadeStep++;
activeMedia.volume = Math.min(1, fadeStep / fadeSteps);
if (fadeStep >= fadeSteps) clearInterval(fadeInterval);
}, 50);
} catch(e) {}
}
// --- Named event handlers (must be named so removeEventListener works) ---
function onMediaEnded() {
clearProgress();
// Fallback: only start countdown if DJ crossfade didn't handle it
if (!djCrossfadeActive) {
// Short clips: instant hard-cut to next if preloaded, no countdown
@ -1136,6 +1187,12 @@
&& activeMedia.duration > COUNTDOWN_SECONDS) {
startDjCrossfade();
}
// Save progress every 7 seconds
var now = Date.now();
if (now - lastProgressSave >= PROGRESS_SAVE_INTERVAL) {
lastProgressSave = now;
saveProgress();
}
}
// --- Media event setup ---