make_post_sell/docs/tickets/mps-2.md

MPS-2: Anonymous Signal Gathering & View Count

Philosophy

Collect every honest signal we can about how people experience content — without recording who they are. No cookies, no IPs, no fingerprints, no user agent strings. A viewer's identity is never captured. Only the aggregate truth of what happened on the page.

The raw signals feed four derived scores: Engagement, Attention, Learning, and Passive Consumption. These scores are computed from simple math now and can be refined with machine learning later — the raw data is the same either way.

A view counts after 7 seconds of wall-clock time on the page with the tab visible. This respects all four pillars: truthful (they were really here), free (no judgment on how they consume), harmonious (same bar for a 30-second clip and a 2-hour film), and loving (assumes good intent from the visitor).

What We Collect

All signals are gathered in JavaScript memory during a page session and sent as a single JSON summary via navigator.sendBeacon() on page unload. One beacon per page visit. Nothing streams, nothing phones home mid-session.

Presence Signals

Signal	How	Why
wall_clock_ms	Date.now() delta from page load to unload	Total time on page
visible_ms	Page Visibility API (visibilitychange) — accumulate time while visible	Tab in foreground vs backgrounded
active_ms	Accumulate time while mouse/touch/keyboard events fire (debounce 2s idle gap)	Human is present and interacting
idle_ms	wall_clock_ms - active_ms	Derived, not tracked separately

Mouse/touch activity: Listen for mousemove, touchstart, keydown — debounced to a boolean "active" flag that goes idle after 2 seconds of silence. We track whether the user is active, never where on the page. No coordinates, no element targets.

Scroll Signals

Signal	How	Why
scroll_depth_max	window.scrollY / document.body.scrollHeight — track max (0.0-1.0)	How far down the page they went
scroll_direction_changes	Count sign changes in scroll delta	Reading (few reversals) vs scanning (many)

Media Playback Signals

Piggyback on existing timeupdate, play, pause, ended, seeked, ratechange events already wired in watch.js.

Signal	How	Why
media_duration_ms	media.duration * 1000 on loadedmetadata	Length of the content
media_play_ms	Accumulate time between play and pause/ended events	Actual playback time
media_play_count	Count play events	Replays = love
media_pause_count	Count pause events (user-initiated, not ended)	Interaction with content
media_seek_count	Count seeked events	Skipping around (study or skip?)
media_seek_back_count	Count seeks where currentTime decreased	Rewinding = learning
media_speed	Last value from ratechange event, default 1.0	2x = efficient, 0.5x = studying
media_completed	Boolean: did ended event fire at least once	Watched/listened to the end
media_percent_played	media_play_ms / media_duration_ms (can exceed 1.0 on replay)	Completion without judging speed

Context Signals (server-side, from the HTTP request)

Signal	How	Why
is_ring_entry	Boolean: full page load vs SPA transition	Front door vs ring navigation
ring_position	Integer: how many SPA transitions from entry	Depth in the ring
referrer_class	Enum: direct, search, social, internal, unknown	Where traffic comes from — derived from Referer header domain, NOT the full URL
referrer_domain	String (128 chars)	Referer hostname (e.g. "www.google.com") — added in MPS-6
referrer_query	String (256 chars)	Search engine query parameter — added in MPS-6
device_class	Enum: mobile, tablet, desktop	Screen size bucket only, not user agent — derived from viewport width sent in beacon

referrer_class is computed server-side by matching the Referer header domain against known patterns (google/bing/ddg → search, twitter/facebook/reddit → social, same domain → internal, everything else → unknown). The raw Referer URL is never stored.

device_class is derived from viewport width sent in the beacon payload: ≤768 → mobile, ≤1024 → tablet, else desktop. No user agent parsing.

Storage

Per-session table: mps_page_session

One row per page visit. This is the raw truth.

id                      UUID PK
shop_id                 FK → mps_shop
product_id              FK → mps_product
created_timestamp       BigInteger (ms) — when beacon was received

-- presence
wall_clock_ms           Integer
visible_ms              Integer
active_ms               Integer

-- scroll
scroll_depth_max        Float (0.0-1.0)
scroll_direction_changes Integer

-- media
media_duration_ms       Integer (NULL if no media)
media_play_ms           Integer (NULL if no media)
media_play_count        SmallInteger
media_pause_count       SmallInteger
media_seek_count        SmallInteger
media_seek_back_count   SmallInteger
media_speed             Float
media_completed         Boolean
media_percent_played    Float

-- context
is_ring_entry           Boolean
ring_position           SmallInteger
referrer_class          SmallInteger (enum: 0=direct 1=search 2=social 3=internal 4=unknown)
referrer_domain         Unicode(128) — extracted domain (e.g. "www.google.com")
referrer_query          Unicode(256) — search engine query (e.g. "lo-fi beats")
device_class            SmallInteger (enum: 0=mobile 1=tablet 2=desktop)

No foreign key to any user/account table. No IP column. No session cookie column. The row is an anonymous fact about what happened, nothing more.

Aggregate columns on mps_product

Updated asynchronously (on beacon receipt or periodic batch):

• view_count (BigInteger, server_default="0") — incremented when a session hits the 7-second visible threshold

Retention

Keep individual mps_page_session rows for 90 days. After that, aggregate into mps_product_daily_stats (one row per product per day with averages and sums) and delete the raw sessions. The daily rollup lives forever.

Derived Scores

Computed from raw signals, either on-read (in Python properties) or in the daily rollup. These are the creator-facing metrics.

Engagement (0.0 – 1.0)

engagement = active_ms / wall_clock_ms

High = clicking, scrolling, pausing, seeking — actively interacting with the content. Low = tab open but idle. This is the "lean-in" metric.

Attention (0.0 – 1.0)

attention = visible_ms / wall_clock_ms

High = tab in foreground the entire time. Low = they switched tabs. Combined with engagement: high attention + low engagement = focused passive watching. High attention + high engagement = deeply studying.

Learning Signal (heuristic score)

Indicators of active study, not passive consumption:

• media_seek_back_count > 0 — they rewound to re-hear something
• media_speed < 1.0 — slowed down to absorb
• scroll_direction_changes > 3 — re-reading sections
• media_pause_count > 2 — pausing to think or take notes
• engagement > 0.7 — high interaction throughout

Score: count of true indicators (0-5). A session with 3+ is "learning mode."

Passive Consumption (heuristic score)

Indicators of lean-back enjoyment:

• attention > 0.7 — tab is in foreground
• engagement < 0.3 — but not interacting much
• media_percent_played > 0.69 — played most/all of the content
• media_play_count == 1 — straight through, no replays
• media_pause_count == 0 — never paused

Score: count of true indicators (0-5). A session with 3+ is "passive consumption" — and this is a GOOD signal. It means the content is worth playing through without interruption.

Machine Learning Hook

The raw mps_page_session rows are feature vectors. Every field is a number. When ML is added later, it can:

• Cluster sessions into behavioral profiles beyond the 4 heuristic scores
• Detect anomalies (bot patterns: zero scroll + zero active + high play count)
• Predict which products will retain based on early session patterns
• Surface "hidden gems" — products with high learning/attention scores but low view counts

The collection schema doesn't change. ML consumes the same rows and produces richer derived scores that feed into dashboards. The raw truth stays raw.

View Count Display

Show the human-readable view count on product and content pages, next to the upload date under the title h1:

Track Title
uploaded to ShopName
Created 3 days ago · 420 views

Formatting

• 0: don't show view count at all (no "0 views")
• 1: "1 view"
• 2-999: exact number ("420 views")
• 1,000-999,999: "1.2K views"
• 1,000,000+: "1.8M views"

Three-layer update (SPA compatibility)

1. Templates (content.j2, product.j2): render in .product-meta-dates
2. watch.py (watch_json): include view_count and human_view_count
3. watch.js (updatePageContent): update the view count span during SPA nav

JavaScript Collector (signals.js)

New file. Lightweight, no dependencies. Loaded on all product/content pages.

Page load:
  → Record start time
  → Attach debounced mousemove/touchstart/keydown → active flag
  → Attach visibilitychange → accumulate visible_ms
  → Attach scroll → track depth + direction changes
  → Attach media events (play/pause/ended/seeked/ratechange)
  → Generate random session token (in-memory only, never persisted)

Page unload (beforeunload / visibilitychange → hidden):
  → Bundle all signals into JSON
  → POST via navigator.sendBeacon('/signals/beacon')
  → Fallback: fetch() with keepalive: true

The beacon payload is ~300 bytes of JSON. One request per page visit. No polling, no websockets, no streaming.

Server Endpoint

POST /signals/beacon — accepts JSON, validates fields, computes referrer_class from the Referer header, inserts one mps_page_session row. If visible_ms >= 7000, also increments product.view_count. Returns 204.

Rate limiting: max 1 beacon per product per client per 60 seconds, enforced by a short-lived server-side cache keyed on (session_token, product_id). The session token is random per page load — it prevents double-counting from accidental double-fires, not tracking users across pages.

Files Changed

File	Change
models/page_session.py	New model: MpsPageSession
models/product.py	Add view_count column, human_view_count property
models/__init__.py	Import new model
views/signals.py	New: beacon endpoint
views/watch.py	Include view_count / human_view_count in JSON
views/product.py	Pass view count to template context
views/content.py	Pass view count to template context
static/js/signals.js	New: client-side signal collector
templates/content.j2	Show view count in .product-meta-dates
templates/product.j2	Show view count in .product-meta-dates
templates/snippets/base_styles.j2	Load signals.js on product pages
static/js/watch.js	Update view count in updatePageContent()
scripts/alembic/versions/	Migration: mps_page_session table + view_count column
tests/test_functional.py	Beacon endpoint tests, view count display tests
tests/test_models.py	human_view_count formatting tests

What This Does NOT Collect

• IP addresses
• User agent strings (device class is viewport width only)
• Cookies or persistent identifiers
• Mouse/touch coordinates
• Clicked element identifiers
• Any form input content
• Referrer URLs (only domain + search query parameter — see MPS-6)
• Cross-page session linking (each page visit is an island)

A viewer could visit every product in a shop and there would be no way to connect those visits to the same person. That is by design.