# MPS-2: Anonymous Signal Gathering & View Count

## Philosophy

Collect every honest signal we can about how people experience content —
without recording who they are. No cookies, no IPs, no fingerprints, no user
agent strings. A viewer's identity is never captured. Only the aggregate truth
of what happened on the page.

The raw signals feed four derived scores: **Engagement**, **Attention**,
**Learning**, and **Passive Consumption**. These scores are computed from simple
math now and can be refined with machine learning later — the raw data is the
same either way.

A **view** counts after 7 seconds of wall-clock time on the page with the tab
visible. This respects all four pillars: truthful (they were really here), free
(no judgment on how they consume), harmonious (same bar for a 30-second clip
and a 2-hour film), and loving (assumes good intent from the visitor).

## What We Collect

All signals are gathered in JavaScript memory during a page session and sent as
a single JSON summary via `navigator.sendBeacon()` on page unload. One beacon
per page visit. Nothing streams, nothing phones home mid-session.

### Presence Signals

| Signal | How | Why |
|--------|-----|-----|
| `wall_clock_ms` | `Date.now()` delta from page load to unload | Total time on page |
| `visible_ms` | Page Visibility API (`visibilitychange`) — accumulate time while `visible` | Tab in foreground vs backgrounded |
| `active_ms` | Accumulate time while mouse/touch/keyboard events fire (debounce 2s idle gap) | Human is present and interacting |
| `idle_ms` | `wall_clock_ms - active_ms` | Derived, not tracked separately |

**Mouse/touch activity**: Listen for `mousemove`, `touchstart`, `keydown` —
debounced to a boolean "active" flag that goes idle after 2 seconds of silence.
We track **whether** the user is active, never **where** on the page. No
coordinates, no element targets.

### Scroll Signals

| Signal | How | Why |
|--------|-----|-----|
| `scroll_depth_max` | `window.scrollY / document.body.scrollHeight` — track max (0.0-1.0) | How far down the page they went |
| `scroll_direction_changes` | Count sign changes in scroll delta | Reading (few reversals) vs scanning (many) |

### Media Playback Signals

Piggyback on existing `timeupdate`, `play`, `pause`, `ended`, `seeked`,
`ratechange` events already wired in watch.js.

| Signal | How | Why |
|--------|-----|-----|
| `media_duration_ms` | `media.duration * 1000` on `loadedmetadata` | Length of the content |
| `media_play_ms` | Accumulate time between `play` and `pause`/`ended` events | Actual playback time |
| `media_play_count` | Count `play` events | Replays = love |
| `media_pause_count` | Count `pause` events (user-initiated, not ended) | Interaction with content |
| `media_seek_count` | Count `seeked` events | Skipping around (study or skip?) |
| `media_seek_back_count` | Count seeks where `currentTime` decreased | Rewinding = learning |
| `media_speed` | Last value from `ratechange` event, default 1.0 | 2x = efficient, 0.5x = studying |
| `media_completed` | Boolean: did `ended` event fire at least once | Watched/listened to the end |
| `media_percent_played` | `media_play_ms / media_duration_ms` (can exceed 1.0 on replay) | Completion without judging speed |

### Context Signals (server-side, from the HTTP request)

| Signal | How | Why |
|--------|-----|-----|
| `is_ring_entry` | Boolean: full page load vs SPA transition | Front door vs ring navigation |
| `ring_position` | Integer: how many SPA transitions from entry | Depth in the ring |
| `referrer_class` | Enum: `direct`, `search`, `social`, `internal`, `unknown` | Where traffic comes from — derived from Referer header domain, NOT the full URL |
| `referrer_domain` | String (128 chars) | Referer hostname (e.g. "www.google.com") — added in MPS-6 |
| `referrer_query` | String (256 chars) | Search engine query parameter — added in MPS-6 |
| `device_class` | Enum: `mobile`, `tablet`, `desktop` | Screen size bucket only, not user agent — derived from viewport width sent in beacon |

`referrer_class` is computed server-side by matching the Referer header domain
against known patterns (google/bing/ddg → search, twitter/facebook/reddit →
social, same domain → internal, everything else → unknown). The raw Referer URL
is never stored.

`device_class` is derived from viewport width sent in the beacon payload:
≤768 → mobile, ≤1024 → tablet, else desktop. No user agent parsing.

## Storage

### Per-session table: `mps_page_session`

One row per page visit. This is the raw truth.

```
id                      UUID PK
shop_id                 FK → mps_shop
product_id              FK → mps_product
created_timestamp       BigInteger (ms) — when beacon was received

-- presence
wall_clock_ms           Integer
visible_ms              Integer
active_ms               Integer

-- scroll
scroll_depth_max        Float (0.0-1.0)
scroll_direction_changes Integer

-- media
media_duration_ms       Integer (NULL if no media)
media_play_ms           Integer (NULL if no media)
media_play_count        SmallInteger
media_pause_count       SmallInteger
media_seek_count        SmallInteger
media_seek_back_count   SmallInteger
media_speed             Float
media_completed         Boolean
media_percent_played    Float

-- context
is_ring_entry           Boolean
ring_position           SmallInteger
referrer_class          SmallInteger (enum: 0=direct 1=search 2=social 3=internal 4=unknown)
referrer_domain         Unicode(128) — extracted domain (e.g. "www.google.com")
referrer_query          Unicode(256) — search engine query (e.g. "lo-fi beats")
device_class            SmallInteger (enum: 0=mobile 1=tablet 2=desktop)
```

No foreign key to any user/account table. No IP column. No session cookie
column. The row is an anonymous fact about what happened, nothing more.

### Aggregate columns on `mps_product`

Updated asynchronously (on beacon receipt or periodic batch):

- `view_count` (BigInteger, server_default="0") — incremented when a session
  hits the 7-second visible threshold

### Retention

Keep individual `mps_page_session` rows for 90 days. After that, aggregate
into `mps_product_daily_stats` (one row per product per day with averages and
sums) and delete the raw sessions. The daily rollup lives forever.

## Derived Scores

Computed from raw signals, either on-read (in Python properties) or in the
daily rollup. These are the creator-facing metrics.

### Engagement (0.0 – 1.0)

```
engagement = active_ms / wall_clock_ms
```

High = clicking, scrolling, pausing, seeking — actively interacting with the
content. Low = tab open but idle. This is the "lean-in" metric.

### Attention (0.0 – 1.0)

```
attention = visible_ms / wall_clock_ms
```

High = tab in foreground the entire time. Low = they switched tabs. Combined
with engagement: high attention + low engagement = focused passive watching.
High attention + high engagement = deeply studying.

### Learning Signal (heuristic score)

Indicators of active study, not passive consumption:

- `media_seek_back_count > 0` — they rewound to re-hear something
- `media_speed < 1.0` — slowed down to absorb
- `scroll_direction_changes > 3` — re-reading sections
- `media_pause_count > 2` — pausing to think or take notes
- `engagement > 0.7` — high interaction throughout

Score: count of true indicators (0-5). A session with 3+ is "learning mode."

### Passive Consumption (heuristic score)

Indicators of lean-back enjoyment:

- `attention > 0.7` — tab is in foreground
- `engagement < 0.3` — but not interacting much
- `media_percent_played > 0.69` — played most/all of the content
- `media_play_count == 1` — straight through, no replays
- `media_pause_count == 0` — never paused

Score: count of true indicators (0-5). A session with 3+ is "passive
consumption" — and this is a GOOD signal. It means the content is worth
playing through without interruption.

### Machine Learning Hook

The raw `mps_page_session` rows are feature vectors. Every field is a number.
When ML is added later, it can:

- Cluster sessions into behavioral profiles beyond the 4 heuristic scores
- Detect anomalies (bot patterns: zero scroll + zero active + high play count)
- Predict which products will retain based on early session patterns
- Surface "hidden gems" — products with high learning/attention scores but
  low view counts

The collection schema doesn't change. ML consumes the same rows and produces
richer derived scores that feed into dashboards. The raw truth stays raw.

## View Count Display

Show the human-readable view count on product and content pages, next to the
upload date under the title h1:

```
Track Title
uploaded to ShopName
Created 3 days ago · 420 views
```

### Formatting

- 0: don't show view count at all (no "0 views")
- 1: "1 view"
- 2-999: exact number ("420 views")
- 1,000-999,999: "1.2K views"
- 1,000,000+: "1.8M views"

### Three-layer update (SPA compatibility)

1. **Templates** (`content.j2`, `product.j2`): render in `.product-meta-dates`
2. **watch.py** (`watch_json`): include `view_count` and `human_view_count`
3. **watch.js** (`updatePageContent`): update the view count span during SPA nav

## JavaScript Collector (`signals.js`)

New file. Lightweight, no dependencies. Loaded on all product/content pages.

```
Page load:
  → Record start time
  → Attach debounced mousemove/touchstart/keydown → active flag
  → Attach visibilitychange → accumulate visible_ms
  → Attach scroll → track depth + direction changes
  → Attach media events (play/pause/ended/seeked/ratechange)
  → Generate random session token (in-memory only, never persisted)

Page unload (beforeunload / visibilitychange → hidden):
  → Bundle all signals into JSON
  → POST via navigator.sendBeacon('/signals/beacon')
  → Fallback: fetch() with keepalive: true
```

The beacon payload is ~300 bytes of JSON. One request per page visit. No
polling, no websockets, no streaming.

## Server Endpoint

`POST /signals/beacon` — accepts JSON, validates fields, computes
`referrer_class` from the Referer header, inserts one `mps_page_session` row.
If `visible_ms >= 7000`, also increments `product.view_count`. Returns 204.

Rate limiting: max 1 beacon per product per client per 60 seconds, enforced
by a short-lived server-side cache keyed on `(session_token, product_id)`.
The session token is random per page load — it prevents double-counting
from accidental double-fires, not tracking users across pages.

## Files Changed

| File | Change |
|------|--------|
| `models/page_session.py` | New model: `MpsPageSession` |
| `models/product.py` | Add `view_count` column, `human_view_count` property |
| `models/__init__.py` | Import new model |
| `views/signals.py` | New: beacon endpoint |
| `views/watch.py` | Include `view_count` / `human_view_count` in JSON |
| `views/product.py` | Pass view count to template context |
| `views/content.py` | Pass view count to template context |
| `static/js/signals.js` | New: client-side signal collector |
| `templates/content.j2` | Show view count in `.product-meta-dates` |
| `templates/product.j2` | Show view count in `.product-meta-dates` |
| `templates/snippets/base_styles.j2` | Load signals.js on product pages |
| `static/js/watch.js` | Update view count in `updatePageContent()` |
| `scripts/alembic/versions/` | Migration: `mps_page_session` table + `view_count` column |
| `tests/test_functional.py` | Beacon endpoint tests, view count display tests |
| `tests/test_models.py` | `human_view_count` formatting tests |

## What This Does NOT Collect

- IP addresses
- User agent strings (device class is viewport width only)
- Cookies or persistent identifiers
- Mouse/touch coordinates
- Clicked element identifiers
- Any form input content
- Referrer URLs (only domain + search query parameter — see MPS-6)
- Cross-page session linking (each page visit is an island)

A viewer could visit every product in a shop and there would be no way to
connect those visits to the same person. That is by design.