# MPS-6: Referrer Analytics — Domain, Query, and Trend Lines

## Problem

MPS-2 stored only a numeric `referrer_class` (0-4) on each page session. This
tells the creator "42% of traffic is from search" but not *which* search engine,
*which* social platform, or *what keywords* people searched. Creators need
actionable referrer intelligence to know where to spend their marketing energy.

The analytics dashboard also lacked trend visualization — all metrics were
point-in-time tables with no temporal context.

## Solution

### 1. Store referrer domain and search query

Add two columns to `mps_page_session`:

| Column | Type | Example |
|--------|------|---------|
| `referrer_domain` | `Unicode(128)` | `"www.google.com"`, `"twitter.com"` |
| `referrer_query` | `Unicode(256)` | `"lo-fi beats to study to"` |

These are populated by refactoring `classify_referrer()` from returning a single
integer to returning a `(class, domain, query)` tuple. The domain is extracted
from the Referer header URL. The query is parsed from search engine URL
parameters (`q=` for Google/DuckDuckGo/Bing, `p=` for Yahoo).

Privacy: the full Referer URL is never stored. Only the domain and the search
query parameter (if present) are kept. Social and unknown referrers store domain
only.

### 2. Add SVG line chart trend visualization

Server-rendered inline SVG polyline charts for temporal trends. No JavaScript
graph libraries — the charts are computed server-side and rendered as a Jinja2
macro (`line_chart`) that produces `<svg>` elements with `<polyline>` paths.

Five trend lines added to both shop-level and per-product analytics:

| Chart | Y-axis | Color |
|-------|--------|-------|
| Session duration | Avg `wall_clock_ms` (daily) | Blue |
| Engagement | Avg `active_ms / wall_clock_ms` (daily) | Green |
| Bounce rate | % sessions with `visible_ms < 7s` (daily) | Red |
| External referrers | Daily count of non-direct, non-internal sessions | Orange |

Each chart spans 28 days with one data point per day. Zero-days are filled so
the polyline is continuous.

### 3. Add keyword and referrer domain tables

Two new analytics sections:

**Top Referrer Domains** — ranked table of external domains driving traffic,
with session count and percentage bar. Excludes direct and internal traffic.

**Top Search Queries** — ranked table of search engine queries that led to the
shop, extracted from `referrer_query`. Also includes internal shop search
keywords from `mps_shop_search_request`.

### 4. Bucketing functions

Six new query functions in `analytics.py`:

- `_daily_buckets()` — daily view counts (bar chart)
- `_daily_avg_duration()` — daily avg session duration
- `_daily_engagement()` — daily avg engagement ratio
- `_daily_bounce_rate()` — daily bounce rate
- `_daily_referrer_counts()` — daily external referrer sessions
- `_top_referrer_domains()` — top N referrer domains
- `_top_referrer_queries()` — top N search engine queries
- `_top_search_keywords()` — top N internal search keywords

## classify_referrer() Refactor

Before (MPS-2):
```python
def classify_referrer(referrer, request_host):
    """Return 0-4 integer class."""
    return 1  # search
```

After (MPS-6):
```python
def classify_referrer(referrer, request_host):
    """Return (class, domain, query) tuple."""
    return (1, "www.google.com", "lo-fi beats")
```

Search engine query extraction:
- Google/Bing/DuckDuckGo: `?q=` parameter
- Yahoo: `?p=` parameter
- Other search engines: `?q=` fallback

## Files Changed

| File | Change |
|------|--------|
| `models/page_session.py` | Add `referrer_domain`, `referrer_query` columns |
| `views/signals.py` | Refactor `classify_referrer()` to return tuple; store domain + query |
| `views/analytics.py` | 8 new bucketing/query functions; pass trend data to templates |
| `templates/analytics.j2` | SVG `line_chart` macro; referrer domain table; keyword table |
| `templates/analytics_product.j2` | SVG `line_chart` macro; referrer domain table |
| `scripts/alembic/versions/f3086e09b052_*.py` | Migration: add `referrer_domain`, `referrer_query` |
| `tests/test_models.py` | 12 tests for `classify_referrer()` tuple return |
| `tests/test_functional.py` | Referrer trend chart rendering tests |

## Depends On

MPS-3 (analytics dashboard infrastructure)
