Isolate parallel test runs with PID-based database filenames

When two processes run pytest simultaneously, they no longer share
test_make_post_sell_master.sqlite. Each gets its own file keyed by PID,
preventing table-exists and readonly-database errors. Also clean up
WAL/SHM journal files on exit.
This commit is contained in:
russell@unturf.com 2026-02-09 16:24:25 -05:00
parent e1a8b2f236
commit fca2f4a051
10 changed files with 1085 additions and 34 deletions

View file

@ -4,48 +4,192 @@
MPS-2 collects anonymous signals and derives Engagement, Attention, Learning,
and Passive Consumption scores. Creators need somewhere to see this data and
act on it.
act on it — know which content pulls people in, which holds them, and where
traffic comes from so they know where to spend time and energy.
## Solution
Add a simple analytics page accessible from the shop dashboard. No graphs
library — clean server-rendered HTML with CSS grid tables. Machine learning
refinements to the derived scores can enhance this page later without changing
its structure.
Add a server-rendered analytics page at `/s/{shop_id}/analytics`. No JS graph
libraries — CSS grid tables, server-computed aggregates, progressive
enhancement. Machine learning can refine the derived scores later without
changing the page structure.
### Content
## Access
- **Top products by views** — ranked list, last 7 / 14 / 21 days
- **Ring entry points** — which products are "front doors" (top 7 by `is_ring_entry` count)
- **Average ring depth** — how far viewers ride the ring (mean `ring_position`)
- **Daily view totals** — last 21 days, per product and shop-wide
- **Engagement leaders** — products with highest average engagement score
- **Attention holders** — products with highest average attention score
- **Study material** — products with highest learning signal (people rewind, slow down, re-read)
- **Background favorites** — products with highest passive consumption score (lean-back plays)
- **Traffic sources** — breakdown by `referrer_class` (direct / search / social / internal)
- **Device split** — mobile vs tablet vs desktop percentages
- Route: `GET /s/{shop_id}/analytics`
- Requires `shop_editor_required` (same as shop settings)
- Link from shop settings page (near existing nav)
### Access
## Page Sections
New route `/shop/analytics` — only visible to shop owner/mods. Link from shop
settings or dashboard nav.
### 1. Overview Strip (shop-wide, last 7 days)
### Privacy
A single row of key numbers at the top:
All data is anonymous aggregates computed from `mps_page_session` rows. No
individual viewer data exists to display even if someone wanted to.
| Metric | Query |
|--------|-------|
| Total views | `COUNT(*) WHERE visible_ms >= 7000 AND created_timestamp > 7d ago` |
| Unique products viewed | `COUNT(DISTINCT product_id) WHERE visible_ms >= 7000` |
| Avg session duration | `AVG(wall_clock_ms)` |
| Avg ring depth | `AVG(ring_position) WHERE ring_position IS NOT NULL` |
| Top traffic source | `MODE(referrer_class)` (show label: direct/search/social/internal) |
| Device split | `% per device_class` shown as "42% mobile · 7% tablet · 51% desktop" |
### 2. Top Products by Views (last 7 / 14 / 21 days)
Ranked table, top 21 products:
| # | Title | Views (7d) | Views (14d) | Views (21d) | Trend |
|---|-------|-----------|------------|------------|-------|
"Trend" = simple arrow: views(7d) > views(14d)/2 → rising, else falling.
Each title links to the product page.
Query: `GROUP BY product_id`, `COUNT(*) WHERE visible_ms >= 7000`, partitioned
by time windows using `created_timestamp`.
### 3. Ring Entry Points (top 7 front doors)
Which products do people land on first?
| # | Title | Ring Entries (21d) | % of All Entries |
|---|-------|--------------------|-----------------|
Query: `COUNT(*) WHERE is_ring_entry = true GROUP BY product_id ORDER BY
count DESC LIMIT 7`.
This tells the creator: "People find your shop through these 7 products — make
sure they're polished."
### 4. Engagement & Attention Leaders (top 7 each)
Two side-by-side tables:
**Engagement Leaders** (highest lean-in):
| # | Title | Avg Engagement | Sessions |
|---|-------|---------------|----------|
`engagement = active_ms / wall_clock_ms` — computed per session, averaged per
product. Only include sessions with `wall_clock_ms >= 7000` (real visits).
**Attention Holders** (highest focused presence):
| # | Title | Avg Attention | Sessions |
|---|-------|--------------|----------|
`attention = visible_ms / wall_clock_ms` — same filtering.
### 5. Study Material vs Background Favorites (top 7 each)
Two side-by-side tables:
**Study Material** (people rewind, slow down, re-read):
| # | Title | Learning Score | Sessions |
|---|-------|---------------|----------|
Learning score per session = count of true indicators:
- `media_seek_back_count > 0`
- `media_speed < 1.0` (and not NULL)
- `scroll_direction_changes > 3`
- `media_pause_count > 2`
- `active_ms / wall_clock_ms > 0.7`
Average per product, ranked. Minimum 7 sessions to qualify.
**Background Favorites** (lean-back plays):
| # | Title | Passive Score | Sessions |
|---|-------|--------------|----------|
Passive score per session = count of true indicators:
- `visible_ms / wall_clock_ms > 0.7`
- `active_ms / wall_clock_ms < 0.3`
- `media_percent_played > 0.69`
- `media_play_count = 1`
- `media_pause_count = 0`
Average per product, ranked. Minimum 7 sessions to qualify.
### 6. Traffic Sources (last 21 days)
Simple breakdown table:
| Source | Sessions | % |
|--------|----------|---|
| Direct | 142 | 42% |
| Search | 69 | 21% |
| Social | 47 | 14% |
| Internal | 70 | 21% |
| Unknown | 7 | 2% |
Query: `COUNT(*) GROUP BY referrer_class`.
### 7. Device Split (last 21 days)
| Device | Sessions | % |
|--------|----------|---|
| Mobile | 210 | 42% |
| Tablet | 42 | 8% |
| Desktop | 252 | 50% |
Query: `COUNT(*) GROUP BY device_class`.
## Query Strategy
All queries run against `mps_page_session` with time filters using
`created_timestamp`. Since timestamps are milliseconds:
```python
cutoff_7d = now_timestamp() - (7 * 24 * 60 * 60 * 1000)
cutoff_14d = now_timestamp() - (14 * 24 * 60 * 60 * 1000)
cutoff_21d = now_timestamp() - (21 * 24 * 60 * 60 * 1000)
```
For the engagement/attention/learning/passive scores, compute per-session in
the SQL query using CASE expressions, then AVG per product. SQLite handles
this fine for shops with < 100K sessions.
For larger shops (future), the 90-day retention + daily rollup from MPS-2
provides pre-aggregated data.
## Template Layout
CSS grid, two-column on desktop, single-column on mobile. No flexbox per
project rules.
```
[Overview Strip — full width]
[Top Products — full width]
[Ring Entry Points — full width]
[Engagement Leaders | Attention Holders — side by side]
[Study Material | Background Favorites — side by side]
[Traffic Sources | Device Split — side by side]
```
Tables use `<table>` with `class="analytics-table"`. No zebra striping —
keep it clean. Product titles are links. Numbers right-aligned.
## Privacy Note
Displayed in a small footer on the page:
> All data is anonymous. No individual viewer can be identified. Counts
> represent aggregate sessions, not people.
## Files Changed
| File | Change |
|------|--------|
| `views/shop.py` | New `analytics` view with aggregate queries |
| `templates/analytics.j2` | New template |
| `static/css/common.css` | Analytics table styles |
| `templates/snippets/shop_nav.j2` | Link to analytics (if exists) |
| `tests/test_functional.py` | Analytics page access tests |
| `routes.py` | New route `shop_analytics` |
| `templates/analytics.j2` | New template with 7 sections |
| `static/css/common.css` | `.analytics-table`, `.analytics-overview` styles |
| `templates/shop_settings.j2` | Link to analytics page |
| `tests/test_functional.py` | Analytics page access + permission tests |
| `tests/test_models.py` | Score computation unit tests |
## Depends On
MPS-2 (signal gathering + `mps_page_session` table)
MPS-2 (signal gathering + `mps_page_session` table + `view_count` column)

View file

@ -122,6 +122,7 @@ def includeme(config):
config.add_route("shop_sales", "/s/{shop_id}/sales")
config.add_route("shop_comments", "/s/{shop_id}/comments")
config.add_route("shop_analytics", "/s/{shop_id}/analytics")
config.add_route("shop_settings", "/s/{shop_id}/settings")
config.add_route(

View file

@ -3177,3 +3177,117 @@ textarea {
gap: 32px;
}
}
/* Analytics Dashboard */
.analytics-page {
max-width: 1000px;
margin-left: auto;
margin-right: auto;
padding: 0 10px;
}
.analytics-overview {
display: grid;
grid-template-columns: repeat(2, 1fr);
gap: 16px;
margin-bottom: 30px;
}
.analytics-overview-item {
display: grid;
gap: 2px;
text-align: center;
padding: 10px;
}
.analytics-overview-value {
font-size: 1.4em;
font-weight: bold;
color: var(--text-primary);
}
.analytics-overview-label {
font-size: 0.8em;
color: var(--text-muted);
text-transform: uppercase;
letter-spacing: 0.05em;
}
.analytics-table-wrap {
overflow-x: auto;
margin-bottom: 30px;
}
.analytics-table {
width: 100%;
border-collapse: collapse;
}
.analytics-table th {
font-size: 0.8em;
text-transform: uppercase;
letter-spacing: 0.05em;
color: var(--text-muted);
border-bottom: 2px solid var(--border-color, #dee2e6);
padding: 6px 8px;
}
.analytics-table td {
padding: 6px 8px;
border-bottom: 1px solid var(--border-light, #e9ecef);
}
.analytics-table tbody tr:last-child td {
border-bottom: none;
}
.analytics-num {
text-align: right;
font-variant-numeric: tabular-nums;
}
.analytics-trend-up {
color: var(--green-color, #a3c765);
}
.analytics-trend-down {
color: var(--text-muted, #999);
}
.analytics-hint {
font-size: 0.85em;
color: var(--text-muted);
margin-top: 0;
margin-bottom: 10px;
}
.analytics-pair {
display: grid;
grid-template-columns: 1fr;
gap: 30px;
margin-bottom: 30px;
}
.analytics-privacy {
font-size: 0.8em;
color: var(--text-muted);
margin-top: 40px;
padding-top: 20px;
border-top: 1px solid var(--border-light, #e9ecef);
}
/* Override default table zebra striping inside analytics tables */
.analytics-table tr:nth-child(even),
.analytics-table tr:nth-child(odd) {
background-color: transparent;
}
@media (min-width: 960px) {
.analytics-overview {
grid-template-columns: repeat(3, 1fr);
}
.analytics-pair {
grid-template-columns: 1fr 1fr;
}
}

View file

@ -40,6 +40,10 @@
<br/>
<a href="/s/{{ request.shop.id }}/comments" class="mps-button product-edit-button">💬 &nbsp Shop Comments</a>
<br/>
<br/>
<a href="/s/{{ request.shop.id }}/analytics" class="mps-button product-edit-button">📊 &nbsp Shop Analytics</a>
<br/>
<br/>
<a href="/s/{{ request.shop.id }}/settings" class="mps-button product-edit-button">⚙ &nbsp Shop Settings</a>

View file

@ -0,0 +1,281 @@
{% extends "base.j2" -%}
{%- block append_to_head_tag_section %}
<title>Analytics - {{ request.shop.name }}</title>
{%- endblock append_to_head_tag_section -%}
{% block content -%}
<section class="analytics-page">
<h3>Analytics</h3>
{% if overview.total_views == 0 %}
<p>No view data yet. Analytics appear after visitors spend at least 7 seconds on your content.</p>
{% else %}
{# --- Section 1: Overview Strip (last 7 days) --- #}
<div class="analytics-overview well">
<div class="analytics-overview-item">
<span class="analytics-overview-value">{{ overview.total_views }}</span>
<span class="analytics-overview-label">Views (7d)</span>
</div>
<div class="analytics-overview-item">
<span class="analytics-overview-value">{{ overview.unique_products }}</span>
<span class="analytics-overview-label">Products Viewed</span>
</div>
<div class="analytics-overview-item">
<span class="analytics-overview-value">{{ overview.avg_session }}</span>
<span class="analytics-overview-label">Avg Session</span>
</div>
<div class="analytics-overview-item">
<span class="analytics-overview-value">{{ overview.avg_ring_depth }}</span>
<span class="analytics-overview-label">Avg Ring Depth</span>
</div>
<div class="analytics-overview-item">
<span class="analytics-overview-value">{{ overview.top_source }}</span>
<span class="analytics-overview-label">Top Source</span>
</div>
<div class="analytics-overview-item">
<span class="analytics-overview-value">{{ overview.device_split }}</span>
<span class="analytics-overview-label">Devices (7d)</span>
</div>
</div>
{# --- Section 2: Top Products by Views --- #}
{% if top_products %}
<h4>Top Products by Views</h4>
<div class="analytics-table-wrap">
<table class="analytics-table">
<thead>
<tr>
<th>#</th>
<th>Title</th>
<th class="analytics-num">7d</th>
<th class="analytics-num">14d</th>
<th class="analytics-num">21d</th>
<th class="analytics-num">Trend</th>
</tr>
</thead>
<tbody>
{% for p in top_products %}
<tr>
<td>{{ p.rank }}</td>
<td><a href="{{ p.url }}">{{ p.title }}</a></td>
<td class="analytics-num">{{ p.views_7d }}</td>
<td class="analytics-num">{{ p.views_14d }}</td>
<td class="analytics-num">{{ p.views_21d }}</td>
<td class="analytics-num">{% if p.trend == "rising" %}<span class="analytics-trend-up">&#9650;</span>{% else %}<span class="analytics-trend-down">&#9660;</span>{% endif %}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endif %}
{# --- Section 3: Ring Entry Points --- #}
{% if ring_entries %}
<h4>Ring Entry Points (top 7 front doors, last 21 days)</h4>
<p class="analytics-hint">People find your shop through these products &mdash; make sure they're polished.</p>
<div class="analytics-table-wrap">
<table class="analytics-table">
<thead>
<tr>
<th>#</th>
<th>Title</th>
<th class="analytics-num">Entries</th>
<th class="analytics-num">% of All</th>
</tr>
</thead>
<tbody>
{% for r in ring_entries %}
<tr>
<td>{{ r.rank }}</td>
<td><a href="{{ r.url }}">{{ r.title }}</a></td>
<td class="analytics-num">{{ r.count }}</td>
<td class="analytics-num">{{ r.pct }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endif %}
{# --- Sections 4 & 5: Engagement & Attention (side by side) --- #}
{% if engagement or attention %}
<div class="analytics-pair">
{% if engagement %}
<div>
<h4>Engagement Leaders</h4>
<p class="analytics-hint">Highest lean-in: active interaction vs wall clock.</p>
<table class="analytics-table">
<thead>
<tr>
<th>#</th>
<th>Title</th>
<th class="analytics-num">Avg</th>
<th class="analytics-num">Sessions</th>
</tr>
</thead>
<tbody>
{% for e in engagement %}
<tr>
<td>{{ e.rank }}</td>
<td><a href="{{ e.url }}">{{ e.title }}</a></td>
<td class="analytics-num">{{ e.score }}</td>
<td class="analytics-num">{{ e.sessions }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endif %}
{% if attention %}
<div>
<h4>Attention Holders</h4>
<p class="analytics-hint">Highest focused presence: visible time vs wall clock.</p>
<table class="analytics-table">
<thead>
<tr>
<th>#</th>
<th>Title</th>
<th class="analytics-num">Avg</th>
<th class="analytics-num">Sessions</th>
</tr>
</thead>
<tbody>
{% for a in attention %}
<tr>
<td>{{ a.rank }}</td>
<td><a href="{{ a.url }}">{{ a.title }}</a></td>
<td class="analytics-num">{{ a.score }}</td>
<td class="analytics-num">{{ a.sessions }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endif %}
</div>
{% endif %}
{# --- Sections 6 & 7: Study Material & Background Favorites (side by side) --- #}
{% if learning or passive %}
<div class="analytics-pair">
{% if learning %}
<div>
<h4>Study Material</h4>
<p class="analytics-hint">People rewind, slow down, re-read. Score out of 5.</p>
<table class="analytics-table">
<thead>
<tr>
<th>#</th>
<th>Title</th>
<th class="analytics-num">Score</th>
<th class="analytics-num">Sessions</th>
</tr>
</thead>
<tbody>
{% for l in learning %}
<tr>
<td>{{ l.rank }}</td>
<td><a href="{{ l.url }}">{{ l.title }}</a></td>
<td class="analytics-num">{{ l.score }}</td>
<td class="analytics-num">{{ l.sessions }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endif %}
{% if passive %}
<div>
<h4>Background Favorites</h4>
<p class="analytics-hint">Lean-back plays: visible but hands-off. Score out of 5.</p>
<table class="analytics-table">
<thead>
<tr>
<th>#</th>
<th>Title</th>
<th class="analytics-num">Score</th>
<th class="analytics-num">Sessions</th>
</tr>
</thead>
<tbody>
{% for b in passive %}
<tr>
<td>{{ b.rank }}</td>
<td><a href="{{ b.url }}">{{ b.title }}</a></td>
<td class="analytics-num">{{ b.score }}</td>
<td class="analytics-num">{{ b.sessions }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endif %}
</div>
{% endif %}
{# --- Sections 8 & 9: Traffic Sources & Device Split (side by side) --- #}
{% if traffic or devices %}
<div class="analytics-pair">
{% if traffic %}
<div>
<h4>Traffic Sources (last 21 days)</h4>
<table class="analytics-table">
<thead>
<tr>
<th>Source</th>
<th class="analytics-num">Sessions</th>
<th class="analytics-num">%</th>
</tr>
</thead>
<tbody>
{% for t in traffic %}
<tr>
<td>{{ t.source }}</td>
<td class="analytics-num">{{ t.count }}</td>
<td class="analytics-num">{{ t.pct }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endif %}
{% if devices %}
<div>
<h4>Device Split (last 21 days)</h4>
<table class="analytics-table">
<thead>
<tr>
<th>Device</th>
<th class="analytics-num">Sessions</th>
<th class="analytics-num">%</th>
</tr>
</thead>
<tbody>
{% for d in devices %}
<tr>
<td>{{ d.device }}</td>
<td class="analytics-num">{{ d.count }}</td>
<td class="analytics-num">{{ d.pct }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endif %}
</div>
{% endif %}
{% endif %}{# end overview.total_views check #}
<p class="analytics-privacy">All data is anonymous. No individual viewer can be identified. Counts represent aggregate sessions, not people.</p>
</section>
{%- endblock -%}

View file

@ -16,7 +16,7 @@ def pytest_configure(config):
SQLite locking conflicts. WAL mode is enabled for better concurrency.
"""
# Get worker ID (e.g., "gw0", "gw1", etc.) for pytest-xdist
worker_id = os.environ.get("PYTEST_XDIST_WORKER", "master")
worker_id = os.environ.get("PYTEST_XDIST_WORKER", f"pid{os.getpid()}")
# Set unique database path for this worker
test_db_path = f"test_make_post_sell_{worker_id}.sqlite"
@ -30,11 +30,13 @@ def pytest_unconfigure(config):
"""Clean up test database after all tests complete."""
if hasattr(config, "test_db_path"):
db_path = config.test_db_path
if os.path.exists(db_path):
try:
os.remove(db_path)
except Exception as e:
print(f"Warning: Could not remove test database {db_path}: {e}")
for suffix in ("", "-wal", "-shm"):
path = db_path + suffix
if os.path.exists(path):
try:
os.remove(path)
except Exception as e:
print(f"Warning: Could not remove {path}: {e}")
@pytest.fixture(scope="session")
@ -48,7 +50,7 @@ def db_engine(request):
from sqlalchemy import create_engine, event
# Use worker-specific database
worker_id = os.environ.get("PYTEST_XDIST_WORKER", "master")
worker_id = os.environ.get("PYTEST_XDIST_WORKER", f"pid{os.getpid()}")
db_path = f"test_make_post_sell_{worker_id}.sqlite"
db_url = f"sqlite:///{db_path}"

View file

@ -3413,3 +3413,83 @@ class TestBeacon(AuthenticatedFunctionalTests):
.count()
)
self.assertEqual(count2, 1)
class TestAnalytics(AuthenticatedFunctionalTests):
"""Functional tests for the /s/{shop_id}/analytics page."""
def test_analytics_requires_editor(self):
"""Unauthenticated users get redirected from analytics."""
shop = self._create_shop_helper()
shop_id = str(shop.id)
self.testapp.get("/log-out")
res = self.testapp.get(f"/s/{shop_id}/analytics", status=302)
self.assertEqual(res.status_int, 302)
def test_analytics_accessible_by_owner(self):
"""Shop owner can access the analytics page."""
shop = self._create_shop_helper()
shop_id = str(shop.id)
res = self.testapp.get(f"/s/{shop_id}/analytics", status=200)
self.assertIn("Analytics", res.body.decode())
def test_analytics_empty_state(self):
"""Analytics page shows empty state message when no data exists."""
shop = self._create_shop_helper()
shop_id = str(shop.id)
res = self.testapp.get(f"/s/{shop_id}/analytics", status=200)
body = res.body.decode()
self.assertIn("No view data yet", body)
def test_analytics_shows_overview_with_data(self):
"""Analytics page shows overview strip when session data exists."""
shop, product = self._setup_shop_and_product()
shop_id = str(shop.id)
# Insert a session via beacon with visible_ms >= 7000
self.testapp.post_json(
"/signals/beacon",
{
"product_id": str(product.id),
"shop_id": shop_id,
"session_token": "analytics_test_1",
"wall_clock_ms": 15000,
"visible_ms": 12000,
"active_ms": 8000,
"viewport_width": 1920,
},
status=204,
)
res = self.testapp.get(f"/s/{shop_id}/analytics", status=200)
body = res.body.decode()
# Should show overview, not empty state
self.assertNotIn("No view data yet", body)
self.assertIn("Views (7d)", body)
def test_analytics_privacy_note(self):
"""Analytics page always shows the privacy note."""
shop = self._create_shop_helper()
shop_id = str(shop.id)
res = self.testapp.get(f"/s/{shop_id}/analytics", status=200)
body = res.body.decode()
self.assertIn("anonymous", body)
def _setup_shop_and_product(self):
"""Create a shop and a content product, return (shop, product)."""
shop = self._create_shop_helper()
res = self.testapp.post(
"/c/new",
{
"title": "Analytics Test Content",
"description": "For testing analytics",
"submit": True,
},
)
if res.status_int == 302:
res.follow()
products = get_all_products(self.dbsession).all()
product = products[0]
return shop, product

View file

@ -3185,3 +3185,74 @@ class TestSignalClassifiers(unittest.TestCase):
def test_device_none(self):
self.assertIsNone(classify_device(None))
class TestAnalyticsHelpers(unittest.TestCase):
"""Unit tests for analytics view helper functions."""
def test_fmt_ms_none(self):
from ..views.analytics import _fmt_ms
self.assertEqual(_fmt_ms(None), "\u2014")
def test_fmt_ms_seconds(self):
from ..views.analytics import _fmt_ms
self.assertEqual(_fmt_ms(42000), "42s")
def test_fmt_ms_minutes(self):
from ..views.analytics import _fmt_ms
self.assertEqual(_fmt_ms(125000), "2m 5s")
def test_fmt_ms_hours(self):
from ..views.analytics import _fmt_ms
self.assertEqual(_fmt_ms(3661000), "1h 1m")
def test_fmt_ms_zero(self):
from ..views.analytics import _fmt_ms
self.assertEqual(_fmt_ms(0), "0s")
def test_fmt_pct_none(self):
from ..views.analytics import _fmt_pct
self.assertEqual(_fmt_pct(None), "\u2014")
def test_fmt_pct_half(self):
from ..views.analytics import _fmt_pct
self.assertEqual(_fmt_pct(0.5), "50%")
def test_fmt_pct_full(self):
from ..views.analytics import _fmt_pct
self.assertEqual(_fmt_pct(1.0), "100%")
def test_fmt_pct_zero(self):
from ..views.analytics import _fmt_pct
self.assertEqual(_fmt_pct(0.0), "0%")
def test_fmt_score_none(self):
from ..views.analytics import _fmt_score
self.assertEqual(_fmt_score(None), "\u2014")
def test_fmt_score_value(self):
from ..views.analytics import _fmt_score
self.assertEqual(_fmt_score(3.14159), "3.1")
def test_fmt_score_zero(self):
from ..views.analytics import _fmt_score
self.assertEqual(_fmt_score(0.0), "0.0")
def test_cutoffs_are_7_14_21_days_apart(self):
from ..views.analytics import _cutoffs
c7, c14, c21 = _cutoffs()
day_ms = 24 * 60 * 60 * 1000
self.assertAlmostEqual(c7 - c14, 7 * day_ms, delta=1000)
self.assertAlmostEqual(c14 - c21, 7 * day_ms, delta=1000)
def test_referrer_labels_complete(self):
from ..views.analytics import REFERRER_LABELS
self.assertEqual(len(REFERRER_LABELS), 5)
self.assertIn(0, REFERRER_LABELS)
self.assertIn(4, REFERRER_LABELS)
def test_device_labels_complete(self):
from ..views.analytics import DEVICE_LABELS
self.assertEqual(len(DEVICE_LABELS), 3)
self.assertEqual(DEVICE_LABELS[0], "Mobile")
self.assertEqual(DEVICE_LABELS[2], "Desktop")

View file

@ -0,0 +1,353 @@
from pyramid.view import view_config
from sqlalchemy import func, case, and_, cast, Float
from . import shop_editor_required
from ..models.page_session import PageSession
from ..models.product import Product
from ..models.meta import now_timestamp
REFERRER_LABELS = {0: "Direct", 1: "Search", 2: "Social", 3: "Internal", 4: "Unknown"}
DEVICE_LABELS = {0: "Mobile", 1: "Tablet", 2: "Desktop"}
MIN_SESSIONS = 7
def _cutoffs():
"""Return 7d, 14d, 21d cutoff timestamps in milliseconds."""
now = now_timestamp()
day_ms = 24 * 60 * 60 * 1000
return now - 7 * day_ms, now - 14 * day_ms, now - 21 * day_ms
def _product_titles(dbsession, product_ids):
"""Batch-fetch product titles and URLs for a set of product IDs."""
if not product_ids:
return {}
products = (
dbsession.query(Product)
.filter(Product.id.in_(product_ids))
.all()
)
return {
p.id: {
"title": p.title,
"url": f"/{'p' if p.is_sellable else 'c'}/{p.id}/{p.slug}",
}
for p in products
}
def _fmt_ms(ms):
"""Format milliseconds as human-readable duration."""
if ms is None:
return "\u2014"
s = int(ms / 1000)
if s < 60:
return f"{s}s"
m, s = divmod(s, 60)
if m < 60:
return f"{m}m {s}s"
h, m = divmod(m, 60)
return f"{h}h {m}m"
def _fmt_pct(val):
"""Format a 0-1 float as a percentage string."""
if val is None:
return "\u2014"
return f"{val * 100:.0f}%"
def _fmt_score(val):
"""Format a 0-5 score with one decimal place."""
if val is None:
return "\u2014"
return f"{val:.1f}"
@view_config(route_name="shop_analytics", renderer="analytics.j2")
@shop_editor_required()
def shop_analytics(request):
shop = request.shop
db = request.dbsession
cutoff_7d, cutoff_14d, cutoff_21d = _cutoffs()
def _view_filter(cutoff):
"""Sessions that count as views (7s visible) after cutoff."""
return and_(
PageSession.shop_id == shop.id,
PageSession.visible_ms >= 7000,
PageSession.created_timestamp > cutoff,
)
def _session_filter(cutoff):
"""Sessions with real engagement (7s wall clock) after cutoff."""
return and_(
PageSession.shop_id == shop.id,
PageSession.wall_clock_ms >= 7000,
PageSession.created_timestamp > cutoff,
)
# --- Section 1: Overview Strip (last 7 days) ---
ov = db.query(
func.count().label("total_views"),
func.count(func.distinct(PageSession.product_id)).label("unique_products"),
func.avg(PageSession.wall_clock_ms).label("avg_session_ms"),
).filter(_view_filter(cutoff_7d)).one()
avg_ring_depth = db.query(func.avg(PageSession.ring_position)).filter(
PageSession.shop_id == shop.id,
PageSession.ring_position.isnot(None),
PageSession.created_timestamp > cutoff_7d,
).scalar()
top_src = db.query(
PageSession.referrer_class, func.count().label("cnt"),
).filter(
_view_filter(cutoff_7d),
).group_by(
PageSession.referrer_class,
).order_by(func.count().desc()).first()
dev_rows_7d = db.query(
PageSession.device_class, func.count().label("cnt"),
).filter(_view_filter(cutoff_7d)).group_by(PageSession.device_class).all()
dev_total_7d = sum(r.cnt for r in dev_rows_7d) or 1
dev_split_str = " \u00b7 ".join(
f"{_fmt_pct(r.cnt / dev_total_7d)} {DEVICE_LABELS.get(r.device_class, '?')}"
for r in sorted(dev_rows_7d, key=lambda r: -(r.cnt or 0))
) or "\u2014"
overview = {
"total_views": ov.total_views or 0,
"unique_products": ov.unique_products or 0,
"avg_session": _fmt_ms(ov.avg_session_ms),
"avg_ring_depth": f"{avg_ring_depth:.1f}" if avg_ring_depth else "\u2014",
"top_source": REFERRER_LABELS.get(top_src[0], "\u2014") if top_src else "\u2014",
"device_split": dev_split_str,
}
# --- Section 2: Top Products by Views (7d / 14d / 21d) ---
views_7d_expr = func.sum(case(
(PageSession.created_timestamp > cutoff_7d, 1), else_=0,
))
views_14d_expr = func.sum(case(
(PageSession.created_timestamp > cutoff_14d, 1), else_=0,
))
top_prods_raw = db.query(
PageSession.product_id,
views_7d_expr.label("views_7d"),
views_14d_expr.label("views_14d"),
func.count().label("views_21d"),
).filter(
_view_filter(cutoff_21d),
).group_by(
PageSession.product_id,
).order_by(views_7d_expr.desc()).limit(21).all()
# --- Section 3: Ring Entry Points (top 7 front doors, 21d) ---
ring_raw = db.query(
PageSession.product_id, func.count().label("cnt"),
).filter(
PageSession.shop_id == shop.id,
PageSession.is_ring_entry == True, # noqa: E712
PageSession.created_timestamp > cutoff_21d,
).group_by(PageSession.product_id).order_by(func.count().desc()).limit(7).all()
ring_total = db.query(func.count()).filter(
PageSession.shop_id == shop.id,
PageSession.is_ring_entry == True, # noqa: E712
PageSession.created_timestamp > cutoff_21d,
).scalar() or 1
# --- Section 4: Engagement Leaders (active_ms / wall_clock_ms) ---
eng_expr = func.avg(
cast(PageSession.active_ms, Float) / func.nullif(PageSession.wall_clock_ms, 0)
)
eng_raw = db.query(
PageSession.product_id, eng_expr.label("score"), func.count().label("sessions"),
).filter(
_session_filter(cutoff_21d),
).group_by(
PageSession.product_id,
).having(func.count() >= MIN_SESSIONS).order_by(eng_expr.desc()).limit(7).all()
# --- Section 5: Attention Holders (visible_ms / wall_clock_ms) ---
att_expr = func.avg(
cast(PageSession.visible_ms, Float) / func.nullif(PageSession.wall_clock_ms, 0)
)
att_raw = db.query(
PageSession.product_id, att_expr.label("score"), func.count().label("sessions"),
).filter(
_session_filter(cutoff_21d),
).group_by(
PageSession.product_id,
).having(func.count() >= MIN_SESSIONS).order_by(att_expr.desc()).limit(7).all()
# --- Section 6: Study Material (learning score: rewind, slow, re-read) ---
learn_expr = (
case((PageSession.media_seek_back_count > 0, 1), else_=0)
+ case(
(and_(PageSession.media_speed.isnot(None), PageSession.media_speed < 1.0), 1),
else_=0,
)
+ case((PageSession.scroll_direction_changes > 3, 1), else_=0)
+ case((PageSession.media_pause_count > 2, 1), else_=0)
+ case(
(
and_(
PageSession.wall_clock_ms > 0,
cast(PageSession.active_ms, Float) / PageSession.wall_clock_ms > 0.7,
),
1,
),
else_=0,
)
)
learn_avg = func.avg(learn_expr)
learn_raw = db.query(
PageSession.product_id, learn_avg.label("score"), func.count().label("sessions"),
).filter(
_session_filter(cutoff_21d),
).group_by(
PageSession.product_id,
).having(func.count() >= MIN_SESSIONS).order_by(learn_avg.desc()).limit(7).all()
# --- Section 7: Background Favorites (passive score: lean-back plays) ---
pass_expr = (
case(
(
and_(
PageSession.wall_clock_ms > 0,
cast(PageSession.visible_ms, Float) / PageSession.wall_clock_ms > 0.7,
),
1,
),
else_=0,
)
+ case(
(
and_(
PageSession.wall_clock_ms > 0,
cast(PageSession.active_ms, Float) / PageSession.wall_clock_ms < 0.3,
),
1,
),
else_=0,
)
+ case((PageSession.media_percent_played > 0.69, 1), else_=0)
+ case((PageSession.media_play_count == 1, 1), else_=0)
+ case((PageSession.media_pause_count == 0, 1), else_=0)
)
pass_avg = func.avg(pass_expr)
pass_raw = db.query(
PageSession.product_id, pass_avg.label("score"), func.count().label("sessions"),
).filter(
_session_filter(cutoff_21d),
).group_by(
PageSession.product_id,
).having(func.count() >= MIN_SESSIONS).order_by(pass_avg.desc()).limit(7).all()
# --- Section 8: Traffic Sources (21d) ---
traffic_raw = db.query(
PageSession.referrer_class, func.count().label("cnt"),
).filter(
_view_filter(cutoff_21d),
).group_by(PageSession.referrer_class).order_by(func.count().desc()).all()
traffic_total = sum(r.cnt for r in traffic_raw) or 1
# --- Section 9: Device Split (21d) ---
device_raw = db.query(
PageSession.device_class, func.count().label("cnt"),
).filter(
_view_filter(cutoff_21d),
).group_by(PageSession.device_class).order_by(func.count().desc()).all()
device_total = sum(r.cnt for r in device_raw) or 1
# --- Resolve all product titles in one batch ---
all_pids = set()
for rows in (top_prods_raw, ring_raw, eng_raw, att_raw, learn_raw, pass_raw):
for r in rows:
all_pids.add(r.product_id)
titles = _product_titles(db, list(all_pids))
def _t(pid):
return titles.get(pid, {}).get("title", "Unknown")
def _u(pid):
return titles.get(pid, {}).get("url", "#")
# --- Build template-ready data ---
top_products = []
for i, r in enumerate(top_prods_raw):
v7, v14 = r.views_7d or 0, r.views_14d or 0
top_products.append({
"rank": i + 1,
"title": _t(r.product_id),
"url": _u(r.product_id),
"views_7d": v7,
"views_14d": v14,
"views_21d": r.views_21d or 0,
"trend": "rising" if v14 > 0 and v7 > v14 / 2 else "falling",
})
ring_entries = [
{
"rank": i + 1,
"title": _t(r.product_id),
"url": _u(r.product_id),
"count": r.cnt,
"pct": _fmt_pct(r.cnt / ring_total),
}
for i, r in enumerate(ring_raw)
]
def _ranked(raw, score_fmt):
return [
{
"rank": i + 1,
"title": _t(r.product_id),
"url": _u(r.product_id),
"score": score_fmt(r.score),
"sessions": r.sessions,
}
for i, r in enumerate(raw)
]
return {
"overview": overview,
"top_products": top_products,
"ring_entries": ring_entries,
"engagement": _ranked(eng_raw, _fmt_pct),
"attention": _ranked(att_raw, _fmt_pct),
"learning": _ranked(learn_raw, _fmt_score),
"passive": _ranked(pass_raw, _fmt_score),
"traffic": [
{
"source": REFERRER_LABELS.get(r.referrer_class, "Unknown"),
"count": r.cnt,
"pct": _fmt_pct(r.cnt / traffic_total),
}
for r in traffic_raw
],
"devices": [
{
"device": DEVICE_LABELS.get(r.device_class, "Unknown"),
"count": r.cnt,
"pct": _fmt_pct(r.cnt / device_total),
}
for r in device_raw
],
}

View file

@ -359,6 +359,7 @@ def product_edit(request):
reforge_discovery_ring_async(
product.shop.id, request.registry["dbsession_factory"]
)
return HTTPFound(product.absolute_edit_url(request))
signed_posts = {}