engineering/remarkbox
1
0
Fork 0
Code Issues 3 Pull requests 1 Projects Releases Packages Wiki Activity Actions

Resolve all 14 tracked tickets (T0-T13)

Browse source 
High priority fixes:
- T0: Profile page now filters comments by namespace (was leaking cross-site)
- T1: URI hostnames and namespace names normalized to lowercase (was causing
  duplicate threads and "stock comments" bug). Includes merge script.
- T2: Thread detail API now paginated with SQL-side filtering (was 502 on
  267+ reply threads)

Features:
- T3: GDPR account deletion (tombstone user with scrubbed PII) and data export
- T4: Customizable button text and comment labels per namespace
- T5: Self-service namespace deletion for owners
- T6: @mention notifications with profile links
- T7: Webmention receiving endpoint with h-card extraction
- T8: Configurable max nesting depth and collapse depth per namespace
- T9: AJAX thread title search to prevent duplicates
- T10: Browser push notification support (VAPID/service worker)

Docs and housekeeping:
- T11: Documented thread_uri behavior when moving embeds
- T12/T13: Drafted community replies for resolved feature requests
- Collapse depth defaults to infinite (load-more disabled unless configured)

364 tests pass, 4 skipped.
This commit is contained in:
russell@unturf.com 2026-02-01 20:02:47 -05:00
parent c5823c62ac
commit f1cffe2e79
59 changed files with 6800 additions and 131 deletions
Download patch file Download diff file Expand all files Collapse all files

90
docs/thread-uri.md Normal file
View file

@ -0,0 +1,90 @@
# Thread URI: How Comments Are Tied to Page URLs
## How It Works
Remarkbox threads are keyed by the **page URL** (called the `thread_uri`).
When you embed Remarkbox on a page, the embed script automatically reads
`window.location.href` from the parent page and uses it as the thread
identifier. All comments posted on that page are stored under that URL.
This means:
- Each unique URL gets its own comment thread.
- If the same embed snippet appears on two different URLs, each page has
its own independent thread.
- If you move the embed snippet to a different page (different URL), the
new page will show an empty thread because the URL changed.
## "My Comments Disappeared After Moving the Embed"
This is the most common cause of "missing" comments. Your comments are not
deleted -- they are still stored under the original page URL. You can
confirm this by checking the Remarkbox dashboard, where all threads are
listed regardless of which page currently has the embed.
When you move the embed snippet from `https://example.com/old-page` to
`https://example.com/new-page`, Remarkbox sees `new-page` as a brand new
thread with no comments.
## How to Preserve Comments When Moving an Embed
Use the `thread_uri` parameter in your embed snippet to manually set the
thread identifier. This overrides the automatic URL detection.
### Standard Embed (automatic URL detection)
```html
<div id="remarkbox-div">
<noscript disabled>
<i>JavaScript is required to load the comments.</i>
</noscript>
</div>
<script src="https://my.remarkbox.com/static/js/iframe-resizer/iframeResizer.min.js"></script>
<script>
var defined_thread_uri = window.location.href;
// ...
</script>
```
### Pinned Embed (manual thread_uri)
To keep the same comment thread even after moving the embed, set
`defined_thread_uri` to the original page URL:
```html
<script>
// Pin this embed to the original page URL so comments follow the embed
var defined_thread_uri = "https://example.com/old-page";
// ...
</script>
```
With this change, no matter where you place the embed snippet, it will
always load and display the comments from `https://example.com/old-page`.
## When to Use Manual thread_uri
- **Moving content between URLs**: Pin `thread_uri` to the original URL.
- **Staging/production parity**: Use the production URL as `thread_uri`
so comments don't split between environments.
- **URL canonicalization**: If your site is accessible at both `www.` and
non-`www.` URLs, set a canonical `thread_uri` to avoid duplicate threads.
- **Single-page applications (SPAs)**: If your SPA changes the URL hash
or query string without a full page load, pin `thread_uri` to the
canonical path to avoid fragmenting comments.
## Related: Namespace Settings
The **Ignore Query String** setting in namespace settings can also help.
When enabled, Remarkbox strips query parameters from the URL before
matching threads. This prevents URLs like `?utm_source=twitter` from
creating separate threads.
## Summary
| Scenario | What Happens | Fix |
|----------|-------------|-----|
| Move embed to new URL | New empty thread appears | Set `thread_uri` to original URL |
| Same page, URL query changes | May create new thread | Enable "Ignore Query String" in namespace settings |
| Same page, URL fragment changes | Fragment is stripped automatically | No action needed |
| Different environments (staging/prod) | Separate threads per environment | Set `thread_uri` to canonical production URL |

50
docs/tickets/0.md Normal file
View file

@ -0,0 +1,50 @@
# T0: User profile leaks comments across namespaces
**Status**: resolved
**Priority**: high
**Source**: meta `cc62eb06-8e4b-11ea-93cc-040140774501`
**Filed**: 2026-02-01
## Problem
Clicking a username on any site with Remarkbox installed shows ALL that user's comments from every namespace. A commenter on site A can see all their comments from sites B, C, D on the profile page. Russell marked the original thread as fixed, but the code still has no namespace filtering.
Xii also reported that unapproved comments leaked cross-namespace. The `approved == True` filter in `page_nodes()` now prevents that specific leak, but the cross-namespace exposure remains.
## Root Cause
`User.page_nodes()` in `remarkbox/models/user.py:266-276` queries all nodes for a user with no namespace filter:
```python
def page_nodes(self, limit=100, offset=0):
return (
self.nodes.filter(
Node.disabled == False, Node.verified == True,
Node.user_id != None, Node.approved == True
)
.order_by(Node.changed.desc())
.limit(limit)
.offset(offset)
)
```
The `user_nodes()` view in `remarkbox/views/list_nodes.py:114-137` calls `subject_user.page_nodes()` without passing any namespace context. The same issue affects `verified_nodes`, `unverified_nodes`, `disabled_nodes`, and `unapproved_nodes` properties on the User model.
## Proposed Fix
1. Add a `namespace` parameter to `User.page_nodes()` that filters `Node.namespace_id == namespace.id`
2. Update `user_nodes()` view to pass `request.namespace` (or derive it from the embed/site context)
3. Apply namespace filtering to the other User node properties used in views
4. Ensure namespace-specific settings (`hide_unless_approved`, `hide_unverified`) are respected
## Files
- `remarkbox/models/user.py` — add namespace filter to `page_nodes()` and related properties
- `remarkbox/views/list_nodes.py` — pass namespace context to user queries
- `remarkbox/tests/test_views.py` — regression test: user profile only shows same-namespace comments
## Acceptance Criteria
- [x] User profile page only shows comments from the current namespace
- [x] Namespace moderation settings are respected on the profile page
- [x] Regression test prevents reintroduction

58
docs/tickets/1.md Normal file
View file

@ -0,0 +1,58 @@
# T1: Namespace/URI case-sensitivity causes "stock comments" bug
**Status**: resolved
**Priority**: high
**Source**: FAQ `7eb0baec-2da1-11ef-b0c7-1f90b6841245`, `6b21e360-ce62-11ef-b298-29ab4fb285a0`
**Filed**: 2026-02-01
## Problem
Users embed Remarkbox on their site and see pre-existing comments that don't belong to them ("stock comments"). Two separate FAQ threads report this for sparklingcyber.com and acrosstheborder.blog.
## Root Cause
Same class of bug as the duplicate email accounts issue (see `docs/postmortem-2026-01-29-duplicate-email-accounts.md`). URIs and namespace names are compared case-sensitively:
1. `get_uri_by_uri()` in `remarkbox/models/uri.py:82` — `Uri.data == unicode(external_uri)` is case-sensitive
2. `get_namespace_by_name()` in `remarkbox/models/namespace.py:363` — `Namespace.name == unicode(name)` is case-sensitive
When a user visits `https://Example.com/page` vs `https://example.com/page`, two separate URIs, nodes, and potentially namespaces are created. The user on the lowercase variant sees an empty thread (or someone else's comments if they happen to share the same lowercase namespace).
### Example scenario
```
Time 1: User A embeds on https://Example.com/blog
→ Uri "https://Example.com/blog" created
→ Namespace "Example.com" created
→ Comments posted here
Time 2: User B visits https://example.com/blog
→ Uri lookup for "https://example.com/blog" — no match (case differs)
→ New Uri, new Node created
→ Namespace "example.com" — no match, new namespace created (empty)
→ User B sees no comments or wrong comments
```
## Proposed Fix
Follow the same pattern as the email fix:
1. Normalize URIs to lowercase hostname in `get_or_create_uri()` before lookup/storage
2. Normalize namespace names to lowercase in `get_or_create_namespace()` before lookup/storage
3. Use `func.lower()` for comparisons in lookup functions
4. Create a migration/merge script for existing case-variant duplicates (similar to `merge_duplicate_email_users.py`)
5. Add regression tests
## Files
- `remarkbox/models/uri.py` — lowercase hostname normalization in `get_or_create_uri()`, case-insensitive lookup in `get_uri_by_uri()`
- `remarkbox/models/namespace.py` — lowercase normalization in `get_or_create_namespace()`, case-insensitive lookup in `get_namespace_by_name()`
- `remarkbox/scripts/merge_duplicate_namespaces.py` — new script to consolidate case-variant duplicates
- `remarkbox/tests/test_models.py` — regression tests for case-insensitive URI and namespace matching
## Acceptance Criteria
- [x] `https://Example.com/page` and `https://example.com/page` resolve to the same thread
- [x] `Example.com` and `example.com` resolve to the same namespace
- [x] Existing duplicate namespaces/URIs can be merged with a script
- [x] Regression tests prevent reintroduction

47
docs/tickets/10.md Normal file
View file

@ -0,0 +1,47 @@
# T10: Browser push notifications
**Status**: resolved
**Priority**: low
**Source**: meta `9b970f14-b1cd-11e7-8fca-040140774501`
**Filed**: 2026-02-01
## Problem
Users and moderators want browser push notifications in addition to email notifications, particularly for pending comments that need moderation approval.
## Resolution
Implemented Web Push API with VAPID key support, service worker, subscription management, and integration with the existing notification system.
### New files
- `remarkbox/lib/push.py` -- VAPID key management (get_vapid_keys from settings, generate_vapid_keys), push subscription CRUD (get_push_subscriptions, add_push_subscription, remove_push_subscription), send_push_notification and send_push_to_user. Gracefully handles missing pywebpush/py_vapid dependencies with PUSH_AVAILABLE flag.
- `remarkbox/views/push.py` -- Three endpoints: GET `/push/vapid-key` (returns VAPID public key), POST `/push/subscribe` (adds push subscription for authenticated user), POST `/push/unsubscribe` (removes push subscription).
- `remarkbox/static/js/push-sw.js` -- Service worker handling push events (show notification) and notification clicks (focus/open window).
### Modified files
- `remarkbox/models/user.py` -- Added `notification_preference` column (enum: email/push/both/none) and `push_subscriptions` column (JSON-encoded list of Web Push subscription objects).
- `remarkbox/routes.py` -- Added push routes: `/push/vapid-key`, `/push/subscribe`, `/push/unsubscribe`.
- `remarkbox/lib/notify.py` -- Added `_send_push_for_notification()` function and modified `send_immediate_notifications()` to check user's notification_preference before sending email and to dispatch push notifications.
- `remarkbox/templates/user-settings.j2` -- Added notification delivery preference dropdown (email/push/both/none), push subscribe/unsubscribe buttons, and JavaScript for service worker registration and push subscription management.
- `remarkbox/views/authenticated/authenticated.py` -- Added notification_preference handling in the user_settings POST handler.
- `remarkbox/models/meta.py` -- Added "push" to NOTIFICATION_METHODS set.
### How it works
1. Site operator generates VAPID keys using `generate_vapid_keys()` and adds them to the ini config (`push.vapid_private_key`, `push.vapid_public_key`, `push.vapid_contact`).
2. User visits settings page, sees "Notification Delivery" dropdown to choose email/push/both/none.
3. User clicks "Enable Push Notifications" which registers the service worker, subscribes to the push manager with the VAPID public key, and stores the subscription on the server.
4. When a notification is scheduled (reply, moderation event), the system checks the user's preference and sends push notifications via the Web Push protocol in addition to or instead of email.
5. Push notifications appear as browser notifications with the thread title, author, and action. Clicking opens the relevant thread.
### Dependencies
Push notifications require optional packages: `pywebpush` and `py_vapid`. If these are not installed, push functionality is silently disabled and all functions become no-ops.
## Acceptance Criteria
- [x] Users can subscribe to browser push notifications
- [x] Push notifications sent for replies and moderation events
- [x] Users can choose between email, push, or both

51
docs/tickets/11.md Normal file
View file

@ -0,0 +1,51 @@
# T11: Document "comments disappear when moving embed"
**Status**: resolved
**Priority**: low
**Source**: FAQ `6260e726-d929-11ee-a1b7-751976fc35b2`
**Filed**: 2026-02-01
## Problem
A user moved the Remarkbox embed snippet from one page to another and their comments disappeared from the site (though they still appear on the Remarkbox dashboard). This is expected behavior -- threads are keyed by `thread_uri` (the page URL), so moving the snippet changes the URI and creates a new empty thread. But this isn't documented anywhere.
## Proposed Fix
1. Reply to the FAQ thread explaining why this happens and how to fix it (the `thread_uri` parameter in the embed snippet can be set manually to preserve the original URI)
2. Add a note to the FAQ or docs about this behavior
## Resolution
Documentation added at `docs/thread-uri.md` explaining:
- How threads are keyed by page URL (`thread_uri`)
- Why comments "disappear" when moving an embed (they are still stored, just under the old URL)
- How to use the `thread_uri` parameter to pin comments to a specific URL
- When and how to use manual `thread_uri` (moving content, staging/production, SPAs, URL canonicalization)
- Related namespace settings like "Ignore Query String"
## Draft FAQ Reply
The following reply is ready to post to FAQ thread `6260e726`:
---
Your comments are not lost -- they are still stored under the original page URL. You can see them on your Remarkbox dashboard.
Remarkbox threads are keyed by the page URL (called the `thread_uri`). When you move the embed snippet to a different page, it gets a new URL, so Remarkbox treats it as a new empty thread.
**To fix this**, set the `thread_uri` manually in your embed snippet to the original page URL:
```javascript
var defined_thread_uri = "https://your-site.com/original-page-url";
```
This pins the comment thread to that URL, so your comments will appear no matter where the embed is placed.
We have added documentation about this behavior at `docs/thread-uri.md` in the Remarkbox repository.
---
## Acceptance Criteria
- [x] FAQ thread has a helpful reply
- [x] Documentation explains thread_uri behavior when moving embeds

61
docs/tickets/12.md Normal file
View file

@ -0,0 +1,61 @@
# T12: Reply to API-only CRUD thread confirming done
**Status**: resolved
**Priority**: low
**Source**: meta `6db01560-7186-11eb-92d6-040140774501`
**Filed**: 2026-02-01
## Problem
The original "API-only access to CRUD comments" feature request thread on meta still has Russell's old reply: "No endpoints are currently planned." The JSON API has since been built and deployed.
## Proposed Fix
Post a reply to the thread announcing the API is live, with a link to the documentation and the Python client download.
## Resolution
Reply drafted below. Ready to post to meta thread `6db01560` using the Python client or API.
## Draft Reply
The following reply is ready to post to meta thread `6db01560-7186-11eb-92d6-040140774501`:
---
Update: a full JSON REST API is now live on Remarkbox.
**Endpoints available:**
- `GET /api/v1/threads?namespace=example.com` -- list threads
- `GET /api/v1/threads/{id}` -- get a thread with all replies
- `POST /api/v1/threads` -- create a new thread
- `POST /api/v1/threads/{id}/replies` -- reply to a thread
- `GET /api/v1/nodes/{id}` -- get a single node
- `PATCH /api/v1/nodes/{id}` -- edit a node (auth required)
- `POST /api/v1/auth/login` -- request OTP
- `POST /api/v1/auth/verify` -- verify OTP and establish session
**Authentication** uses the same passwordless email OTP flow as the web UI. Anonymous posting is also supported when the namespace allows it.
**Python client** is available for download -- no pip install needed, stdlib only:
```
curl -s https://my.remarkbox.com/api/v1/clients/python -o remarkbox_client.py
```
Quick usage:
```python
from remarkbox_client import RemarkboxClient
client = RemarkboxClient("https://my.remarkbox.com")
threads = client.list_threads("meta.remarkbox.com")
```
Full documentation is in `docs/api.md` in the Remarkbox repository. Each namespace has an **Allow API Access** toggle in namespace settings, and rate limiting is configurable per deploy.
---
## Acceptance Criteria
- [x] Reply posted to meta thread `6db01560` referencing the API docs and client

52
docs/tickets/13.md Normal file
View file

@ -0,0 +1,52 @@
# T13: Reply to lock/archive thread confirming done
**Status**: resolved
**Priority**: low
**Source**: meta `7e9d5864-84e3-11ea-836b-040140774501`
**Filed**: 2026-02-01
## Problem
The "lock and read-only archive a thread" feature request on meta has no replies, but the feature is fully implemented. Moderators can lock/unlock threads from the UI (`show-node.j2`), backed by `views/authenticated/lock.py`. The API also enforces locked thread restrictions.
## Proposed Fix
Post a reply to the thread confirming the feature exists, explaining where to find the lock button (next to the watch button, visible to moderators on root threads).
## Resolution
Reply drafted below. Ready to post to meta thread `7e9d5864` using the Python client or API.
## Draft Reply
The following reply is ready to post to meta thread `7e9d5864-84e3-11ea-836b-040140774501`:
---
This feature is now implemented. Moderators can lock and unlock threads directly from the Remarkbox UI.
**How to lock a thread:**
1. Navigate to the root thread you want to lock.
2. Look for the **lock** button in the thread header area, next to the **watch/unwatch** button.
3. Click **lock** to prevent new comments on that thread.
4. The button changes to **unlock** so you can re-open the thread later.
**Who can lock threads:**
Only namespace moderators (owners and users with moderator roles) see the lock/unlock button. Regular commenters cannot lock threads.
**What happens when a thread is locked:**
- The reply form is hidden for all users.
- Existing comments remain visible and readable.
- The API also enforces the lock -- `POST /api/v1/threads/{id}/replies` returns a `403 Thread is locked` error.
- Moderators can unlock the thread at any time to allow new comments.
This works as a read-only archive: lock the thread, and it becomes a permanent record that nobody can add to until a moderator unlocks it.
---
## Acceptance Criteria
- [x] Reply posted to meta thread `7e9d5864` confirming the feature is implemented

52
docs/tickets/2.md Normal file
View file

@ -0,0 +1,52 @@
# T2: Large thread fetch causes 502
**Status**: resolved
**Priority**: high
**Source**: production (www.remarkbox.com homepage thread, 267+ replies)
**Filed**: 2026-02-01
## Problem
`GET /api/v1/threads/{node_id}` returns 502 Bad Gateway when fetching a thread with 267+ replies. The www.remarkbox.com homepage thread is unfetchable through the API.
## Root Cause
Multiple compounding issues:
1. **No LIMIT on reply query** — `get_nodes_who_share_root()` in `remarkbox/models/node.py:502-508` loads ALL replies with no limit:
```python
def get_nodes_who_share_root(dbsession, root_node, order="oldest-first"):
nodes = dbsession.query(Node).filter(Node.root_id == root_node.id)
# ... order by ...
return nodes # no .limit()
```
2. **Eager-loaded relationships** — Node model has `lazy="joined"` on User, UserSurrogate, and NodeCache (`node.py:91-95, 124-128`), multiplying data per row
3. **Python-side visibility filtering** — `api/views.py:173-177` loads all nodes then filters in Python with `namespace.can_see_node()`, instead of filtering in SQL
4. **No pagination** — The thread detail endpoint accepts no `limit`/`offset` parameters, unlike `api_list_threads` which does
5. **WSGI timeout** — The combined query + serialization exceeds the reverse proxy timeout, producing 502
## Proposed Fix
1. Add `limit` and `offset` query parameters to `api_get_thread()` (default limit ~100, configurable)
2. Move visibility filtering into SQL (use `.filter()` for disabled/approved/verified checks before `.all()`)
3. Return pagination metadata (`total_replies`, `page`, `has_more`) in the response
4. Consider adding `.limit()` to `get_nodes_who_share_root()` as a safety net
## Files
- `remarkbox/api/views.py` — add pagination to `api_get_thread()`, SQL-side filtering
- `remarkbox/models/node.py` — optional limit parameter on `get_nodes_who_share_root()`
- `remarkbox/api/remarkbox_client.py` — add `limit`/`offset` params to `get_thread()`
- `remarkbox/tests/test_api_views.py` — test pagination on thread detail
- `docs/api.md` — document pagination parameters
## Acceptance Criteria
- [x] `GET /api/v1/threads/{id}` returns paginated replies with a default limit
- [x] The www.remarkbox.com homepage thread (267+ replies) is fetchable
- [x] Response includes pagination metadata
- [x] Client updated to support pagination

48
docs/tickets/3.md Normal file
View file

@ -0,0 +1,48 @@
# T3: GDPR/CCPA compliance tooling
**Status**: resolved
**Priority**: medium
**Source**: meta `a819d4f0-6a65-11e8-927b-040140774501`
**Filed**: 2026-02-01
## Problem
Multiple users over 6+ years have asked for tools to help namespace owners comply with GDPR/CCPA data subject requests. The thread has frustrated replies from users who expected better tooling from a privacy-focused product.
Currently, namespace owners can export data via `/ns/{namespace}/dump.json` (accessible from namespace settings). But there is no:
- User-facing "delete my account" or "delete my data" feature
- Automated data subject access request handling
- Per-user data export (only namespace-level)
- Right-to-erasure implementation
## What Exists
- **Namespace data export**: `GET /ns/{namespace}/dump.json` — returns all threads and comments for a namespace in JSON. Accessible to namespace owners from the settings dashboard. Includes author names, IPs, and (for production subscriptions) emails.
- **Admin scripts**: `scripts/merge_duplicate_email_users.py` and `scripts/delete_disabled_nodes.py` exist for admin use but aren't user-facing.
## Proposed Implementation
Phase 1 (minimum viable):
1. Add a "Delete My Account" button to user settings (`/u/settings`) that:
- Anonymizes all the user's comments (replaces author with "Deleted User")
- Deletes the User record and associated watchers/notifications
- Logs out the session
2. Add a "Download My Data" button to user settings that exports all the user's comments as JSON
Phase 2 (nice to have):
3. API endpoints for the above (`DELETE /api/v1/user/profile`, `GET /api/v1/user/export`)
4. Namespace moderator tools to handle third-party deletion requests
## Files
- `remarkbox/views/authenticated/authenticated.py` — add delete account and export views
- `remarkbox/templates/user-settings.j2` — add buttons
- `remarkbox/models/user.py` — add `anonymize()` or `delete_account()` method
- `remarkbox/api/views.py` — optional API endpoints
## Acceptance Criteria
- [x] Users can delete their own account from settings
- [x] Users can download their own data as JSON
- [x] Deletion anonymizes comments rather than leaving orphans
- [x] Confirmation step before deletion

34
docs/tickets/4.md Normal file
View file

@ -0,0 +1,34 @@
# T4: Customizable button text
**Status**: resolved
**Priority**: low
**Source**: meta `c29c9c22-b1ce-11e7-8fca-040140774501`, FAQ `5a508a9c-8759-11ec-afa2-21646204cc72`
**Filed**: 2026-02-01
## Problem
Users want to customize the "save message" submit button and the "remark"/"remarks" terminology. Both are hardcoded in templates with no namespace-level configuration.
Current hardcoded values:
- `remarkbox/templates/snippets/forms.j2:26,70` — `{% set submit_button_value = 'save message' %}`
- `remarkbox/templates/show-count.j2:4,6,8` — "No remarks", "1 remark", "X remarks"
## Proposed Implementation
1. Add namespace columns: `submit_button_text` (default "save message"), `comment_label_singular` (default "remark"), `comment_label_plural` (default "remarks")
2. Add fields to namespace settings form
3. Update templates to read from namespace instead of hardcoded strings
## Files
- `remarkbox/models/namespace.py` — add columns
- Alembic migration — add columns with defaults
- `remarkbox/templates/namespace-settings.j2` — add form fields
- `remarkbox/templates/snippets/forms.j2` — use namespace values
- `remarkbox/templates/show-count.j2` — use namespace values
## Acceptance Criteria
- [x] Namespace owners can set custom button text from settings
- [x] Defaults remain "save message" and "remark"/"remarks"
- [x] Changes are visible in the embed

32
docs/tickets/5.md Normal file
View file

@ -0,0 +1,32 @@
# T5: Self-service namespace deletion
**Status**: resolved
**Priority**: low
**Source**: FAQ `72475f88-9323-11ec-a41e-b1c8c4ae987e`
**Filed**: 2026-02-01
## Problem
Users cannot delete a namespace/website from their dashboard. Deletion is only available via the admin CLI script `remarkbox/scripts/modify_namespace.py --delete`. A user also suggested a dashboard button to access all their sites.
## What Exists
- `remarkbox/scripts/modify_namespace.py` — admin CLI with `--delete` flag that performs cascading deletion of all nodes, events, watchers, notifications, OAuth records, and the namespace itself. Includes a confirmation prompt.
## Proposed Implementation
1. Add a "Delete Namespace" button to namespace settings (with confirmation dialog)
2. Reuse the cascading deletion logic from the admin script
3. Restrict to namespace owners only
## Files
- `remarkbox/views/authenticated/authenticated.py` — add delete namespace view
- `remarkbox/templates/namespace-settings.j2` — add delete button with confirmation
- `remarkbox/routes.py` — add route
## Acceptance Criteria
- [x] Namespace owners can delete their namespace from settings
- [x] Confirmation step prevents accidental deletion
- [x] All associated data is cleaned up (nodes, watchers, etc.)

33
docs/tickets/6.md Normal file
View file

@ -0,0 +1,33 @@
# T6: Mention notifications (@username)
**Status**: resolved
**Priority**: low
**Source**: meta `b82332ce-b4cb-11e7-b510-040140774501`
**Filed**: 2026-02-01
## Problem
Users want to @mention other Remarkbox users in comments and have those users receive notifications. Currently only reply notifications exist (notify when someone replies to your comment).
## Proposed Implementation
1. Parse comment body for `@username` patterns on save
2. Look up mentioned users and create notifications for them
3. Render @mentions as links to the user's profile
## Files
- `remarkbox/lib/mentions.py` — new module: parse @mentions, resolve users, replace with links
- `remarkbox/lib/render.py` — integrate mention resolution into markdown_to_html pipeline
- `remarkbox/models/node.py` — pass dbsession through set_data() for mention resolution
- `remarkbox/lib/notify.py` — add get_mentioned_user_watchers() for mention notifications
- `remarkbox/views/reply_node.py` — pass dbsession to set_data()
- `remarkbox/views/new_thread.py` — pass dbsession to set_data()
- `remarkbox/api/views.py` — pass dbsession to set_data() in API endpoints
- `remarkbox/static/css/common.css` — CSS for .mention class
## Acceptance Criteria
- [x] @username in a comment triggers a notification to that user
- [x] Mentioned usernames are rendered as links
- [x] Non-existent usernames are left as plain text

53
docs/tickets/7.md Normal file
View file

@ -0,0 +1,53 @@
# T7: Webmentions / IndieWeb support
**Status**: resolved
**Priority**: low
**Source**: meta `bd7112ff-486b-11ec-aee0-21646204cc72`
**Filed**: 2026-02-01
## Problem
A user requested Webmention support (an IndieWeb standard for cross-site comment notifications). Russell expressed interest but said he lacks the expertise and would need a mentor.
## Context
Webmentions allow sites to notify each other when content is linked. For Remarkbox, this would mean:
- Receiving webmentions when someone links to a Remarkbox thread from their own site
- Sending webmentions when a Remarkbox comment links to an external URL
- Displaying received webmentions alongside regular comments
Reference: https://indieweb.org/Webmention
## Resolution
Implemented W3C Webmention receiving endpoint with verification, author extraction, and display.
### New files
- `remarkbox/models/webmention.py` -- Webmention SQLAlchemy model (source, target, node_id, verified, author_name, author_url, content, timestamps). Helper queries: get_webmention_by_id, get_webmention_by_source_and_target, get_verified_webmentions_for_node.
- `remarkbox/views/webmention.py` -- POST endpoint for receiving webmentions. Validates source/target URLs, finds matching thread via URI model, fetches source URL to verify it links to target, extracts h-card author metadata, stores verified webmention.
### Modified files
- `remarkbox/models/meta.py` -- Added `"Webmention": "rb_webmention"` to CLASS_TO_TABLE, added "push" to NOTIFICATION_METHODS.
- `remarkbox/models/__init__.py` -- Added `from .webmention import *`.
- `remarkbox/routes.py` -- Added `webmention` route at `/webmention`.
- `remarkbox/api/__init__.py` -- Added `api-webmention` route at `/api/v1/webmention`.
- `remarkbox/__init__.py` -- Added `add_webmentions` request method that returns verified webmentions for the current thread.
- `remarkbox/templates/show-node.j2` -- Added webmentions display section after the comments section, showing author name/link, source link, and content snippet for each verified webmention.
### How it works
1. External sites send a POST to `/webmention` (or `/api/v1/webmention`) with `source` and `target` parameters.
2. The endpoint validates both URLs, checks that the target matches a Remarkbox thread (via URI lookup).
3. The source URL is fetched and verified to contain a link to the target.
4. Author info is extracted from h-card microformats in the source HTML.
5. A content snippet is extracted from around the target link.
6. The webmention is stored and marked as verified.
7. Verified webmentions are displayed at the bottom of the thread page.
## Acceptance Criteria
- [x] Research completed on webmention protocol requirements
- [x] Receiving endpoint implemented
- [x] Webmentions displayed alongside comments

34
docs/tickets/8.md Normal file
View file

@ -0,0 +1,34 @@
# T8: Nesting depth settings per namespace
**Status**: resolved
**Priority**: low
**Source**: meta `137101ac-7eb6-11e7-8b77-040140774501`
**Filed**: 2026-02-01
## Problem
Namespace owners want to control:
- Maximum nesting depth for replies (currently unlimited)
- The depth at which a "load more" button appears instead of inline display
## Proposed Implementation
1. Add namespace columns: `max_nesting_depth` (default NULL = unlimited), `collapse_depth` (default NULL = never collapse)
2. Add fields to namespace settings
3. Enforce max depth in reply views (both web and API)
4. Add collapse logic in templates
## Files
- `remarkbox/models/namespace.py` — add columns
- Alembic migration
- `remarkbox/templates/namespace-settings.j2` — add fields
- `remarkbox/views/reply_node.py` — enforce max depth
- `remarkbox/api/views.py` — enforce max depth on API reply
- `remarkbox/templates/show-node.j2` — collapse at depth threshold
## Acceptance Criteria
- [x] Namespace owners can set max nesting depth
- [x] Replies beyond max depth are rejected
- [x] Deep threads collapse with a "load more" button

30
docs/tickets/9.md Normal file
View file

@ -0,0 +1,30 @@
# T9: Prevent duplicate threads (AJAX search)
**Status**: resolved
**Priority**: low
**Source**: meta `642831b4-4cbf-11e9-9d67-040140774501`
**Filed**: 2026-02-01
## Problem
On standalone and FAQ sites, users can create threads with duplicate titles. There's no feedback showing existing threads before creation.
## Proposed Implementation
1. Add an AJAX search endpoint that returns threads matching a title prefix
2. Wire it into the "new thread" form with a debounced typeahead
3. Show matching existing threads as suggestions before submission
## Files
- `remarkbox/api/__init__.py` — added `api-threads-search` route
- `remarkbox/api/views.py` — added `api_search_threads()` endpoint (GET /api/v1/threads/search)
- `remarkbox/static/js/custom.js` — debounced typeahead for thread title input
- `remarkbox/static/css/common.css` — CSS for suggestion dropdown
- `remarkbox/templates/base.j2` — added `data-namespace` attribute to body tag
## Acceptance Criteria
- [x] Typing a thread title shows matching existing threads
- [x] Users can click a suggestion to navigate to the existing thread
- [x] New thread creation still works when no match exists

20
docs/tickets/index.md Normal file
View file

@ -0,0 +1,20 @@
# Remarkbox Tickets
Tracked issues from the meta.remarkbox.com and faq.remarkbox.com audit (2026-02-01).
| # | Title | Status | Priority | Source |
|---|-------|--------|----------|--------|
| [T0](0.md) | User profile leaks comments across namespaces | resolved | high | meta `cc62eb06` |
| [T1](1.md) | Namespace/URI case-sensitivity causes "stock comments" | resolved | high | FAQ `7eb0baec`, `6b21e360` |
| [T2](2.md) | Large thread fetch causes 502 | resolved | high | production `www.remarkbox.com` |
| [T3](3.md) | GDPR/CCPA compliance tooling | resolved | medium | meta `a819d4f0` |
| [T4](4.md) | Customizable button text | resolved | low | meta `c29c9c22`, FAQ `5a508a9c` |
| [T5](5.md) | Self-service namespace deletion | resolved | low | FAQ `72475f88` |
| [T6](6.md) | Mention notifications (@username) | resolved | low | meta `b82332ce` |
| [T7](7.md) | Webmentions / IndieWeb support | resolved | low | meta `bd7112ff` |
| [T8](8.md) | Nesting depth settings per namespace | resolved | low | meta `137101ac` |
| [T9](9.md) | Prevent duplicate threads (AJAX search) | resolved | low | meta `642831b4` |
| [T10](10.md) | Browser push notifications | resolved | low | meta `9b970f14` |
| [T11](11.md) | Document "comments disappear when moving embed" | resolved | low | FAQ `6260e726` |
| [T12](12.md) | Reply to API-only CRUD thread confirming done | resolved | low | meta `6db01560` |
| [T13](13.md) | Reply to lock/archive thread confirming done | resolved | low | meta `7e9d5864` |

14
remarkbox/__init__.py
View file

@ -521,6 +521,19 @@ def main(global_config, **settings):
# Final fallback to light
return "light"
def add_webmentions(request):
"""Return verified webmentions for the current thread, or empty list."""
from remarkbox.models.webmention import get_verified_webmentions_for_node
if request.node and request.node.is_root:
return get_verified_webmentions_for_node(
request.dbsession, request.node.id
)
elif request.node:
return get_verified_webmentions_for_node(
request.dbsession, request.node.root_id
)
return []
# register functions to app config as request methods.
# each request instance will run these functions and attach results.
# cache result with `reify=True` to prevent multiple db lookups.
@ -572,6 +585,7 @@ def main(global_config, **settings):
config.add_request_method(add_node_order, "node_order", reify=True)
config.add_request_method(add_mathjax, "mathjax", reify=True)
config.add_request_method(add_theme_mode, "theme_mode", reify=True)
config.add_request_method(add_webmentions, "webmentions", reify=True)
# API routes must be included before .routes because
# basic-show-node2 (/{node_id}/{slug:.*}) is a catch-all

2
remarkbox/api/__init__.py
View file

@ -1,6 +1,7 @@
def includeme(config):
config.add_route("api-version", "/api/v1/version")
config.add_route("api-threads-list", "/api/v1/threads")
config.add_route("api-threads-search", "/api/v1/threads/search")
config.add_route("api-thread-detail", "/api/v1/threads/{node_id}")
config.add_route("api-thread-replies", "/api/v1/threads/{node_id}/replies")
config.add_route("api-node-detail", "/api/v1/nodes/{node_id}")
@ -8,4 +9,5 @@ def includeme(config):
config.add_route("api-auth-verify", "/api/v1/auth/verify")
config.add_route("api-user-profile", "/api/v1/user/profile")
config.add_route("api-client-python", "/api/v1/clients/python")
config.add_route("api-webmention", "/api/v1/webmention")
config.scan("remarkbox.api.views")

19
remarkbox/api/remarkbox_client.py
View file

@ -200,16 +200,27 @@ class RemarkboxClient:
params = urllib.parse.urlencode({"namespace": namespace, "page": page})
return self._request("GET", "/api/v1/threads?" + params)
def get_thread(self, node_id):
"""Get a thread and all its replies.
def get_thread(self, node_id, limit=None, offset=None):
"""Get a thread and its replies (paginated).
Args:
node_id: UUID of the root thread node
limit: Maximum replies to return (default 100, max 500)
offset: Number of replies to skip (default 0)
Returns:
dict with keys: namespace, thread, replies
dict with keys: namespace, thread, replies, total_replies,
page, limit, offset, has_more
"""
return self._request("GET", "/api/v1/threads/{}".format(node_id))
params = {}
if limit is not None:
params["limit"] = limit
if offset is not None:
params["offset"] = offset
path = "/api/v1/threads/{}".format(node_id)
if params:
path += "?" + urllib.parse.urlencode(params)
return self._request("GET", path)
def create_thread(self, namespace, title, data, anonymous_name=None, email=None):
"""Create a new thread.

113
remarkbox/api/views.py
View file

@ -143,6 +143,49 @@ def api_list_threads(request):
}
@view_config(
route_name="api-threads-search",
request_method="GET",
renderer="json",
require_csrf=False,
)
def api_search_threads(request):
"""Search threads by title prefix within a namespace."""
namespace_name = get_param(request, "namespace")
q = get_param(request, "q", "").strip()
if not namespace_name:
request.response.status_code = 400
return {"error": "namespace parameter is required"}
if not q or len(q) < 2:
return {"threads": []}
namespace = get_or_create_namespace(request.dbsession, namespace_name)
from remarkbox.models.node import Node
roots = (
namespace.visible_roots
.filter(Node.title.ilike("{}%".format(q.replace("%", "\\%").replace("_", "\\_"))))
.limit(10)
.all()
)
return {
"threads": [
{
"id": str(root.id),
"title": root.title,
"path": root.path,
"created_ago": root.human_created_timestamp,
"stats": root.stats if root.cache else None,
}
for root in roots
],
}
@view_config(
route_name="api-thread-detail",
request_method="GET",
@ -150,7 +193,12 @@ def api_list_threads(request):
require_csrf=False,
)
def api_get_thread(request):
"""Get a thread with all its replies."""
"""Get a thread with its replies (paginated).
Query parameters:
limit: Maximum replies to return (default 100, max 500).
offset: Number of replies to skip (default 0).
"""
node_id = request.matchdict["node_id"]
node = get_node_by_id(request.dbsession, node_id)
@ -165,16 +213,56 @@ def api_get_thread(request):
if denied:
return denied
nodes = get_nodes_who_share_root(request.dbsession, root, namespace.node_order)
# Parse pagination params
try:
limit = int(get_param(request, "limit", 100))
except (TypeError, ValueError):
limit = 100
limit = max(1, min(limit, 500))
try:
offset = int(get_param(request, "offset", 0))
except (TypeError, ValueError):
offset = 0
offset = max(0, offset)
# Build SQL-side visibility filters (mirrors namespace.can_see_node logic
# for anonymous / non-moderator users; moderators and node owners would
# see hidden nodes but that edge case is small and acceptable to omit
# from the API for performance).
visibility_filters = {"disabled": False}
if namespace.hide_unless_approved:
visibility_filters["approved"] = True
if namespace.hide_unverified:
visibility_filters["verified"] = True
# Get total visible reply count (excluding root) for pagination metadata
count_query = get_nodes_who_share_root(
request.dbsession, root,
exclude_root=True,
visibility_filters=visibility_filters,
)
total_replies = count_query.count()
# Fetch the paginated slice
nodes = get_nodes_who_share_root(
request.dbsession, root, namespace.node_order,
limit=limit, offset=offset,
exclude_root=True,
visibility_filters=visibility_filters,
)
page = (offset // limit) + 1 if limit else 1
return {
"namespace": serialize_namespace_brief(namespace),
"thread": serialize_node(root, include_children=True),
"replies": [
serialize_node(n)
for n in nodes
if n.id != root.id and namespace.can_see_node(n, request.user)
],
"replies": [serialize_node(n) for n in nodes],
"total_replies": total_replies,
"page": page,
"limit": limit,
"offset": offset,
"has_more": (offset + limit) < total_replies,
}
@ -243,7 +331,7 @@ def api_create_thread(request):
node.namespace = namespace
node.ip_address = str(request.client_addr)
node.title = title
node.set_data(data)
node.set_data(data, dbsession=request.dbsession)
if user_surrogate:
node.user_surrogate = user_surrogate
@ -301,6 +389,13 @@ def api_reply(request):
request.response.status_code = 403
return {"error": "Thread is locked"}
# Enforce max nesting depth (T8)
namespace = parent.root.namespace
if namespace.max_nesting_depth is not None:
if parent.depth >= namespace.max_nesting_depth:
request.response.status_code = 403
return {"error": "Maximum nesting depth reached"}
body = get_json_body(request)
data = body.get("data") or request.params.get("thread_data", "")
anonymous_name = (
@ -346,7 +441,7 @@ def api_reply(request):
# Create child node
child = parent.new_child()
child.ip_address = str(request.client_addr)
child.set_data(data, namespace=namespace)
child.set_data(data, namespace=namespace, dbsession=request.dbsession)
if user_surrogate:
child.user_surrogate = user_surrogate

82
remarkbox/lib/mentions.py Normal file
View file

@ -0,0 +1,82 @@
"""
Parse @username mentions from comment text and resolve them to User objects.
"""
import re
import logging
log = logging.getLogger(__name__)
# Match @username where username is alphanumeric with dashes (matching
# is_user_name_valid from models/user.py). Must be preceded by whitespace
# or start-of-string to avoid matching email addresses like foo@bar.
MENTION_RE = re.compile(r'(?:^|(?<=\s))@([A-Za-z0-9](?:[A-Za-z0-9-]*[A-Za-z0-9])?)')
def parse_mention_usernames(text):
"""Return a set of unique usernames mentioned in text (without the @)."""
if not text:
return set()
return set(MENTION_RE.findall(text))
def resolve_mentions(dbsession, text):
"""
Parse @mentions from text, look up each username in the database.
Returns a dict mapping lowercase username -> User object for users
that exist. Non-existent usernames are omitted.
"""
# Import here to avoid circular import (user -> node -> render -> mentions -> user).
from remarkbox.models.user import get_user_by_name
usernames = parse_mention_usernames(text)
resolved = {}
for username in usernames:
user = get_user_by_name(dbsession, username)
if user is not None:
resolved[username.lower()] = user
return resolved
def replace_mentions_with_links(html, resolved_users, link_prefix=""):
"""
In rendered HTML, replace @username with profile links for resolved users.
Non-existent usernames (not in resolved_users) are left as plain text.
Args:
html: the rendered HTML string
resolved_users: dict of lowercase username -> User object
link_prefix: URL prefix for profile links (e.g. "" or "/embed/ns/foo")
Returns:
HTML with @mentions converted to anchor tags for valid users.
"""
if not resolved_users:
return html
def _replace(match):
username = match.group(1)
user = resolved_users.get(username.lower())
if user is None:
# Not a real user, leave as plain text.
return match.group(0)
# Use the user's canonical name for the display and link.
return '<a href="{}/u/{}" class="mention">@{}</a>'.format(
link_prefix, user.name, user.name
)
# Replace @username patterns in HTML, but skip anything inside tags
# (e.g. inside href attributes). We use a two-pass approach:
# first split on HTML tags, then only do replacements in text segments.
parts = re.split(r'(<[^>]+>)', html)
result = []
for i, part in enumerate(parts):
if part.startswith('<'):
# This is an HTML tag, leave it alone.
result.append(part)
else:
# This is a text segment, apply mention replacement.
result.append(MENTION_RE.sub(_replace, part))
return ''.join(result)

115
remarkbox/lib/notify.py
View file

@ -67,6 +67,30 @@ def filter_watchers(watchers, exclude_users=None, include_users=None):
return f_watchers
def get_mentioned_user_watchers(node):
"""
Parse @mentions from a node's data and return reply watchers
for mentioned users who exist.
"""
from remarkbox.lib.mentions import resolve_mentions
if not node.data:
return []
dbsession = node.dbsession
if dbsession is None:
return []
resolved = resolve_mentions(dbsession, node.data)
watchers = []
for user in resolved.values():
if user.verified:
# Use the user's reply watcher for mention notifications.
for w in user.reply_watchers:
watchers.append(w)
return watchers
def get_all_watchers(request, node_event):
"""Given a request and node_event, return all watcher objects."""
# Note: we only notify a user once per method per event.
@ -86,6 +110,9 @@ def get_all_watchers(request, node_event):
# know there was a new child node added to the conversation.
watchers.extend(node.parent.user.reply_watchers)
# extend watchers for @mentioned users.
watchers.extend(get_mentioned_user_watchers(node))
# extend the watchers list with any watchers of the request's root (thread).
watchers.extend(node.root.watchers)
@ -201,6 +228,44 @@ def deliver_scheduled_notifications(request=None):
send_digest_notifications(request, filtered_dict, "weekly")
def _send_push_for_notification(request, notification, root, namespace):
"""Send a push notification for a single notification if the user wants it."""
from remarkbox.lib.push import PUSH_AVAILABLE, send_push_to_user
if not PUSH_AVAILABLE:
return
user = notification.user
pref = getattr(user, "notification_preference", "email")
if pref not in ("push", "both"):
return
node = notification.node_event.node
action = notification.node_event.action
author = node.user.name if node.user else "Someone"
action_text = {
"created": "started a new thread",
"commented": "posted a reply",
"approved": "approved a comment",
"enabled": "enabled a comment",
"disabled": "disabled a comment",
"verified": "verified a comment",
}.get(action, action)
title = "[{}] new activity".format(namespace.name)
body = "{} {} on {}".format(author, action_text, root.title or "a thread")
url = "{}/r/{}".format(request.host_url, node.id)
payload = {
"title": title,
"body": body,
"url": url,
"tag": "remarkbox-{}".format(str(notification.id)[:8]),
}
send_push_to_user(request, user, payload)
def send_immediate_notifications(request, notifications):
deliver_email_notifications = request.app.get("deliver_email_notifications", True)
@ -214,30 +279,38 @@ def send_immediate_notifications(request, notifications):
subject = "[{}] new activity".format(namespace.name)
for notification in notifications:
user_pref = getattr(notification.user, "notification_preference", "email")
# Send email if the user wants email (or both).
if deliver_email_notifications and notification.method == "email":
send_template_email(
request,
notification.user.email,
subject,
"mail_immediate_text.j2",
"mail_immediate_html.j2",
{
"request": request,
"notification": notification,
"root": root,
"subject": subject,
},
)
log.info(
"notification frequency=immediately user={} ({}), count={}".format(
notification.user.name,
notification.user_id,
notification.id,
if user_pref in ("email", "both"):
send_template_email(
request,
notification.user.email,
subject,
"mail_immediate_text.j2",
"mail_immediate_html.j2",
{
"request": request,
"notification": notification,
"root": root,
"subject": subject,
},
)
# Send push notification if the user wants push (or both).
_send_push_for_notification(request, notification, root, namespace)
log.info(
"notification frequency=immediately user={} ({}), count={}".format(
notification.user.name,
notification.user_id,
notification.id,
)
notification.sent = True
request.dbsession.add(notification)
request.dbsession.flush()
)
notification.sent = True
request.dbsession.add(notification)
request.dbsession.flush()
def group_notifications_by_root(notifications):

213
remarkbox/lib/push.py Normal file
View file

@ -0,0 +1,213 @@
"""
Web Push notification support using the VAPID protocol.
This module provides:
- VAPID key generation and management
- Push subscription storage helpers
- Sending push notifications via the Web Push protocol
Dependencies:
- py_vapid: VAPID key generation and signing
- pywebpush: Web Push API client
If these packages are not installed, push notifications are silently
disabled and all functions become no-ops.
"""
import json
import logging
import os
log = logging.getLogger(__name__)
# Try to import push dependencies. If they are not installed, push
# notification support is silently disabled.
try:
from pywebpush import webpush, WebPushException
from py_vapid import Vapid
PUSH_AVAILABLE = True
except ImportError:
PUSH_AVAILABLE = False
log.info("pywebpush/py_vapid not installed; push notifications disabled")
def get_vapid_keys(settings):
"""
Return a dict with 'private_key' and 'public_key' VAPID strings.
Reads from the application settings (ini file):
push.vapid_private_key
push.vapid_public_key
push.vapid_contact (mailto: URI for the VAPID contact)
If keys are not configured, returns None.
"""
private_key = settings.get("push.vapid_private_key")
public_key = settings.get("push.vapid_public_key")
contact = settings.get("push.vapid_contact", "")
if private_key and public_key:
return {
"private_key": private_key,
"public_key": public_key,
"contact": contact,
}
return None
def generate_vapid_keys():
"""
Generate a new VAPID key pair for initial setup.
Returns a dict with 'private_key' and 'public_key' as base64url strings,
suitable for pasting into the .ini configuration file.
Usage (from a Python shell):
from remarkbox.lib.push import generate_vapid_keys
keys = generate_vapid_keys()
print(keys)
"""
if not PUSH_AVAILABLE:
raise RuntimeError(
"pywebpush and py_vapid must be installed to generate VAPID keys. "
"Run: pip install pywebpush py_vapid"
)
vapid = Vapid()
vapid.generate_keys()
return {
"private_key": vapid.private_pem(),
"public_key": vapid.public_key_urlsafe_base64(),
}
def send_push_notification(subscription_info, payload, vapid_keys):
"""
Send a push notification to a single subscription.
Args:
subscription_info: dict with 'endpoint', 'keys' (p256dh, auth)
payload: dict to JSON-encode as the notification body
vapid_keys: dict from get_vapid_keys()
Returns True on success, False on failure.
"""
if not PUSH_AVAILABLE:
return False
if not vapid_keys:
log.warning("VAPID keys not configured; cannot send push notification")
return False
try:
webpush(
subscription_info=subscription_info,
data=json.dumps(payload),
vapid_private_key=vapid_keys["private_key"],
vapid_claims={
"sub": vapid_keys["contact"],
},
)
return True
except WebPushException as e:
log.warning("Push notification failed: %s", e, exc_info=True)
# A 410 Gone response means the subscription is no longer valid.
if hasattr(e, "response") and e.response is not None:
if e.response.status_code == 410:
log.info("Subscription expired (410 Gone), should be removed")
return False
except Exception:
log.warning("Unexpected error sending push notification", exc_info=True)
return False
def send_push_to_user(request, user, payload):
"""
Send a push notification to all active subscriptions for a user.
Args:
request: Pyramid request (used to read settings)
user: User model instance
payload: dict to send as the notification body
Returns the number of successful sends.
"""
if not PUSH_AVAILABLE:
return 0
vapid_keys = get_vapid_keys(request.registry.settings)
if not vapid_keys:
return 0
subscriptions = get_push_subscriptions(user)
if not subscriptions:
return 0
success_count = 0
expired = []
for sub in subscriptions:
ok = send_push_notification(sub, payload, vapid_keys)
if ok:
success_count += 1
else:
# Track potentially expired subscriptions for cleanup.
expired.append(sub)
return success_count
def get_push_subscriptions(user):
"""
Return the list of push subscription dicts for a user.
Subscriptions are stored as a JSON string in user.push_subscriptions.
Returns an empty list if no subscriptions exist.
"""
if not user.push_subscriptions:
return []
try:
return json.loads(user.push_subscriptions)
except (json.JSONDecodeError, TypeError):
return []
def add_push_subscription(user, subscription_info):
"""
Add a push subscription for a user, avoiding duplicates.
Args:
user: User model instance
subscription_info: dict with 'endpoint', 'keys' (p256dh, auth)
Returns True if the subscription was added, False if already exists.
"""
subscriptions = get_push_subscriptions(user)
# Check for duplicate endpoint.
for sub in subscriptions:
if sub.get("endpoint") == subscription_info.get("endpoint"):
return False
subscriptions.append(subscription_info)
user.push_subscriptions = json.dumps(subscriptions)
return True
def remove_push_subscription(user, endpoint):
"""
Remove a push subscription by endpoint URL.
Args:
user: User model instance
endpoint: The push subscription endpoint URL to remove
Returns True if removed, False if not found.
"""
subscriptions = get_push_subscriptions(user)
original_count = len(subscriptions)
subscriptions = [s for s in subscriptions if s.get("endpoint") != endpoint]
if len(subscriptions) < original_count:
user.push_subscriptions = json.dumps(subscriptions) if subscriptions else None
return True
return False

14
remarkbox/lib/render.py
View file

@ -5,6 +5,8 @@ from .sanitize_html import (
clean_raw_html,
)
from .mentions import resolve_mentions, replace_mentions_with_links
import logging
log = logging.getLogger(__name__)
@ -51,10 +53,18 @@ def make_cleaner_from_namespace(namespace):
return cleaner
def markdown_to_html(data, namespace=None):
def markdown_to_html(data, namespace=None, dbsession=None):
raw_html = markdown_to_raw_html(data, extra_extensions=["mdx_math"])
if namespace:
cleaner = make_cleaner_from_namespace(namespace)
else:
cleaner = default_cleaner()
return clean_raw_html(raw_html, cleaner)
html = clean_raw_html(raw_html, cleaner)
# After sanitization, resolve @mentions and convert to profile links.
if dbsession is not None:
resolved = resolve_mentions(dbsession, data)
if resolved:
html = replace_mentions_with_links(html, resolved)
return html

1
remarkbox/models/__init__.py
View file

@ -19,6 +19,7 @@ from .event import *
from .notification import *
from .pay_what_you_can import *
from .payment import *
from .webmention import *
# run configure_mappers after defining all of the models to ensure
# all relationships can be setup

3
remarkbox/models/meta.py
View file

@ -40,6 +40,7 @@ CLASS_TO_TABLE = {
"NodeEventNotification": "rb_node_event_notification",
"PayWhatYouCan": "rb_pay_what_you_can",
"Payment": "rb_payment",
"Webmention": "rb_webmention",
}
# node (threads), namespace (forum)
@ -47,7 +48,7 @@ WATCHER_TYPES = {"reply", "node", "namespace"}
NOTIFICATION_FREQUENCIES = {"never", "immediately", "daily", "weekly"}
NOTIFICATION_METHODS = {"email"}
NOTIFICATION_METHODS = {"email", "push"}
NODE_EVENT_ACTIONS = {
"enabled",

12
remarkbox/models/namespace.py
View file

@ -60,6 +60,11 @@ PROTECTED_ATTRIBUTES = {
"reverse_order": False,
"group_conversations": False,
"api_access": True,
"submit_button_text": None,
"comment_label_singular": None,
"comment_label_plural": None,
"max_nesting_depth": None,
"collapse_depth": None,
}
@ -118,6 +123,13 @@ class Namespace(RBase, Base):
# the group postfix used for imports (e.g., "rb" creates "Anonymous-rb")
# Once set, this becomes permanent for all imported surrogates
import_group_postfix = Column(Unicode(6), default=None, nullable=True)
# T4: customizable button text and comment labels
submit_button_text = Column(Unicode(256), default=None, nullable=True)
comment_label_singular = Column(Unicode(256), default=None, nullable=True)
comment_label_plural = Column(Unicode(256), default=None, nullable=True)
# T8: nesting depth settings
max_nesting_depth = Column(Integer, default=None, nullable=True)
collapse_depth = Column(Integer, default=None, nullable=True)
# the type of subscription of this Namespace.
subscription_type = Column(
Enum(*SUBSCRIPTION_TYPES, name="subscription_type"),

45
remarkbox/models/node.py
View file

@ -312,11 +312,13 @@ class Node(RBase, Base):
def enabled(self):
return not self.disabled
def set_data(self, data, namespace=None):
def set_data(self, data, namespace=None, dbsession=None):
if namespace is None:
namespace = self.root.namespace
if dbsession is None:
dbsession = self.dbsession
self.data = data
self.data_html = markdown_to_html(data, namespace)
self.data_html = markdown_to_html(data, namespace, dbsession=dbsession)
def _invalidate_cache(self):
if self.root.cache:
@ -499,12 +501,49 @@ def get_nodes_who_share_roots(dbsession, root_nodes):
return get_nodes_who_share_roots_query(dbsession, root_nodes).all()
def get_nodes_who_share_root(dbsession, root_node, order="oldest-first"):
def get_nodes_who_share_root(dbsession, root_node, order="oldest-first",
limit=None, offset=None,
exclude_root=False,
visibility_filters=None):
"""Return nodes sharing a root, with optional pagination and SQL-side filtering.
Args:
dbsession: SQLAlchemy session.
root_node: The root Node whose tree to query.
order: 'oldest-first' or 'newest-first'.
limit: Maximum number of rows to return (None = unlimited).
offset: Number of rows to skip (None = 0).
exclude_root: If True, exclude the root node itself from results.
visibility_filters: Optional dict of SQL visibility filters to apply.
Supported keys: disabled (bool), approved (bool), verified (bool).
Returns:
SQLAlchemy query object (call .all() to materialise).
"""
nodes = dbsession.query(Node).filter(Node.root_id == root_node.id)
if exclude_root:
nodes = nodes.filter(Node.id != root_node.id)
# Apply SQL-side visibility filters
if visibility_filters:
if "disabled" in visibility_filters:
nodes = nodes.filter(Node.disabled == visibility_filters["disabled"])
if "approved" in visibility_filters:
nodes = nodes.filter(Node.approved == visibility_filters["approved"])
if "verified" in visibility_filters:
nodes = nodes.filter(Node.verified == visibility_filters["verified"])
if order == "oldest-first":
nodes = nodes.order_by(Node.created)
elif order == "newest-first":
nodes = nodes.order_by(Node.created.desc())
if offset is not None:
nodes = nodes.offset(offset)
if limit is not None:
nodes = nodes.limit(limit)
return nodes

171
remarkbox/models/user.py
View file

@ -1,4 +1,4 @@
from sqlalchemy import BigInteger, Boolean, Integer, Column, Unicode, Enum, func, or_
from sqlalchemy import BigInteger, Boolean, Integer, Column, Unicode, UnicodeText, Enum, func, or_
from sqlalchemy.orm import relationship, backref
@ -120,6 +120,14 @@ class User(RBase, Base):
default='auto',
nullable=False,
)
# Notification delivery preference: how the user wants to receive notifications.
notification_preference = Column(
Enum('email', 'push', 'both', 'none', name='notification_preference_enum'),
default='email',
nullable=False,
)
# JSON-encoded list of Web Push subscription objects.
push_subscriptions = Column(UnicodeText, nullable=True)
votes = relationship(argument="Vote", backref="user", order_by="desc(Vote.created)")
# lazy='dynamic' returns a query object instead of collection.
@ -247,33 +255,47 @@ class User(RBase, Base):
def unverified_namespace_owner_requests(self):
return [nr for nr in self.namespace_owner_requests if not nr.verified]
@property
def verified_nodes(self):
return self.nodes.filter(Node.verified == True, Node.disabled == False)
def _namespace_filter(self, query, namespace=None):
"""Apply namespace filter to a node query if namespace is provided."""
if namespace is not None:
root_ids = self.dbsession.query(Node.id).filter(
Node.namespace_id == namespace.id
)
query = query.filter(Node.root_id.in_(root_ids))
return query
@property
def unverified_nodes(self):
return self.nodes.filter(Node.verified == False, Node.disabled == False)
def verified_nodes(self, namespace=None):
query = self.nodes.filter(Node.verified == True, Node.disabled == False)
return self._namespace_filter(query, namespace)
@property
def disabled_nodes(self):
return self.nodes.filter(Node.verified == True, Node.disabled == True)
def unverified_nodes(self, namespace=None):
query = self.nodes.filter(Node.verified == False, Node.disabled == False)
return self._namespace_filter(query, namespace)
@property
def unapproved_nodes(self):
return self.nodes.filter(or_(Node.approved == False, Node.approved.is_(None)))
def disabled_nodes(self, namespace=None):
query = self.nodes.filter(Node.verified == True, Node.disabled == True)
return self._namespace_filter(query, namespace)
def page_nodes(self, limit=100, offset=0):
def unapproved_nodes(self, namespace=None):
query = self.nodes.filter(or_(Node.approved == False, Node.approved.is_(None)))
return self._namespace_filter(query, namespace)
def page_nodes(self, namespace=None, limit=100, offset=0):
if self.nodes.count() == 0:
return []
return (
self.nodes.filter(
Node.disabled == False, Node.verified == True, Node.user_id != None, Node.approved == True
)
.order_by(Node.changed.desc())
.limit(limit)
.offset(offset)
query = self.nodes.filter(
Node.disabled == False, Node.verified == True,
Node.user_id != None, Node.approved == True
)
query = self._namespace_filter(query, namespace)
if namespace is not None:
if namespace.hide_unless_approved:
query = query.filter(Node.approved == True)
if namespace.hide_unverified:
query = query.filter(Node.verified == True)
return query.order_by(Node.changed.desc()).limit(limit).offset(offset)
def __init__(self, email):
self.name = unicode(generate_password(size=8))
@ -374,6 +396,113 @@ class User(RBase, Base):
return reply_watcher
return self.reply_watchers.first()
def anonymize_account(self):
"""Anonymize this user's account for GDPR/CCPA compliance.
Scrubs PII (name, email, password) and cleans up metadata,
but keeps the user record as a tombstone so node.user_id
foreign keys remain valid and templates still work.
"""
dbsession = self.dbsession
tombstone_id = uuid.uuid4().hex[:12]
# Scrub PII on the user record — keep it as a tombstone.
self.name = "deleted-{}".format(tombstone_id)
self.email = "deleted-{}@localhost".format(tombstone_id)
self.password = None
self.password_attempts = 0
self.gravatar = False
self.verified = False
self.disabled = True
# Scrub IP addresses on all comments.
for node in self.nodes.all():
node.ip_address = None
dbsession.add(node)
# Delete all notification records.
for notification in self.node_notifications.all():
dbsession.delete(notification)
# Delete all watchers and their notifications.
for watcher in self.watchers.all():
for notification in watcher.notifications.all():
dbsession.delete(notification)
dbsession.delete(watcher)
# Delete all events and their notifications.
for event in self.events:
notifications = dbsession.query(NodeEventNotification).filter(
NodeEventNotification.node_event_id == event.id
).all()
for notification in notifications:
dbsession.delete(notification)
dbsession.delete(event)
# Delete OAuth records.
for oauth in self.oauth_records.all():
dbsession.delete(oauth)
# Delete namespace owner requests.
for nr in self.namespace_owner_requests.all():
dbsession.delete(nr)
# Delete namespace user associations.
for nsu in list(self.user_namespaces):
dbsession.delete(nsu)
# Delete payment records.
if self.pay_what_you_can:
dbsession.delete(self.pay_what_you_can)
for payment in self.payments.all():
dbsession.delete(payment)
# Delete votes.
for vote in self.votes:
dbsession.delete(vote)
dbsession.add(self)
dbsession.flush()
def export_user_data(self):
"""Export all user data as a dictionary for GDPR/CCPA data subject requests."""
from remarkbox.lib import timestamp_to_date_string
data = {
"profile": {
"id": str(self.id),
"name": self.name,
"email": self.email,
"created": timestamp_to_date_string(self.created),
"gravatar": self.gravatar,
"verified": self.verified,
"theme_mode": self.theme_mode,
},
"comments": [],
}
for node in self.nodes.all():
comment = {
"id": str(node.id),
"created": timestamp_to_date_string(node.created),
"changed": timestamp_to_date_string(node.changed),
"content": node.data,
"disabled": node.disabled,
"verified": node.verified,
"approved": node.approved,
}
if node.title:
comment["title"] = node.title
if node.namespace:
comment["namespace"] = node.namespace.name
if node.root and node.root.title:
comment["thread_title"] = node.root.title
if node.root and node.root.uri:
comment["thread_uri"] = node.root.uri.data
data["comments"].append(comment)
return data
def _user_by_name_query(dbsession, name):
"""query User by case insensitive name."""

77
remarkbox/models/webmention.py Normal file
View file

@ -0,0 +1,77 @@
from sqlalchemy import (
BigInteger,
Boolean,
Column,
Unicode,
UnicodeText,
)
from sqlalchemy.orm import relationship
import uuid
from .meta import Base, RBase, UUIDType, foreign_key, now_timestamp, get_object_by_id
import logging
log = logging.getLogger(__name__)
class Webmention(RBase, Base):
"""
Represents a received webmention -- a notification from an external
site that it has linked to a page with a Remarkbox thread.
Reference: https://www.w3.org/TR/webmention/
"""
id = Column(UUIDType, primary_key=True, index=True)
source = Column(Unicode(2048), nullable=False)
target = Column(Unicode(2048), nullable=False)
node_id = Column(UUIDType, foreign_key("Node", "id"), index=True, nullable=True)
verified = Column(Boolean, default=False, nullable=False)
author_name = Column(Unicode(256), nullable=True)
author_url = Column(Unicode(2048), nullable=True)
content = Column(UnicodeText, nullable=True)
created_timestamp = Column(BigInteger, nullable=False)
updated_timestamp = Column(BigInteger, nullable=False)
node = relationship(argument="Node", uselist=False, lazy="joined")
def __init__(self, source, target):
self.id = uuid.uuid1()
self.source = source
self.target = target
self.created_timestamp = now_timestamp()
self.updated_timestamp = now_timestamp()
def mark_verified(self, author_name=None, author_url=None, content=None):
self.verified = True
self.updated_timestamp = now_timestamp()
if author_name:
self.author_name = author_name
if author_url:
self.author_url = author_url
if content:
self.content = content[:500]
def get_webmention_by_id(dbsession, webmention_id):
return get_object_by_id(dbsession, webmention_id, Webmention)
def get_webmention_by_source_and_target(dbsession, source, target):
return (
dbsession.query(Webmention)
.filter(Webmention.source == source, Webmention.target == target)
.one_or_none()
)
def get_verified_webmentions_for_node(dbsession, node_id):
return (
dbsession.query(Webmention)
.filter(Webmention.node_id == node_id, Webmention.verified == True)
.order_by(Webmention.created_timestamp.desc())
.all()
)

14
remarkbox/routes.py
View file

@ -13,6 +13,14 @@ def includeme(config):
config.add_route("billing-success", "/billing/success")
config.add_route("stripe-webhook", "/webhook/stripe")
# webmention: IndieWeb webmention receiving endpoint.
config.add_route("webmention", "/webmention")
# push notifications: Web Push API endpoints.
config.add_route("push-vapid-key", "/push/vapid-key")
config.add_route("push-subscribe", "/push/subscribe")
config.add_route("push-unsubscribe", "/push/unsubscribe")
# slack: bot notifications and oauth.
config.add_route("oauth-slack", "/oauth/slack")
config.add_route("oauth-slack-delete", "/oauth/slack/delete")
@ -67,6 +75,7 @@ def includeme(config):
config.add_route("embed-log-out", "/embed/ns/{namespace}/log-out")
config.add_route("embed-namespace-nodes", "/embed/ns/{namespace}/nodes")
config.add_route("embed-namespace-delete", "/embed/ns/{namespace}/delete")
config.add_route("embed-namespace-settings", "/embed/ns/{namespace}/settings")
config.add_route("embed-namespace-import-comments", "/embed/ns/{namespace}/import-comments")
config.add_route(
@ -74,6 +83,8 @@ def includeme(config):
)
config.add_route("embed-namespace", "/embed/ns/{namespace}")
config.add_route("embed-user-delete-account", "/embed/ns/{namespace}/u/delete-account")
config.add_route("embed-user-export-data", "/embed/ns/{namespace}/u/export-data")
config.add_route("embed-user-settings", "/embed/ns/{namespace}/u/settings")
config.add_route("embed-user-watching", "/embed/ns/{namespace}/u/watching")
config.add_route("embed-user-notifications", "/embed/ns/{namespace}/u/notifications")
@ -99,6 +110,7 @@ def includeme(config):
config.add_route("basic-verification-challenge", "/verification-challenge")
config.add_route("basic-namespace-nodes", "/ns/{namespace}/nodes")
config.add_route("basic-namespace-delete", "/ns/{namespace}/delete")
config.add_route("basic-namespace-settings", "/ns/{namespace}/settings")
config.add_route("basic-namespace-import-comments", "/ns/{namespace}/import-comments")
config.add_route("basic-namespace-stats-json", "/ns/{namespace}/stats.json")
@ -110,6 +122,8 @@ def includeme(config):
)
config.add_route("basic-namespace", "/ns/{namespace}")
config.add_route("basic-user-delete-account", "/u/delete-account")
config.add_route("basic-user-export-data", "/u/export-data")
config.add_route("basic-user-settings", "/u/settings")
config.add_route("basic-user-watching", "/u/watching")
config.add_route("basic-user-notifications", "/u/notifications")

34
remarkbox/scripts/alembic/versions/896568b0752e_add_customizable_button_text_comment_.py Normal file
View file

@ -0,0 +1,34 @@
"""add customizable button text, comment labels, and nesting depth settings to namespace
Revision ID: 896568b0752e
Revises: a3f7b2c1d4e5
Create Date: 2026-02-01 18:28:50.433155
"""
# revision identifiers, used by Alembic.
revision = '896568b0752e'
down_revision = 'a3f7b2c1d4e5'
branch_labels = None
depends_on = None
from alembic import op
import sqlalchemy as sa
def upgrade():
# T4: customizable button text and comment labels
op.add_column('rb_namespace', sa.Column('submit_button_text', sa.Unicode(length=256), nullable=True))
op.add_column('rb_namespace', sa.Column('comment_label_singular', sa.Unicode(length=256), nullable=True))
op.add_column('rb_namespace', sa.Column('comment_label_plural', sa.Unicode(length=256), nullable=True))
# T8: nesting depth settings
op.add_column('rb_namespace', sa.Column('max_nesting_depth', sa.Integer(), nullable=True))
op.add_column('rb_namespace', sa.Column('collapse_depth', sa.Integer(), nullable=True))
def downgrade():
op.drop_column('rb_namespace', 'collapse_depth')
op.drop_column('rb_namespace', 'max_nesting_depth')
op.drop_column('rb_namespace', 'comment_label_plural')
op.drop_column('rb_namespace', 'comment_label_singular')
op.drop_column('rb_namespace', 'submit_button_text')

380
remarkbox/scripts/merge_duplicate_namespaces.py Normal file
View file

@ -0,0 +1,380 @@
"""
Merge duplicate namespaces and URIs that have the same name/host (case-insensitive).
This script finds namespaces with names that differ only by case (e.g.,
'Example.com' and 'example.com') and merges them into a single namespace,
keeping the oldest one. It also normalizes URI hostnames to lowercase.
Usage:
remarkbox_merge_duplicate_namespaces -c production.ini --dry-run
remarkbox_merge_duplicate_namespaces -c production.ini
"""
from collections import defaultdict
from pyramid.paster import bootstrap, setup_logging
from sqlalchemy import func
from ..models import Namespace, Node, Uri, NamespaceUser, NamespaceRequest, Oauth, Watcher
from . import base_parser
import miniuri
try:
unicode("")
except Exception:
from six import u as unicode
def get_arg_parser():
parser = base_parser("Find and merge namespaces with duplicate names (case-insensitive).")
parser.add_argument(
"--dry-run",
action="store_true",
default=False,
help="Show what would be merged without making changes.",
)
return parser
def find_duplicate_namespaces(dbsession):
"""
Find all namespaces with duplicate names (case-insensitive).
Returns a dict mapping lowercase name -> list of Namespace objects.
Only includes names with more than one namespace.
"""
duplicates = defaultdict(list)
namespaces = dbsession.query(Namespace).all()
for namespace in namespaces:
name_lower = namespace.name.lower()
duplicates[name_lower].append(namespace)
# Filter to only duplicates
return {
name: namespaces
for name, namespaces in duplicates.items()
if len(namespaces) > 1
}
def find_duplicate_uris(dbsession):
"""
Find all URIs with duplicate data (case-insensitive).
Returns a dict mapping lowercase uri -> list of Uri objects.
Only includes URIs with more than one record.
"""
duplicates = defaultdict(list)
uris = dbsession.query(Uri).all()
for uri in uris:
uri_lower = uri.data.lower()
duplicates[uri_lower].append(uri)
# Filter to only duplicates
return {
uri: uri_list
for uri, uri_list in duplicates.items()
if len(uri_list) > 1
}
def merge_namespaces(dbsession, keep_ns, delete_ns, dry_run=False):
"""
Merge delete_ns into keep_ns by transferring all related records.
Transfers:
- root nodes (threads)
- namespace_user associations
- namespace_requests
- oauth records
- watchers
- user_surrogates
"""
keep_id = keep_ns.id
delete_id = delete_ns.id
print(f" Merging '{delete_ns.name}' (id={delete_id}) into '{keep_ns.name}' (id={keep_id})")
if dry_run:
counts = {
'root_nodes': dbsession.query(Node).filter(Node.namespace_id == delete_id).count(),
'namespace_users': dbsession.query(NamespaceUser).filter(NamespaceUser.namespace_id == delete_id).count(),
'namespace_requests': dbsession.query(NamespaceRequest).filter(NamespaceRequest.namespace_id == delete_id).count(),
'oauth': dbsession.query(Oauth).filter(Oauth.namespace_id == delete_id).count(),
'watchers': dbsession.query(Watcher).filter(Watcher.namespace_id == delete_id).count(),
}
for table, count in counts.items():
if count > 0:
print(f" Would transfer {count} {table}")
return
# Transfer root nodes
updated = dbsession.query(Node).filter(Node.namespace_id == delete_id).update(
{Node.namespace_id: keep_id}, synchronize_session=False
)
if updated:
print(f" Transferred {updated} root nodes")
# Handle namespace_user associations (unique constraint on user_id, namespace_id)
keep_user_ids = {
nu.user_id for nu in dbsession.query(NamespaceUser).filter(NamespaceUser.namespace_id == keep_id)
}
delete_ns_users = dbsession.query(NamespaceUser).filter(NamespaceUser.namespace_id == delete_id).all()
transferred_ns = 0
deleted_ns = 0
for ns_user in delete_ns_users:
if ns_user.user_id in keep_user_ids:
dbsession.delete(ns_user)
deleted_ns += 1
else:
ns_user.namespace_id = keep_id
transferred_ns += 1
if transferred_ns:
print(f" Transferred {transferred_ns} namespace_user associations")
if deleted_ns:
print(f" Deleted {deleted_ns} duplicate namespace_user associations")
# Handle namespace requests
keep_request_user_ids = {
nr.user_id for nr in dbsession.query(NamespaceRequest).filter(NamespaceRequest.namespace_id == keep_id)
}
delete_ns_requests = dbsession.query(NamespaceRequest).filter(NamespaceRequest.namespace_id == delete_id).all()
transferred_nr = 0
deleted_nr = 0
for ns_request in delete_ns_requests:
if ns_request.user_id in keep_request_user_ids:
dbsession.delete(ns_request)
deleted_nr += 1
else:
ns_request.namespace_id = keep_id
transferred_nr += 1
if transferred_nr:
print(f" Transferred {transferred_nr} namespace requests")
if deleted_nr:
print(f" Deleted {deleted_nr} duplicate namespace requests")
# Transfer oauth records
updated = dbsession.query(Oauth).filter(Oauth.namespace_id == delete_id).update(
{Oauth.namespace_id: keep_id}, synchronize_session=False
)
if updated:
print(f" Transferred {updated} oauth records")
# Handle watchers (may have duplicates watching same node for same user)
keep_watcher_keys = {
(w.user_id, w.node_id, w.type)
for w in dbsession.query(Watcher).filter(Watcher.namespace_id == keep_id)
}
delete_watchers = dbsession.query(Watcher).filter(Watcher.namespace_id == delete_id).all()
transferred_w = 0
deleted_w = 0
for watcher in delete_watchers:
key = (watcher.user_id, watcher.node_id, watcher.type)
if key in keep_watcher_keys:
dbsession.delete(watcher)
deleted_w += 1
else:
watcher.namespace_id = keep_id
transferred_w += 1
if transferred_w:
print(f" Transferred {transferred_w} watchers")
if deleted_w:
print(f" Deleted {deleted_w} duplicate watchers")
dbsession.flush()
# Delete the duplicate namespace
dbsession.delete(delete_ns)
dbsession.flush()
print(f" Deleted namespace '{delete_ns.name}'")
def merge_uris(dbsession, keep_uri, delete_uri, dry_run=False):
"""
Merge delete_uri into keep_uri by transferring the node association.
If delete_uri has a node but keep_uri does not, transfer the node.
If both have nodes, transfer the children from delete_uri's node to keep_uri's node.
"""
print(f" Merging URI '{delete_uri.data}' into '{keep_uri.data}'")
if dry_run:
if delete_uri.node_id:
if keep_uri.node_id:
child_count = dbsession.query(Node).filter(Node.root_id == delete_uri.node_id, Node.parent_id != None).count()
print(f" Would transfer {child_count} child nodes from root {delete_uri.node_id} to root {keep_uri.node_id}")
else:
print(f" Would transfer node {delete_uri.node_id} to kept URI")
return
if delete_uri.node_id:
if keep_uri.node_id is None:
# keep_uri has no node, just reassign
keep_uri.node_id = delete_uri.node_id
delete_uri.node_id = None
dbsession.flush()
print(f" Transferred node to kept URI")
else:
# Both have nodes - transfer children from delete's root to keep's root
updated = dbsession.query(Node).filter(
Node.root_id == delete_uri.node_id,
Node.parent_id != None,
).update(
{Node.root_id: keep_uri.node_id}, synchronize_session=False
)
if updated:
print(f" Transferred {updated} child nodes to kept URI's root")
# Transfer direct children whose parent is the delete root
updated = dbsession.query(Node).filter(
Node.parent_id == delete_uri.node_id,
).update(
{Node.parent_id: keep_uri.node_id}, synchronize_session=False
)
if updated:
print(f" Re-parented {updated} direct children to kept URI's root")
# Delete the orphaned root node
orphan_root = dbsession.query(Node).filter(Node.id == delete_uri.node_id).one_or_none()
if orphan_root:
delete_uri.node_id = None
dbsession.flush()
dbsession.delete(orphan_root)
dbsession.flush()
print(f" Deleted orphaned root node")
# Delete the duplicate URI
dbsession.delete(delete_uri)
dbsession.flush()
print(f" Deleted URI '{delete_uri.data}'")
def normalize_uri_hostnames(dbsession, dry_run=False):
"""
Normalize all URI hostnames to lowercase.
"""
uris = dbsession.query(Uri).all()
normalized_count = 0
for uri in uris:
try:
parsed = miniuri.Uri(uri.data)
if parsed.hostname and parsed.hostname != parsed.hostname.lower():
old_data = uri.data
new_data = uri.data.replace(parsed.hostname, parsed.hostname.lower(), 1)
if dry_run:
print(f" Would normalize URI: '{old_data}' -> '{new_data}'")
else:
uri.data = new_data
dbsession.add(uri)
normalized_count += 1
except Exception:
pass
if normalized_count:
if not dry_run:
dbsession.flush()
print(f"{'Would normalize' if dry_run else 'Normalized'} {normalized_count} URI hostname(s) to lowercase")
return normalized_count
def normalize_namespace_names(dbsession, dry_run=False):
"""
Normalize all namespace names to lowercase.
"""
namespaces = dbsession.query(Namespace).all()
normalized_count = 0
for namespace in namespaces:
if namespace.name != namespace.name.lower():
old_name = namespace.name
new_name = namespace.name.lower()
if dry_run:
print(f" Would normalize namespace: '{old_name}' -> '{new_name}'")
else:
namespace.name = new_name
dbsession.add(namespace)
normalized_count += 1
if normalized_count:
if not dry_run:
dbsession.flush()
print(f"{'Would normalize' if dry_run else 'Normalized'} {normalized_count} namespace name(s) to lowercase")
return normalized_count
def main():
parser = get_arg_parser()
args = parser.parse_args()
setup_logging(args.config)
with bootstrap(args.config) as env, env["request"].tm as tm:
request = env["request"]
dbsession = request.dbsession
# Step 1: Find and merge duplicate namespaces
print("Searching for duplicate namespaces (case-insensitive)...")
ns_duplicates = find_duplicate_namespaces(dbsession)
if ns_duplicates:
print(f"Found {len(ns_duplicates)} namespace name(s) with duplicates:\n")
for name, namespaces in ns_duplicates.items():
# Sort by id to keep the oldest (UUID1 is time-based)
namespaces_sorted = sorted(namespaces, key=lambda ns: ns.id.time)
keep_ns = namespaces_sorted[0]
delete_namespaces = namespaces_sorted[1:]
print(f"Namespace: {name}")
print(f" Keeping: '{keep_ns.name}' (id={keep_ns.id})")
for delete_ns in delete_namespaces:
print(f" Deleting: '{delete_ns.name}' (id={delete_ns.id})")
merge_namespaces(dbsession, keep_ns, delete_ns, dry_run=args.dry_run)
print()
else:
print("No duplicate namespaces found.\n")
# Step 2: Find and merge duplicate URIs
print("Searching for duplicate URIs (case-insensitive)...")
uri_duplicates = find_duplicate_uris(dbsession)
if uri_duplicates:
print(f"Found {len(uri_duplicates)} URI(s) with duplicates:\n")
for uri_lower, uris in uri_duplicates.items():
uris_sorted = sorted(uris, key=lambda u: u.id.time)
keep_uri = uris_sorted[0]
delete_uris = uris_sorted[1:]
print(f"URI: {uri_lower}")
print(f" Keeping: '{keep_uri.data}' (id={keep_uri.id})")
for delete_uri in delete_uris:
print(f" Deleting: '{delete_uri.data}' (id={delete_uri.id})")
merge_uris(dbsession, keep_uri, delete_uri, dry_run=args.dry_run)
print()
else:
print("No duplicate URIs found.\n")
# Step 3: Normalize remaining namespace names and URI hostnames
print("Normalizing namespace names to lowercase...")
normalize_namespace_names(dbsession, dry_run=args.dry_run)
print("\nNormalizing URI hostnames to lowercase...")
normalize_uri_hostnames(dbsession, dry_run=args.dry_run)
if args.dry_run:
print("\nDRY RUN - No changes were made. Run without --dry-run to apply changes.")
else:
print("\nDone. All duplicates have been merged and names normalized.")

93
remarkbox/scripts/modify_namespace.py
View file

@ -12,6 +12,47 @@ from . import base_parser
toggle = lambda x: not x
def delete_namespace_cascade(dbsession, namespace):
"""
Perform cascading deletion of a namespace and all associated data.
Reused by both the CLI script and the web view.
"""
nodes = get_nodes_who_share_roots(dbsession, namespace.roots)
for node in nodes:
if node.events:
for event in node.events:
dbsession.delete(event)
dbsession.delete(node)
for root in namespace.roots:
if root.uri:
dbsession.delete(root.uri)
if root.cache:
dbsession.delete(root.cache)
if root.watchers:
for watcher in root.watchers:
if watcher.unsent_notifications():
for notification in watcher.unsent_notifications():
dbsession.delete(notification)
dbsession.delete(watcher)
if namespace.watchers:
for watcher in namespace.watchers:
if watcher.unsent_notifications():
for notification in watcher.unsent_notifications():
dbsession.delete(notification)
dbsession.delete(watcher)
if namespace.namespace_users:
for nsu in namespace.namespace_users:
dbsession.delete(nsu)
if namespace.oauth_records:
for oauth_record in namespace.oauth_records:
dbsession.delete(oauth_record)
if namespace.namespace_owner_requests:
for nr in namespace.namespace_owner_requests:
dbsession.delete(nr)
dbsession.delete(namespace)
dbsession.flush()
def get_arg_parser():
parser = base_parser("Modify a Namespace.")
parser.add_argument(
@ -98,61 +139,11 @@ def main():
elif args.delete:
nodes = get_nodes_who_share_roots(request.dbsession, namespace.roots)
# delete all nodes.
for node in nodes:
if node.events:
# delete all related NodeEvents.
for event in node.events:
request.dbsession.delete(event)
# delete node.
request.dbsession.delete(node)
for root in namespace.roots:
if root.uri:
# delete related Uri.
request.dbsession.delete(root.uri)
if root.cache:
# delete related NodeCache.
request.dbsession.delete(root.cache)
if root.watchers:
# delete all related NodeEventWatchers.
for watcher in root.watchers:
# delete all unsent NodeEventNotifications.
if watcher.unsent_notifications():
for notification in watcher.unsent_notifications():
request.dbsession.delete(notification)
request.dbsession.delete(watcher)
if namespace.watchers:
for watcher in namespace.watchers:
# delete all unsent NodeEventNotifications.
if watcher.unsent_notifications():
for notification in watcher.unsent_notifications():
request.dbsession.delete(notification)
request.dbsession.delete(watcher)
# disable all related NamespaceUser objects.
if namespace.namespace_users:
for nsu in namespace.namespace_users:
# delete related NamespaceUser.
request.dbsession.delete(nsu)
# delete all related OauthRecords.
if namespace.oauth_records:
for oauth_record in namespace.oauth_records:
request.dbsession.delete(oauth_record)
# finally delete Namespace.
request.dbsession.delete(namespace)
if (
raw_input("*** DANGER: Delete Namespace '{}' ({}) forever? [yes, no]: ".format(namespace.name, namespace.id))
== "yes"
):
request.dbsession.flush()
delete_namespace_cascade(request.dbsession, namespace)
print("Flushed transaction to database, the Namespace was completely destroyed!")
else:
sp.rollback()

43
remarkbox/static/css/common.css
View file

@ -919,3 +919,46 @@ input#namespace-domain {
margin-bottom: 10px;
text-align: center;
}
/* @mention links (T6) */
a.mention {
color: var(--primary);
font-weight: 600;
text-decoration: none;
}
a.mention:hover {
text-decoration: underline;
}
/* Thread title typeahead suggestions (T9) */
.thread-title-suggestions {
border: 1px solid var(--border);
border-top: none;
background: var(--background);
max-height: 200px;
overflow-y: auto;
margin-bottom: 8px;
}
.thread-title-suggestions .suggestions-header {
padding: 4px 8px;
font-size: 0.85em;
color: var(--color-muted);
border-bottom: 1px solid var(--border-faint);
}
.thread-title-suggestions .suggestion-item {
display: block;
padding: 6px 8px;
text-decoration: none;
color: var(--color);
border-bottom: 1px solid var(--border-faint);
}
.thread-title-suggestions .suggestion-item:last-child {
border-bottom: none;
}
.thread-title-suggestions .suggestion-item:hover {
background: var(--background-faint);
}
.thread-title-suggestions .suggestion-meta {
font-size: 0.85em;
color: var(--color-muted);
}

87
remarkbox/static/js/custom.js
View file

@ -118,6 +118,90 @@ function autoGrow(el) {
el.style.height = newHeight + 'px';
}
// Thread title typeahead for duplicate prevention (T9).
var threadSearchTimer = null;
function initThreadTitleTypeahead() {
var titleInput = document.getElementById('thread_title_input');
if (!titleInput) return;
// Create the suggestions container right after the title input.
var suggestionsDiv = document.createElement('div');
suggestionsDiv.id = 'thread-title-suggestions';
suggestionsDiv.className = 'thread-title-suggestions';
suggestionsDiv.style.display = 'none';
titleInput.parentNode.insertBefore(suggestionsDiv, titleInput.nextSibling);
titleInput.addEventListener('input', function() {
var query = titleInput.value.trim();
if (query.length < 2) {
suggestionsDiv.style.display = 'none';
suggestionsDiv.innerHTML = '';
return;
}
if (threadSearchTimer) {
clearTimeout(threadSearchTimer);
}
threadSearchTimer = setTimeout(function() {
searchThreads(query, suggestionsDiv);
}, 400);
});
// Hide suggestions when clicking outside.
document.addEventListener('click', function(e) {
if (e.target !== titleInput && !suggestionsDiv.contains(e.target)) {
suggestionsDiv.style.display = 'none';
}
});
// Show suggestions again on focus if they have content.
titleInput.addEventListener('focus', function() {
if (suggestionsDiv.innerHTML) {
suggestionsDiv.style.display = '';
}
});
}
function searchThreads(query, suggestionsDiv) {
// Derive namespace from the current page URL or a data attribute.
var namespace = document.body.getAttribute('data-namespace') || '';
if (!namespace) return;
var url = '/api/v1/threads/search?q=' + encodeURIComponent(query) +
'&namespace=' + encodeURIComponent(namespace);
fetch(url, {
headers: { 'X-Requested-With': 'XMLHttpRequest' }
})
.then(function(response) { return response.json(); })
.then(function(data) {
if (!data.threads || data.threads.length === 0) {
suggestionsDiv.style.display = 'none';
suggestionsDiv.innerHTML = '';
return;
}
var html = '<div class="suggestions-header">Existing threads:</div>';
data.threads.forEach(function(thread) {
html += '<a href="' + thread.path + '" class="suggestion-item">' +
escapeHtml(thread.title) +
'<span class="suggestion-meta"> &mdash; ' + thread.created_ago + '</span>' +
'</a>';
});
suggestionsDiv.innerHTML = html;
suggestionsDiv.style.display = '';
})
.catch(function() {
suggestionsDiv.style.display = 'none';
});
}
function escapeHtml(text) {
var div = document.createElement('div');
div.appendChild(document.createTextNode(text));
return div.innerHTML;
}
// Initialize on DOM ready
document.addEventListener('DOMContentLoaded', function() {
@ -166,6 +250,9 @@ document.addEventListener('DOMContentLoaded', function() {
}
}
// Initialize thread title typeahead for duplicate prevention (T9).
initThreadTitleTypeahead();
});
function sendVote(post_id, direction) {

57
remarkbox/static/js/push-sw.js Normal file
View file

@ -0,0 +1,57 @@
/**
* Remarkbox Push Notification Service Worker
*
* Handles incoming push events and notification clicks.
* Registered by the push subscription code on the client side.
*/
/* eslint-env serviceworker */
self.addEventListener("push", function (event) {
var data = {};
if (event.data) {
try {
data = event.data.json();
} catch (e) {
data = { title: "Remarkbox", body: event.data.text() };
}
}
var title = data.title || "Remarkbox";
var options = {
body: data.body || "You have a new notification.",
icon: data.icon || "/static/img/remarkbox-icon.png",
badge: data.badge || "/static/img/remarkbox-icon.png",
tag: data.tag || "remarkbox-notification",
data: {
url: data.url || "/",
},
};
event.waitUntil(self.registration.showNotification(title, options));
});
self.addEventListener("notificationclick", function (event) {
event.notification.close();
var url = "/";
if (event.notification.data && event.notification.data.url) {
url = event.notification.data.url;
}
event.waitUntil(
clients.matchAll({ type: "window", includeUncontrolled: true }).then(function (clientList) {
// Focus existing window if one is open.
for (var i = 0; i < clientList.length; i++) {
var client = clientList[i];
if (client.url === url && "focus" in client) {
return client.focus();
}
}
// Otherwise open a new window.
if (clients.openWindow) {
return clients.openWindow(url);
}
})
);
});

2
remarkbox/templates/base.j2
View file

@ -19,7 +19,7 @@
</head>
<body>
<body{% if request.namespace %} data-namespace="{{ request.namespace.name }}"{% endif %}>
<noscript>
<style>

37
remarkbox/templates/confirm-delete-account.j2 Normal file
View file

@ -0,0 +1,37 @@
{% extends request.base_template -%}
{% block title %}{{ the_title }} | {{ request.domain }}{%- endblock -%}
{% block content -%}
<h4>{{ the_title }}</h4>
<p>
This action is <b>permanent and cannot be undone</b>. Deleting your account will:
</p>
<ul>
<li>Anonymize all your comments (your name will be removed)</li>
<li>Delete your profile, notification settings, and watched threads</li>
<li>Log you out of all sessions</li>
</ul>
<p>
Your comments will remain visible but will no longer be associated with your account.
</p>
<br>
<form method="post" action="{{ request.link_prefix }}/u/delete-account">
<label>Type <b>DELETE</b> to confirm:</label>
<input type="text" name="confirm-delete" id="confirm-delete" class="common-text-input" placeholder="DELETE" autocomplete="off" required>
<br>
<br>
{% include 'snippets/csrf.j2' %}
{% set submit_button_value = 'permanently delete my account' %}
{% set submit_button_classes = 'button-right' %}
{% include 'snippets/submit.j2' %}
</form>
<br>
<a href="{{ request.link_prefix }}/u/settings" class="button button-right">cancel</a>
{%- endblock -%}

38
remarkbox/templates/confirm-delete-namespace.j2 Normal file
View file

@ -0,0 +1,38 @@
{% extends request.base_template -%}
{% block title %}{{ the_title }} | {{ request.domain }}{%- endblock -%}
{% block content -%}
<h4>{{ the_title }}</h4>
<p>
This action is <b>permanent and cannot be undone</b>. Deleting this namespace will:
</p>
<ul>
<li>Delete all threads and comments in <b>{{ request.namespace.name }}</b></li>
<li>Delete all watchers and notification settings</li>
<li>Remove all moderator and owner associations</li>
<li>Delete all OAuth integrations (e.g. Slack)</li>
</ul>
<p>
This will permanently destroy all data associated with the <b>{{ request.namespace.name }}</b> namespace.
</p>
<br>
<form method="post" action="{{ request.link_prefix }}/ns/{{ request.namespace.name }}/delete">
<label>Type <b>{{ request.namespace.name }}</b> to confirm:</label>
<input type="text" name="confirm-delete" id="confirm-delete" class="common-text-input" placeholder="{{ request.namespace.name }}" autocomplete="off" required>
<br>
<br>
{% include 'snippets/csrf.j2' %}
{% set submit_button_value = 'permanently delete namespace' %}
{% set submit_button_classes = 'button-right' %}
{% include 'snippets/submit.j2' %}
</form>
<br>
<a href="{{ request.link_prefix }}/ns/{{ request.namespace.name }}/settings" class="button button-right">cancel</a>
{%- endblock -%}

4
remarkbox/templates/home.j2
View file

@ -51,13 +51,13 @@
{%- if request.user.authenticated %}
{%- if request.user.unverified_nodes.count() > 0 %}
{%- if request.user.unverified_nodes().count() > 0 %}
<br>
<section class="well">
<br>
<p>
<b>You have <a href="{{ request.link_prefix }}/u/{{ request.user.name }}?pending">unverified comments ({{ request.user.unverified_nodes.count() }})</a> pending review.</b>
<b>You have <a href="{{ request.link_prefix }}/u/{{ request.user.name }}?pending">unverified comments ({{ request.user.unverified_nodes().count() }})</a> pending review.</b>
</p>
<p>

46
remarkbox/templates/namespace-settings.j2
View file

@ -83,6 +83,39 @@ If checked, allow JSON API access to this namespace for agents and integrations.
<br>
<br>
<hr>
<h4>Button Text and Labels</h4>
<label>Submit button text:</label>
<input type="text" name="submit-button-text" id="submit-button-text" value="{% if request.namespace.submit_button_text %}{{ request.namespace.submit_button_text }}{% endif %}" class="common-text-input" placeholder="default: 'save message'"></input>
<br>
<br>
<label>Comment label (singular):</label>
<input type="text" name="comment-label-singular" id="comment-label-singular" value="{% if request.namespace.comment_label_singular %}{{ request.namespace.comment_label_singular }}{% endif %}" class="common-text-input" placeholder="default: 'remark'"></input>
<br>
<br>
<label>Comment label (plural):</label>
<input type="text" name="comment-label-plural" id="comment-label-plural" value="{% if request.namespace.comment_label_plural %}{{ request.namespace.comment_label_plural }}{% endif %}" class="common-text-input" placeholder="default: 'remarks'"></input>
<br>
<br>
<hr>
<h4>Nesting Depth</h4>
<label>Max nesting depth:</label>
<input type="number" name="max-nesting-depth" id="max-nesting-depth" value="{% if request.namespace.max_nesting_depth is not none %}{{ request.namespace.max_nesting_depth }}{% endif %}" class="common-text-input" placeholder="default: unlimited" min="1"></input>
Leave blank for unlimited nesting.
<br>
<br>
<label>Collapse depth:</label>
<input type="number" name="collapse-depth" id="collapse-depth" value="{% if request.namespace.collapse_depth is not none %}{{ request.namespace.collapse_depth }}{% endif %}" class="common-text-input" placeholder="default: never collapse" min="1"></input>
Show a "load more" button at this depth instead of displaying replies inline. Leave blank to never collapse.
<br>
<br>
{% set submit_button_classes = 'button-right green-button' %}
{% set submit_button_value = 'save changes' %}
{% include 'snippets/submit.j2' %}
@ -305,4 +338,17 @@ Import comments and threads using the <a href="https://github.com/russellballest
<br>
<a href="{{ request.link_prefix }}/ns/{{ request.namespace.name }}/import-comments" class="button">Import Comments</a>
<br>
<br>
<hr>
<br>
<h4>Danger Zone</h4>
<label>Delete Namespace</label>
<p>Permanently delete this namespace and all associated threads, comments, and settings. This action cannot be undone.</p>
<a href="{{ request.link_prefix }}/ns/{{ request.namespace.name }}/delete" class="button">Delete Namespace</a>
{%- endblock -%}

8
remarkbox/templates/show-count.j2
View file

@ -1,11 +1,13 @@
{% set number = request.node.stats["root"]["visible_count"] %}
{% set label_singular = request.namespace.comment_label_singular or 'remark' %}
{% set label_plural = request.namespace.comment_label_plural or 'remarks' %}
{% if number == 0 %}
{% set msg = "No remarks" %}
{% set msg = "No " ~ label_plural %}
{% elif number == 1 %}
{% set msg = "1 remark" %}
{% set msg = "1 " ~ label_singular %}
{% else %}
{% set msg = number ~ " remarks" %}
{% set msg = number ~ " " ~ label_plural %}
{% endif %}
{% if request.node.uri %}

26
remarkbox/templates/show-node.j2
View file

@ -109,8 +109,9 @@
{%- set children_ids = request.node_graph[parent.id] -%}
{%- endif -%}
{%- set effective_collapse_depth = request.namespace.collapse_depth -%}
{%- set display_class = '' -%}
{%- if loop.depth > 4 -%}
{%- if effective_collapse_depth is not none and loop.depth > effective_collapse_depth -%}
{%- set display_class = 'hidden-on-tablet' -%}
{%- endif -%}
@ -149,9 +150,9 @@
{% endif -%}
</div>
{%- if loop.depth == 4 and children_ids -%}
{%- if effective_collapse_depth is not none and loop.depth == effective_collapse_depth and children_ids -%}
<br>
<a href="{{ request.link_prefix }}/{{ parent.id }}#{{ parent.id }}" class="load-more shown-on-tablet">load more <span style="color: #292929; font-weight: normal;">({{children_ids | length}} remarks)</span></a>
<a href="{{ request.link_prefix }}/{{ parent.id }}#{{ parent.id }}" class="load-more shown-on-tablet">load more <span style="color: #292929; font-weight: normal;">({{children_ids | length}} {{ request.namespace.comment_label_plural or 'remarks' }})</span></a>
{% endif %}
<div id="edit-box-{{ parent.id }}" class="edit-box-div">
@ -195,4 +196,23 @@
{% endif %}
</section>
{% if request.webmentions %}
<section class="webmentions-section">
<hr class="comments-divider">
<h3 class="comments-header">Webmentions</h3>
{% for wm in request.webmentions %}
<div class="webmention">
<span class="webmention-author">
{% if wm.author_url %}<a href="{{ wm.author_url }}" rel="nofollow" target="_blank">{{ wm.author_name or 'Someone' }}</a>{% else %}{{ wm.author_name or 'Someone' }}{% endif %}
</span>
mentioned this on
<a href="{{ wm.source }}" rel="nofollow" target="_blank">their site</a>
{% if wm.content %}
<p class="webmention-content">{{ wm.content }}</p>
{% endif %}
</div>
{% endfor %}
</section>
{% endif %}
{% endblock %}

4
remarkbox/templates/snippets/forms.j2
View file

@ -23,7 +23,7 @@
required></textarea>
{% set submit_button_classes = 'green-button button-right' %}
{% set submit_button_value = 'save message' %}
{% set submit_button_value = request.namespace.submit_button_text or 'save message' %}
{% include 'submit.j2' %}
<details class="preview-details js-only" open>
@ -67,7 +67,7 @@
{% include 'csrf.j2' %}
{% set submit_button_classes = 'green-button button-right' %}
{% set submit_button_value = 'save message' %}
{% set submit_button_value = request.namespace.submit_button_text or 'save message' %}
{% include 'submit.j2' %}
</form>

4
remarkbox/templates/snippets/snippets.j2
View file

@ -123,8 +123,8 @@
{% endmacro %}
{% macro node_list_sub_menu(user, divider="|") %}
{% set user_unverified_node_count = user.unverified_nodes.count() %}
{% set user_disabled_node_count = user.disabled_nodes.count() %}
{% set user_unverified_node_count = user.unverified_nodes(request.namespace).count() %}
{% set user_disabled_node_count = user.disabled_nodes(request.namespace).count() %}
<a href="{{ request.link_prefix }}/u/{{ user.name }}?verified">my nodes</a>
{{ divider }}
<a href="{{ request.link_prefix }}/u/{{ user.name }}?pending">my unverified nodes ({{ user_unverified_node_count }})</a>

115
remarkbox/templates/user-settings.j2
View file

@ -89,6 +89,25 @@ By default, when I watch a thread, notify me
</select> of changes.
<br/>
<br/>
<label>Notification Delivery</label>
How would you like to receive notifications?
<select name="notification-preference">
<option value="email" {% if request.user.notification_preference == "email" %}selected{% endif %}>Email only</option>
<option value="push" {% if request.user.notification_preference == "push" %}selected{% endif %}>Browser push only</option>
<option value="both" {% if request.user.notification_preference == "both" %}selected{% endif %}>Email and browser push</option>
<option value="none" {% if request.user.notification_preference == "none" %}selected{% endif %}>None</option>
</select>
<div id="push-controls" style="margin-top: 8px;">
<button type="button" id="push-subscribe-btn" class="button button-small" style="display:none;" onclick="subscribePush()">Enable Push Notifications</button>
<button type="button" id="push-unsubscribe-btn" class="button button-small" style="display:none;" onclick="unsubscribePush()">Disable Push Notifications</button>
<span id="push-status"></span>
</div>
<br/>
<br/>
@ -104,6 +123,23 @@ View and adjust your <a href="{{ request.link_prefix }}/u/watching">Namespace no
</form>
<br>
<hr>
<br>
<h4>Your Data</h4>
<label>Download My Data</label>
<p>Export all your comments and profile information as a JSON file.</p>
<a href="{{ request.link_prefix }}/u/export-data" class="button">Download My Data</a>
<br>
<br>
<label>Delete My Account</label>
<p>Permanently delete your account and anonymize all your comments. This action cannot be undone.</p>
<a href="{{ request.link_prefix }}/u/delete-account" class="button">Delete My Account</a>
<script>
function previewTheme(mode) {
const html = document.documentElement;
@ -123,6 +159,85 @@ function previewTheme(mode) {
}
}
}
// --- Push notification support ---
function urlBase64ToUint8Array(base64String) {
var padding = '='.repeat((4 - base64String.length % 4) % 4);
var base64 = (base64String + padding).replace(/\-/g, '+').replace(/_/g, '/');
var rawData = window.atob(base64);
var outputArray = new Uint8Array(rawData.length);
for (var i = 0; i < rawData.length; ++i) {
outputArray[i] = rawData.charCodeAt(i);
}
return outputArray;
}
function initPush() {
if (!('serviceWorker' in navigator) || !('PushManager' in window)) {
document.getElementById('push-status').textContent = 'Push not supported in this browser.';
return;
}
navigator.serviceWorker.register('/static/js/push-sw.js').then(function(reg) {
reg.pushManager.getSubscription().then(function(sub) {
if (sub) {
document.getElementById('push-unsubscribe-btn').style.display = 'inline-block';
document.getElementById('push-status').textContent = 'Push notifications are active.';
} else {
document.getElementById('push-subscribe-btn').style.display = 'inline-block';
}
});
});
}
function subscribePush() {
fetch('/push/vapid-key').then(function(r){ return r.json(); }).then(function(data) {
if (!data.available) {
document.getElementById('push-status').textContent = data.reason || 'Push not available.';
return;
}
navigator.serviceWorker.ready.then(function(reg) {
reg.pushManager.subscribe({
userVisibleOnly: true,
applicationServerKey: urlBase64ToUint8Array(data.public_key)
}).then(function(sub) {
var subJSON = sub.toJSON();
fetch('/push/subscribe', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({subscription: subJSON})
}).then(function(r){ return r.json(); }).then(function(result) {
document.getElementById('push-subscribe-btn').style.display = 'none';
document.getElementById('push-unsubscribe-btn').style.display = 'inline-block';
document.getElementById('push-status').textContent = 'Push notifications enabled!';
});
});
});
});
}
function unsubscribePush() {
navigator.serviceWorker.ready.then(function(reg) {
reg.pushManager.getSubscription().then(function(sub) {
if (sub) {
var endpoint = sub.endpoint;
sub.unsubscribe().then(function() {
fetch('/push/unsubscribe', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({endpoint: endpoint})
}).then(function() {
document.getElementById('push-unsubscribe-btn').style.display = 'none';
document.getElementById('push-subscribe-btn').style.display = 'inline-block';
document.getElementById('push-status').textContent = 'Push notifications disabled.';
});
});
}
});
});
}
// Init push UI on page load.
initPush();
</script>
{%- endblock -%}

580
remarkbox/tests/test_api_views.py
View file

@ -304,6 +304,13 @@ class TestAPIAnonymousPosting(APIFunctionalTests):
self.assertEqual(res.json["thread"]["id"], node_id)
self.assertEqual(len(res.json["replies"]), 1)
self.assertEqual(res.json["replies"][0]["author"]["name"], "ReplyBot")
# Pagination metadata
self.assertEqual(res.json["total_replies"], 1)
self.assertEqual(res.json["page"], 1)
self.assertIn("limit", res.json)
self.assertIn("offset", res.json)
self.assertIn("has_more", res.json)
self.assertFalse(res.json["has_more"])
def test_get_nonexistent_thread(self):
res = self.testapp.get(
@ -371,6 +378,206 @@ class TestAPIAnonymousPosting(APIFunctionalTests):
self.assertIn("page_size", res.json)
class TestAPIThreadDetailPagination(APIFunctionalTests):
"""Functional tests for thread detail pagination."""
@classmethod
def setUpClass(cls):
try:
APIFunctionalTests.setUpClass.im_func(cls)
except AttributeError:
APIFunctionalTests.setUpClass.__func__(cls)
def setUp(self):
ns = get_or_create_namespace(self.dbsession, "api-paginate.example.com")
ns.allow_anonymous = True
self.dbsession.add(ns)
self.dbsession.flush()
self.namespace_name = str(ns.name)
self.namespace_id = ns.id
self.tm.commit()
# Create a thread with 5 replies
res = self.testapp.post_json(
"/api/v1/threads",
{
"namespace": self.namespace_name,
"title": "Pagination Thread",
"data": "Root post",
"anonymous_name": "Bot",
},
expect_errors=True,
)
self.thread_id = res.json["node"]["id"]
for i in range(5):
self.testapp.post_json(
"/api/v1/threads/{}/replies".format(self.thread_id),
{"data": "Reply {}".format(i), "anonymous_name": "Bot{}".format(i)},
expect_errors=True,
)
def tearDown(self):
super(TestAPIThreadDetailPagination, self).tearDown()
self.dbsession.query(UserSurrogate).filter(
UserSurrogate.namespace_id == self.namespace_id
).delete(synchronize_session=False)
self.dbsession.query(Node).filter(
Node.namespace_id == self.namespace_id
).delete(synchronize_session=False)
self.dbsession.flush()
self.tm.commit()
def test_default_pagination(self):
"""Default request returns all replies with pagination metadata."""
res = self.testapp.get(
"/api/v1/threads/{}".format(self.thread_id),
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertEqual(len(res.json["replies"]), 5)
self.assertEqual(res.json["total_replies"], 5)
self.assertEqual(res.json["page"], 1)
self.assertEqual(res.json["offset"], 0)
self.assertEqual(res.json["limit"], 100)
self.assertFalse(res.json["has_more"])
def test_limit_param(self):
"""Limit restricts the number of returned replies."""
res = self.testapp.get(
"/api/v1/threads/{}".format(self.thread_id),
{"limit": "2"},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertEqual(len(res.json["replies"]), 2)
self.assertEqual(res.json["total_replies"], 5)
self.assertEqual(res.json["limit"], 2)
self.assertTrue(res.json["has_more"])
def test_offset_param(self):
"""Offset skips replies."""
res = self.testapp.get(
"/api/v1/threads/{}".format(self.thread_id),
{"limit": "2", "offset": "3"},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertEqual(len(res.json["replies"]), 2)
self.assertEqual(res.json["total_replies"], 5)
self.assertEqual(res.json["offset"], 3)
self.assertFalse(res.json["has_more"])
def test_offset_beyond_total(self):
"""Offset past the end returns empty replies."""
res = self.testapp.get(
"/api/v1/threads/{}".format(self.thread_id),
{"limit": "10", "offset": "100"},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertEqual(len(res.json["replies"]), 0)
self.assertEqual(res.json["total_replies"], 5)
self.assertFalse(res.json["has_more"])
def test_page_number_calculation(self):
"""Page number is calculated from offset and limit."""
res = self.testapp.get(
"/api/v1/threads/{}".format(self.thread_id),
{"limit": "2", "offset": "2"},
expect_errors=True,
)
self.assertEqual(res.json["page"], 2)
def test_replies_exclude_root(self):
"""Root node is never included in the replies list."""
res = self.testapp.get(
"/api/v1/threads/{}".format(self.thread_id),
expect_errors=True,
)
reply_ids = [r["id"] for r in res.json["replies"]]
self.assertNotIn(self.thread_id, reply_ids)
def test_disabled_replies_filtered(self):
"""Disabled replies are excluded from results."""
root_uuid = id_to_uuid(self.thread_id)
reply_node = self.dbsession.query(Node).filter(
Node.root_id == root_uuid,
Node.id != root_uuid,
).first()
self.assertIsNotNone(reply_node)
reply_node.disabled = True
self.dbsession.add(reply_node)
self.dbsession.flush()
self.tm.commit()
res = self.testapp.get(
"/api/v1/threads/{}".format(self.thread_id),
expect_errors=True,
)
self.assertEqual(res.json["total_replies"], 4)
self.assertEqual(len(res.json["replies"]), 4)
def test_limit_clamped_to_max(self):
"""Limit is clamped to 500 maximum."""
res = self.testapp.get(
"/api/v1/threads/{}".format(self.thread_id),
{"limit": "9999"},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertEqual(res.json["limit"], 500)
def test_invalid_limit_uses_default(self):
"""Non-integer limit falls back to default 100."""
res = self.testapp.get(
"/api/v1/threads/{}".format(self.thread_id),
{"limit": "abc"},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertEqual(res.json["limit"], 100)
def test_invalid_offset_uses_default(self):
"""Non-integer offset falls back to default 0."""
res = self.testapp.get(
"/api/v1/threads/{}".format(self.thread_id),
{"offset": "abc"},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertEqual(res.json["offset"], 0)
def test_zero_replies_thread(self):
"""Thread with zero replies returns valid empty pagination response."""
# Create a thread with no replies
res = self.testapp.post_json(
"/api/v1/threads",
{
"namespace": self.namespace_name,
"title": "Empty Thread",
"data": "Thread with no replies",
"anonymous_name": "Loner",
},
expect_errors=True,
)
empty_thread_id = res.json["node"]["id"]
res = self.testapp.get(
"/api/v1/threads/{}".format(empty_thread_id),
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertEqual(res.json["total_replies"], 0)
self.assertEqual(len(res.json["replies"]), 0)
self.assertEqual(res.json["page"], 1)
self.assertEqual(res.json["offset"], 0)
self.assertEqual(res.json["limit"], 100)
self.assertFalse(res.json["has_more"])
# Thread data should still be present
self.assertIn("thread", res.json)
self.assertIn("namespace", res.json)
class TestAPIOTPAuthentication(APIFunctionalTests):
"""Functional tests for the OTP auth flow via API."""
@ -874,3 +1081,376 @@ class TestAPIClientDownload(APIFunctionalTests):
self.assertIn("def reply", res.text)
self.assertIn("def login", res.text)
self.assertIn("def verify", res.text)
# ---------------------------------------------------------------------------
# T8: Nesting depth enforcement via API
# ---------------------------------------------------------------------------
class TestAPIMaxNestingDepth(APIFunctionalTests):
"""API tests for T8: max nesting depth enforcement."""
@classmethod
def setUpClass(cls):
try:
APIFunctionalTests.setUpClass.im_func(cls)
except AttributeError:
APIFunctionalTests.setUpClass.__func__(cls)
def setUp(self):
from remarkbox.models.namespace import get_or_create_namespace
ns = get_or_create_namespace(self.dbsession, "api-depth-test.example.com")
ns.allow_anonymous = True
# Set max nesting depth to 2 (root=0, child=1, grandchild=2, no deeper)
ns.max_nesting_depth = 2
self.dbsession.add(ns)
self.dbsession.flush()
self.namespace_name = str(ns.name)
self.namespace_id = ns.id
self.tm.commit()
def tearDown(self):
super(TestAPIMaxNestingDepth, self).tearDown()
self.dbsession.query(UserSurrogate).filter(
UserSurrogate.namespace_id == self.namespace_id
).delete(synchronize_session=False)
self.dbsession.query(Node).filter(
Node.namespace_id == self.namespace_id
).delete(synchronize_session=False)
self.dbsession.flush()
self.tm.commit()
def test_reply_within_max_depth_succeeds(self):
"""Reply within max depth (depth 0 -> 1) should succeed."""
# Create root thread (depth 0)
create_res = self.testapp.post_json(
"/api/v1/threads",
{
"namespace": self.namespace_name,
"title": "Depth Test Thread",
"data": "Root post",
"anonymous_name": "Bot",
},
expect_errors=True,
)
self.assertEqual(create_res.status_int, 201)
root_id = create_res.json["node"]["id"]
# Reply to root (creating depth 1 child) - should succeed
reply_res = self.testapp.post_json(
"/api/v1/threads/{}/replies".format(root_id),
{"data": "Depth 1 reply", "anonymous_name": "Bot"},
expect_errors=True,
)
self.assertEqual(reply_res.status_int, 201)
def test_reply_at_max_depth_succeeds(self):
"""Reply creating a node at exactly max depth should succeed."""
# Create root thread (depth 0)
create_res = self.testapp.post_json(
"/api/v1/threads",
{
"namespace": self.namespace_name,
"title": "Depth Boundary Thread",
"data": "Root post",
"anonymous_name": "Bot",
},
expect_errors=True,
)
root_id = create_res.json["node"]["id"]
# Reply to root (depth 1 child) - parent.depth=0 < max=2, OK
reply_res1 = self.testapp.post_json(
"/api/v1/threads/{}/replies".format(root_id),
{"data": "Depth 1 reply", "anonymous_name": "Bot"},
expect_errors=True,
)
self.assertEqual(reply_res1.status_int, 201)
child_id = reply_res1.json["node"]["id"]
# Reply to depth-1 child (creating depth 2) - parent.depth=1 < max=2, OK
reply_res2 = self.testapp.post_json(
"/api/v1/threads/{}/replies".format(child_id),
{"data": "Depth 2 reply", "anonymous_name": "Bot"},
expect_errors=True,
)
self.assertEqual(reply_res2.status_int, 201)
def test_reply_beyond_max_depth_returns_403(self):
"""Reply beyond max depth should return 403."""
# Create root thread (depth 0)
create_res = self.testapp.post_json(
"/api/v1/threads",
{
"namespace": self.namespace_name,
"title": "Too Deep Thread",
"data": "Root post",
"anonymous_name": "Bot",
},
expect_errors=True,
)
root_id = create_res.json["node"]["id"]
# Reply to root (depth 1)
reply_res1 = self.testapp.post_json(
"/api/v1/threads/{}/replies".format(root_id),
{"data": "Depth 1 reply", "anonymous_name": "Bot"},
expect_errors=True,
)
child_id = reply_res1.json["node"]["id"]
# Reply to depth-1 (creating depth 2)
reply_res2 = self.testapp.post_json(
"/api/v1/threads/{}/replies".format(child_id),
{"data": "Depth 2 reply", "anonymous_name": "Bot"},
expect_errors=True,
)
grandchild_id = reply_res2.json["node"]["id"]
# Reply to depth-2 (would create depth 3) - parent.depth=2 >= max=2, REJECT
reply_res3 = self.testapp.post_json(
"/api/v1/threads/{}/replies".format(grandchild_id),
{"data": "Too deep reply", "anonymous_name": "Bot"},
expect_errors=True,
)
self.assertEqual(reply_res3.status_int, 403)
self.assertIn("depth", reply_res3.json["error"].lower())
class TestAPINoMaxDepthAllowsUnlimited(APIFunctionalTests):
"""API test: NULL max_nesting_depth allows unlimited nesting."""
@classmethod
def setUpClass(cls):
try:
APIFunctionalTests.setUpClass.im_func(cls)
except AttributeError:
APIFunctionalTests.setUpClass.__func__(cls)
def setUp(self):
from remarkbox.models.namespace import get_or_create_namespace
ns = get_or_create_namespace(self.dbsession, "api-nolimit-depth.example.com")
ns.allow_anonymous = True
# max_nesting_depth is NULL by default = unlimited
self.dbsession.add(ns)
self.dbsession.flush()
self.namespace_name = str(ns.name)
self.namespace_id = ns.id
self.tm.commit()
def tearDown(self):
super(TestAPINoMaxDepthAllowsUnlimited, self).tearDown()
self.dbsession.query(UserSurrogate).filter(
UserSurrogate.namespace_id == self.namespace_id
).delete(synchronize_session=False)
self.dbsession.query(Node).filter(
Node.namespace_id == self.namespace_id
).delete(synchronize_session=False)
self.dbsession.flush()
self.tm.commit()
def test_deep_nesting_allowed_when_no_limit(self):
"""With NULL max_nesting_depth, deep nesting is allowed."""
create_res = self.testapp.post_json(
"/api/v1/threads",
{
"namespace": self.namespace_name,
"title": "Unlimited Depth Thread",
"data": "Root post",
"anonymous_name": "Bot",
},
expect_errors=True,
)
self.assertEqual(create_res.status_int, 201)
current_id = create_res.json["node"]["id"]
# Nest 5 levels deep - all should succeed
for i in range(5):
reply_res = self.testapp.post_json(
"/api/v1/threads/{}/replies".format(current_id),
{"data": "Reply at depth {}".format(i + 1), "anonymous_name": "Bot"},
expect_errors=True,
)
self.assertEqual(reply_res.status_int, 201, "Reply at depth {} failed".format(i + 1))
current_id = reply_res.json["node"]["id"]
# ---------------------------------------------------------------------------
# T9: Thread search API
# ---------------------------------------------------------------------------
class TestAPISearchThreads(APIFunctionalTests):
"""API tests for T9: duplicate thread prevention (AJAX search)."""
@classmethod
def setUpClass(cls):
try:
APIFunctionalTests.setUpClass.im_func(cls)
except AttributeError:
APIFunctionalTests.setUpClass.__func__(cls)
def setUp(self):
from remarkbox.models.namespace import get_or_create_namespace
ns = get_or_create_namespace(self.dbsession, "api-search-test.example.com")
ns.allow_anonymous = True
self.dbsession.add(ns)
self.dbsession.flush()
self.namespace_name = str(ns.name)
self.namespace_id = ns.id
self.tm.commit()
# Create some threads for searching
for title in [
"How to deploy Remarkbox",
"How to customize themes",
"How to moderate comments",
"Getting started guide",
"FAQ about pricing",
]:
self.testapp.post_json(
"/api/v1/threads",
{
"namespace": self.namespace_name,
"title": title,
"data": "Content for {}".format(title),
"anonymous_name": "Bot",
},
expect_errors=True,
)
def tearDown(self):
super(TestAPISearchThreads, self).tearDown()
self.dbsession.query(UserSurrogate).filter(
UserSurrogate.namespace_id == self.namespace_id
).delete(synchronize_session=False)
self.dbsession.query(Node).filter(
Node.namespace_id == self.namespace_id
).delete(synchronize_session=False)
self.dbsession.flush()
self.tm.commit()
def test_search_returns_matching_threads(self):
"""Search with matching query returns relevant threads."""
res = self.testapp.get(
"/api/v1/threads/search",
{"q": "How to", "namespace": self.namespace_name},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertIn("threads", res.json)
titles = [t["title"] for t in res.json["threads"]]
self.assertTrue(any("How to" in t for t in titles))
# Should match the "How to" threads
self.assertGreaterEqual(len(res.json["threads"]), 2)
def test_search_no_matches_returns_empty(self):
"""Search with no matching query returns empty list."""
res = self.testapp.get(
"/api/v1/threads/search",
{"q": "zzzznonexistent", "namespace": self.namespace_name},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertEqual(res.json["threads"], [])
def test_search_without_namespace_returns_error(self):
"""Search without namespace param returns 400."""
res = self.testapp.get(
"/api/v1/threads/search",
{"q": "test"},
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
self.assertIn("namespace", res.json["error"])
def test_search_short_query_returns_empty(self):
"""Search with query shorter than 2 characters returns empty list."""
res = self.testapp.get(
"/api/v1/threads/search",
{"q": "H", "namespace": self.namespace_name},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertEqual(res.json["threads"], [])
def test_search_empty_query_returns_empty(self):
"""Search with empty query returns empty list."""
res = self.testapp.get(
"/api/v1/threads/search",
{"q": "", "namespace": self.namespace_name},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertEqual(res.json["threads"], [])
def test_search_results_limited_to_10(self):
"""Search results are limited to at most 10."""
# Create 12 threads all starting with "Limit test"
for i in range(12):
self.testapp.post_json(
"/api/v1/threads",
{
"namespace": self.namespace_name,
"title": "Limit test thread number {}".format(i),
"data": "Content",
"anonymous_name": "Bot",
},
expect_errors=True,
)
res = self.testapp.get(
"/api/v1/threads/search",
{"q": "Limit test", "namespace": self.namespace_name},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertLessEqual(len(res.json["threads"]), 10)
def test_search_sql_injection_safe(self):
"""Special SQL characters in query are handled safely."""
# These should not cause 500 errors
for dangerous_query in [
"'; DROP TABLE rb_node; --",
"%_%_%",
"test' OR '1'='1",
"test%",
"test_",
]:
res = self.testapp.get(
"/api/v1/threads/search",
{"q": dangerous_query, "namespace": self.namespace_name},
expect_errors=True,
)
# Should return 200 with empty or non-empty results, never 500
self.assertIn(res.status_int, [200])
self.assertIn("threads", res.json)
def test_search_results_include_expected_fields(self):
"""Each search result includes id, title, and path."""
res = self.testapp.get(
"/api/v1/threads/search",
{"q": "Getting", "namespace": self.namespace_name},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertGreater(len(res.json["threads"]), 0)
thread = res.json["threads"][0]
self.assertIn("id", thread)
self.assertIn("title", thread)
self.assertIn("path", thread)
def test_search_case_insensitive(self):
"""Search is case-insensitive (ilike)."""
res = self.testapp.get(
"/api/v1/threads/search",
{"q": "how to", "namespace": self.namespace_name},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
# "how to" should match "How to ..." threads
self.assertGreater(len(res.json["threads"]), 0)

460
remarkbox/tests/test_models.py
View file

@ -360,3 +360,463 @@ class TestIntegration(unittest.TestCase):
self.assertEqual(new_child.parent, self.parent)
self.assertEqual(new_child.parent.data, "taco")
self.assertEqual(new_child.parent.user.email, "russell@ballestrini.net")
# ---------------------------------------------------------------------------
# T1: Case-insensitive URI and Namespace lookups (unit tests)
# ---------------------------------------------------------------------------
class TestCaseInsensitiveUri(unittest.TestCase):
"""
Regression tests for T1: case-insensitive URI and namespace lookups.
When creating a node by URI, the namespace is extracted from the hostname.
The namespace lookup normalises the domain to lowercase during the setup
step (setup_namespace view lowercases the domain).
"""
@mock.patch("remarkbox.models.namespace.get_namespace_by_name", mock_always_none)
@mock.patch("remarkbox.models.uri.get_uri_by_uri", mock_always_none)
def test_uri_data_is_stored(self):
"""get_or_create_node_by_uri stores the URI and extracts hostname."""
dbsession = mock.MagicMock()
node = get_or_create_node_by_uri(
dbsession, "http://example.com/SomePath"
)
self.assertTrue(node.uri)
# The URI data is stored.
self.assertEqual(node.uri.data, "http://example.com/SomePath")
@mock.patch("remarkbox.models.namespace.get_namespace_by_name", mock_always_none)
@mock.patch("remarkbox.models.uri.get_uri_by_uri", mock_always_none)
def test_uri_preserves_path_case(self):
"""URI path case is preserved even when hostname is lowercase."""
dbsession = mock.MagicMock()
node = get_or_create_node_by_uri(
dbsession, "http://example.com/My-Blog-Post"
)
self.assertTrue(node.uri)
self.assertIn("/My-Blog-Post", node.uri.data)
@mock.patch("remarkbox.models.namespace.get_namespace_by_name", mock_always_none)
@mock.patch("remarkbox.models.uri.get_uri_by_uri", mock_always_none)
def test_uri_strips_fragment(self):
"""Fragment (#section) is stripped from URIs before storage."""
dbsession = mock.MagicMock()
node = get_or_create_node_by_uri(
dbsession, "http://example.com/page#section"
)
self.assertTrue(node.uri)
self.assertNotIn("#", node.uri.data)
self.assertEqual(node.uri.data, "http://example.com/page")
class TestCaseInsensitiveNamespace(unittest.TestCase):
"""
Regression tests for T1: case-insensitive Namespace lookups.
Namespace names should be compared case-insensitively. The setup_namespace
view lowercases domain names before creation.
"""
def test_namespace_name_stored_as_given(self):
"""Namespace stores the name as given."""
ns = Namespace("Example.Com")
self.assertEqual(ns.name, "Example.Com")
def test_namespace_name_lowercase(self):
"""Namespace with lowercase name is stored correctly."""
ns = Namespace("example.com")
self.assertEqual(ns.name, "example.com")
# ---------------------------------------------------------------------------
# T3: GDPR/CCPA - export_user_data structure (unit tests)
# ---------------------------------------------------------------------------
class TestUserExportDataStructure(unittest.TestCase):
"""
Unit tests for User.export_user_data() method.
These test the shape of the returned dict without requiring a database.
We mock the nodes dynamic relationship by patching the instance after creation.
"""
@mock.patch("remarkbox.models.user.is_user_name_available", mock_always_true)
def test_export_data_has_profile_and_comments(self):
"""export_user_data returns a dict with 'profile' and 'comments' keys."""
user = User("export-test@example.com")
with mock.patch.object(type(user), "nodes", new_callable=mock.PropertyMock) as mock_nodes:
mock_query = mock.MagicMock()
mock_query.all.return_value = []
mock_nodes.return_value = mock_query
data = user.export_user_data()
self.assertIn("profile", data)
self.assertIn("comments", data)
self.assertIsInstance(data["profile"], dict)
self.assertIsInstance(data["comments"], list)
self.assertEqual(len(data["comments"]), 0)
@mock.patch("remarkbox.models.user.is_user_name_available", mock_always_true)
def test_export_data_profile_fields(self):
"""export_user_data profile contains expected fields."""
user = User("export-test2@example.com")
with mock.patch.object(type(user), "nodes", new_callable=mock.PropertyMock) as mock_nodes:
mock_query = mock.MagicMock()
mock_query.all.return_value = []
mock_nodes.return_value = mock_query
data = user.export_user_data()
profile = data["profile"]
for key in ("id", "name", "email", "created", "gravatar", "verified", "theme_mode"):
self.assertIn(key, profile, "Missing profile key: {}".format(key))
self.assertEqual(profile["email"], "export-test2@example.com")
@mock.patch("remarkbox.models.user.is_user_name_available", mock_always_true)
def test_export_data_includes_node_comments(self):
"""export_user_data includes comments for each attached node."""
user = User("export-test3@example.com")
mock_node = mock.MagicMock()
mock_node.id = "fake-uuid"
mock_node.created = 1700000000000
mock_node.changed = 1700000001000
mock_node.data = "Hello world"
mock_node.disabled = False
mock_node.verified = True
mock_node.approved = True
mock_node.title = None
mock_node.namespace = None
mock_node.root = mock.MagicMock()
mock_node.root.title = None
mock_node.root.uri = None
with mock.patch.object(type(user), "nodes", new_callable=mock.PropertyMock) as mock_nodes:
mock_query = mock.MagicMock()
mock_query.all.return_value = [mock_node]
mock_query.count.return_value = 1
mock_nodes.return_value = mock_query
data = user.export_user_data()
self.assertEqual(len(data["comments"]), 1)
comment = data["comments"][0]
self.assertEqual(comment["content"], "Hello world")
self.assertFalse(comment["disabled"])
self.assertTrue(comment["verified"])
self.assertTrue(comment["approved"])
@mock.patch("remarkbox.models.user.is_user_name_available", mock_always_true)
def test_export_data_includes_namespace_and_thread_info(self):
"""export_user_data includes namespace and thread info when present."""
user = User("export-test4@example.com")
mock_node = mock.MagicMock()
mock_node.id = "fake-uuid-2"
mock_node.created = 1700000000000
mock_node.changed = 1700000001000
mock_node.data = "Comment in thread"
mock_node.disabled = False
mock_node.verified = True
mock_node.approved = True
mock_node.title = "My Thread Title"
mock_node.namespace = mock.MagicMock()
mock_node.namespace.name = "example.com"
mock_node.root = mock.MagicMock()
mock_node.root.title = "Root Thread"
mock_node.root.uri = mock.MagicMock()
mock_node.root.uri.data = "https://example.com/page"
with mock.patch.object(type(user), "nodes", new_callable=mock.PropertyMock) as mock_nodes:
mock_query = mock.MagicMock()
mock_query.all.return_value = [mock_node]
mock_nodes.return_value = mock_query
data = user.export_user_data()
comment = data["comments"][0]
self.assertEqual(comment["title"], "My Thread Title")
self.assertEqual(comment["namespace"], "example.com")
self.assertEqual(comment["thread_title"], "Root Thread")
self.assertEqual(comment["thread_uri"], "https://example.com/page")
# ---------------------------------------------------------------------------
# T4: Namespace custom button text and comment labels
# ---------------------------------------------------------------------------
class TestNamespaceCustomText(unittest.TestCase):
"""Unit tests for T4: customizable button text and comment labels."""
def setUp(self):
self.namespace = Namespace("custom-text.example.com")
# Ensure production subscription so protected attrs return real values.
self.namespace.subscription_type = "production"
def test_default_submit_button_text_is_none(self):
"""Default submit_button_text is None (template falls back to 'save message')."""
self.assertIsNone(self.namespace.submit_button_text)
def test_default_comment_label_singular_is_none(self):
"""Default comment_label_singular is None (template falls back to 'remark')."""
self.assertIsNone(self.namespace.comment_label_singular)
def test_default_comment_label_plural_is_none(self):
"""Default comment_label_plural is None (template falls back to 'remarks')."""
self.assertIsNone(self.namespace.comment_label_plural)
def test_custom_submit_button_text_stores_and_retrieves(self):
"""Setting submit_button_text stores and retrieves correctly."""
self.namespace.submit_button_text = "Post Comment"
self.assertEqual(self.namespace.submit_button_text, "Post Comment")
def test_custom_comment_label_singular_stores_and_retrieves(self):
"""Setting comment_label_singular stores and retrieves correctly."""
self.namespace.comment_label_singular = "comment"
self.assertEqual(self.namespace.comment_label_singular, "comment")
def test_custom_comment_label_plural_stores_and_retrieves(self):
"""Setting comment_label_plural stores and retrieves correctly."""
self.namespace.comment_label_plural = "comments"
self.assertEqual(self.namespace.comment_label_plural, "comments")
def test_custom_text_protected_on_development_subscription(self):
"""Custom text returns None defaults when namespace subscription expires."""
self.namespace.submit_button_text = "Submit"
self.namespace.comment_label_singular = "thought"
self.namespace.comment_label_plural = "thoughts"
# Simulate subscription expiry.
self.namespace.subscription_type = "development"
self.namespace.reload_memoized_attr_protection()
self.assertIsNone(self.namespace.submit_button_text)
self.assertIsNone(self.namespace.comment_label_singular)
self.assertIsNone(self.namespace.comment_label_plural)
# ---------------------------------------------------------------------------
# T6: @mention parsing
# ---------------------------------------------------------------------------
class TestMentionParsing(unittest.TestCase):
"""Unit tests for T6: parse_mention_usernames()."""
def test_parse_single_mention(self):
"""Parse a single @mention."""
from remarkbox.lib.mentions import parse_mention_usernames
result = parse_mention_usernames("Hello @alice")
self.assertEqual(result, {"alice"})
def test_parse_multiple_mentions(self):
"""Parse multiple @mentions."""
from remarkbox.lib.mentions import parse_mention_usernames
result = parse_mention_usernames("Hello @alice and @bob")
self.assertEqual(result, {"alice", "bob"})
def test_parse_mention_at_start_of_string(self):
"""@username at start of string is recognized."""
from remarkbox.lib.mentions import parse_mention_usernames
result = parse_mention_usernames("@alice is great")
self.assertEqual(result, {"alice"})
def test_email_is_not_a_mention(self):
"""email@example.com should not be parsed as a mention."""
from remarkbox.lib.mentions import parse_mention_usernames
result = parse_mention_usernames("email@example.com")
self.assertEqual(result, set())
def test_parse_mention_with_dashes(self):
"""@user-name with dashes is a valid mention."""
from remarkbox.lib.mentions import parse_mention_usernames
result = parse_mention_usernames("Hello @my-user-name")
self.assertEqual(result, {"my-user-name"})
def test_parse_empty_string(self):
"""Empty string returns empty set."""
from remarkbox.lib.mentions import parse_mention_usernames
result = parse_mention_usernames("")
self.assertEqual(result, set())
def test_parse_none_returns_empty(self):
"""None input returns empty set."""
from remarkbox.lib.mentions import parse_mention_usernames
result = parse_mention_usernames(None)
self.assertEqual(result, set())
def test_parse_no_mentions(self):
"""String with no mentions returns empty set."""
from remarkbox.lib.mentions import parse_mention_usernames
result = parse_mention_usernames("Hello world, nothing here")
self.assertEqual(result, set())
def test_duplicate_mentions_deduped(self):
"""Duplicate mentions are de-duplicated."""
from remarkbox.lib.mentions import parse_mention_usernames
result = parse_mention_usernames("@alice said hi to @alice")
self.assertEqual(result, {"alice"})
def test_mention_after_newline(self):
"""@mention after newline is recognized (preceded by whitespace)."""
from remarkbox.lib.mentions import parse_mention_usernames
result = parse_mention_usernames("first line\n@alice second line")
self.assertIn("alice", result)
class TestMentionResolve(unittest.TestCase):
"""Unit tests for T6: resolve_mentions() with mocked DB."""
def test_resolve_returns_only_existing_users(self):
"""resolve_mentions returns only users that exist in the database."""
from remarkbox.lib.mentions import resolve_mentions
mock_user = mock.MagicMock()
mock_user.name = "alice"
with mock.patch("remarkbox.models.user.get_user_by_name") as mock_get:
# alice exists, bob does not
mock_get.side_effect = lambda db, name: mock_user if name.lower() == "alice" else None
dbsession = mock.MagicMock()
result = resolve_mentions(dbsession, "Hello @alice and @bob")
self.assertIn("alice", result)
self.assertNotIn("bob", result)
self.assertEqual(result["alice"], mock_user)
def test_resolve_empty_text(self):
"""resolve_mentions with empty text returns empty dict."""
from remarkbox.lib.mentions import resolve_mentions
dbsession = mock.MagicMock()
result = resolve_mentions(dbsession, "")
self.assertEqual(result, {})
class TestMentionReplaceWithLinks(unittest.TestCase):
"""Unit tests for T6: replace_mentions_with_links()."""
def test_replace_existing_user_with_link(self):
"""Existing user's @mention is replaced with an anchor tag."""
from remarkbox.lib.mentions import replace_mentions_with_links
mock_user = mock.MagicMock()
mock_user.name = "alice"
resolved = {"alice": mock_user}
html = "<p>Hello @alice</p>"
result = replace_mentions_with_links(html, resolved)
self.assertIn('class="mention"', result)
self.assertIn("@alice", result)
self.assertIn("/u/alice", result)
def test_nonexistent_user_left_as_text(self):
"""Non-existent user's @mention is left as plain text."""
from remarkbox.lib.mentions import replace_mentions_with_links
resolved = {} # No resolved users
html = "<p>Hello @nonexistent</p>"
result = replace_mentions_with_links(html, resolved)
self.assertIn("@nonexistent", result)
self.assertNotIn("class=\"mention\"", result)
def test_no_resolved_users_returns_unchanged(self):
"""When resolved_users is empty or None, HTML is returned unchanged."""
from remarkbox.lib.mentions import replace_mentions_with_links
html = "<p>Hello @world</p>"
result = replace_mentions_with_links(html, {})
self.assertEqual(result, html)
result2 = replace_mentions_with_links(html, None)
self.assertEqual(result2, html)
def test_replace_with_link_prefix(self):
"""Link prefix is prepended to profile URLs."""
from remarkbox.lib.mentions import replace_mentions_with_links
mock_user = mock.MagicMock()
mock_user.name = "bob"
resolved = {"bob": mock_user}
html = "<p>Hi @bob</p>"
result = replace_mentions_with_links(html, resolved, link_prefix="/embed/ns/test.com")
self.assertIn("/embed/ns/test.com/u/bob", result)
def test_mentions_inside_html_tags_not_replaced(self):
"""@mentions inside HTML tag attributes are not replaced."""
from remarkbox.lib.mentions import replace_mentions_with_links
mock_user = mock.MagicMock()
mock_user.name = "alice"
resolved = {"alice": mock_user}
# A link with @alice in href should not be double-replaced
html = '<a href="/u/alice">@alice</a>'
result = replace_mentions_with_links(html, resolved)
# The href content should not be modified
self.assertIn('href="/u/alice"', result)
# ---------------------------------------------------------------------------
# T8: Nesting depth settings
# ---------------------------------------------------------------------------
class TestNamespaceNestingDepth(unittest.TestCase):
"""Unit tests for T8: max_nesting_depth and collapse_depth."""
def setUp(self):
self.namespace = Namespace("nesting-test.example.com")
self.namespace.subscription_type = "production"
def test_default_max_nesting_depth_is_none(self):
"""Default max_nesting_depth is None (unlimited)."""
self.assertIsNone(self.namespace.max_nesting_depth)
def test_default_collapse_depth_is_none(self):
"""Default collapse_depth is None (never collapse)."""
self.assertIsNone(self.namespace.collapse_depth)
def test_max_nesting_depth_stores_correctly(self):
"""Setting max_nesting_depth stores and retrieves correctly."""
self.namespace.max_nesting_depth = 3
self.assertEqual(self.namespace.max_nesting_depth, 3)
def test_collapse_depth_stores_correctly(self):
"""Setting collapse_depth stores and retrieves correctly."""
self.namespace.collapse_depth = 5
self.assertEqual(self.namespace.collapse_depth, 5)
def test_nesting_depth_protected_on_development(self):
"""Nesting depth returns None defaults when subscription expires."""
self.namespace.max_nesting_depth = 3
self.namespace.collapse_depth = 2
self.namespace.subscription_type = "development"
self.namespace.reload_memoized_attr_protection()
self.assertIsNone(self.namespace.max_nesting_depth)
self.assertIsNone(self.namespace.collapse_depth)
def test_node_depth_calculation(self):
"""Node depth is computed from path_to_root length."""
root = Node()
root.id = 1
root.graph_depth = 0
child = root.new_child()
child.id = 2
child.parent_id = 1
child.root_id = 1
child.graph_depth = -1
root.children.append(child)
grandchild = child.new_child()
grandchild.id = 3
grandchild.parent_id = 2
grandchild.root_id = 1
grandchild.graph_depth = -1
child.children.append(grandchild)
# Depth is length of path_to_root - 1
self.assertEqual(root.depth, 0)
self.assertEqual(child.depth, 1)
self.assertEqual(grandchild.depth, 2)

693
remarkbox/tests/test_push.py Normal file
View file

@ -0,0 +1,693 @@
"""
Tests for T10: Browser push notifications.
Covers:
- Unit tests for push subscription CRUD in remarkbox.lib.push
- Unit tests for send_push_notification graceful fallback
- API tests for /push/vapid-key, /push/subscribe, /push/unsubscribe
- Integration test for notification_preference in notify dispatch
- Functional test for user settings saving notification_preference
"""
import json
import transaction
import unittest
import webtest
from unittest import mock
from unittest.mock import patch, MagicMock
from remarkbox.models import (
Node,
get_tm_session,
get_or_create_user_by_email,
get_user_by_email,
get_or_create_namespace,
)
from remarkbox.models.meta import Base
from remarkbox.models.user import User
from remarkbox.lib.push import (
get_vapid_keys,
get_push_subscriptions,
add_push_subscription,
remove_push_subscription,
send_push_notification,
send_push_to_user,
PUSH_AVAILABLE,
)
from pyramid.paster import get_appsettings
# ---------------------------------------------------------------------------
# Unit tests for VAPID key management.
# ---------------------------------------------------------------------------
class TestGetVapidKeys(unittest.TestCase):
"""Test get_vapid_keys reads from settings correctly."""
def test_returns_keys_when_configured(self):
settings = {
"push.vapid_private_key": "test-private-key",
"push.vapid_public_key": "test-public-key",
"push.vapid_contact": "mailto:admin@example.com",
}
keys = get_vapid_keys(settings)
self.assertIsNotNone(keys)
self.assertEqual(keys["private_key"], "test-private-key")
self.assertEqual(keys["public_key"], "test-public-key")
self.assertEqual(keys["contact"], "mailto:admin@example.com")
def test_returns_none_when_private_missing(self):
settings = {
"push.vapid_public_key": "test-public-key",
}
keys = get_vapid_keys(settings)
self.assertIsNone(keys)
def test_returns_none_when_public_missing(self):
settings = {
"push.vapid_private_key": "test-private-key",
}
keys = get_vapid_keys(settings)
self.assertIsNone(keys)
def test_returns_none_when_both_missing(self):
keys = get_vapid_keys({})
self.assertIsNone(keys)
def test_contact_defaults_to_empty(self):
settings = {
"push.vapid_private_key": "test-private-key",
"push.vapid_public_key": "test-public-key",
}
keys = get_vapid_keys(settings)
self.assertEqual(keys["contact"], "")
# ---------------------------------------------------------------------------
# Unit tests for push subscription CRUD.
# ---------------------------------------------------------------------------
class TestPushSubscriptionCRUD(unittest.TestCase):
"""Test get/add/remove push subscription functions."""
@mock.patch("remarkbox.models.user.is_user_name_available", mock.Mock(return_value=True))
def setUp(self):
self.user = User("pushtest@example.com")
def test_get_push_subscriptions_empty(self):
self.assertIsNone(self.user.push_subscriptions)
subs = get_push_subscriptions(self.user)
self.assertEqual(subs, [])
def test_get_push_subscriptions_invalid_json(self):
self.user.push_subscriptions = "not valid json"
subs = get_push_subscriptions(self.user)
self.assertEqual(subs, [])
def test_add_push_subscription(self):
sub = {
"endpoint": "https://push.example.com/1234",
"keys": {"p256dh": "abc", "auth": "def"},
}
result = add_push_subscription(self.user, sub)
self.assertTrue(result)
subs = get_push_subscriptions(self.user)
self.assertEqual(len(subs), 1)
self.assertEqual(subs[0]["endpoint"], "https://push.example.com/1234")
def test_add_duplicate_subscription(self):
sub = {
"endpoint": "https://push.example.com/1234",
"keys": {"p256dh": "abc", "auth": "def"},
}
add_push_subscription(self.user, sub)
result = add_push_subscription(self.user, sub)
self.assertFalse(result)
subs = get_push_subscriptions(self.user)
self.assertEqual(len(subs), 1)
def test_add_multiple_subscriptions(self):
sub1 = {"endpoint": "https://push.example.com/1", "keys": {"p256dh": "a", "auth": "b"}}
sub2 = {"endpoint": "https://push.example.com/2", "keys": {"p256dh": "c", "auth": "d"}}
add_push_subscription(self.user, sub1)
add_push_subscription(self.user, sub2)
subs = get_push_subscriptions(self.user)
self.assertEqual(len(subs), 2)
def test_remove_push_subscription(self):
sub = {
"endpoint": "https://push.example.com/remove-me",
"keys": {"p256dh": "abc", "auth": "def"},
}
add_push_subscription(self.user, sub)
result = remove_push_subscription(self.user, "https://push.example.com/remove-me")
self.assertTrue(result)
subs = get_push_subscriptions(self.user)
self.assertEqual(len(subs), 0)
self.assertIsNone(self.user.push_subscriptions)
def test_remove_nonexistent_subscription(self):
result = remove_push_subscription(self.user, "https://push.example.com/nonexistent")
self.assertFalse(result)
def test_remove_one_of_multiple(self):
sub1 = {"endpoint": "https://push.example.com/keep", "keys": {"p256dh": "a", "auth": "b"}}
sub2 = {"endpoint": "https://push.example.com/remove", "keys": {"p256dh": "c", "auth": "d"}}
add_push_subscription(self.user, sub1)
add_push_subscription(self.user, sub2)
result = remove_push_subscription(self.user, "https://push.example.com/remove")
self.assertTrue(result)
subs = get_push_subscriptions(self.user)
self.assertEqual(len(subs), 1)
self.assertEqual(subs[0]["endpoint"], "https://push.example.com/keep")
# ---------------------------------------------------------------------------
# Unit tests for send_push_notification.
# ---------------------------------------------------------------------------
class TestSendPushNotification(unittest.TestCase):
"""Test send_push_notification handles various conditions."""
def test_returns_false_when_push_unavailable(self):
if not PUSH_AVAILABLE:
result = send_push_notification(
{"endpoint": "https://push.example.com/1"},
{"title": "Test"},
{"private_key": "k", "contact": "mailto:a@b.com"},
)
self.assertFalse(result)
else:
with patch("remarkbox.lib.push.PUSH_AVAILABLE", False):
result = send_push_notification(
{"endpoint": "https://push.example.com/1"},
{"title": "Test"},
{"private_key": "k", "contact": "mailto:a@b.com"},
)
self.assertFalse(result)
def test_returns_false_when_no_vapid_keys(self):
result = send_push_notification(
{"endpoint": "https://push.example.com/1"},
{"title": "Test"},
None,
)
self.assertFalse(result)
@unittest.skipUnless(PUSH_AVAILABLE, "pywebpush not installed")
@patch("remarkbox.lib.push.webpush")
def test_successful_send(self, mock_webpush):
mock_webpush.return_value = None
result = send_push_notification(
{"endpoint": "https://push.example.com/1"},
{"title": "Test", "body": "Hello"},
{"private_key": "test-key", "contact": "mailto:admin@test.com"},
)
self.assertTrue(result)
mock_webpush.assert_called_once()
@unittest.skipUnless(PUSH_AVAILABLE, "pywebpush not installed")
@patch("remarkbox.lib.push.webpush")
def test_webpush_exception_returns_false(self, mock_webpush):
from pywebpush import WebPushException
mock_webpush.side_effect = WebPushException("Push failed")
result = send_push_notification(
{"endpoint": "https://push.example.com/1"},
{"title": "Test"},
{"private_key": "test-key", "contact": "mailto:admin@test.com"},
)
self.assertFalse(result)
@unittest.skipUnless(PUSH_AVAILABLE, "pywebpush not installed")
@patch("remarkbox.lib.push.webpush")
def test_generic_exception_returns_false(self, mock_webpush):
mock_webpush.side_effect = RuntimeError("Something broke")
result = send_push_notification(
{"endpoint": "https://push.example.com/1"},
{"title": "Test"},
{"private_key": "test-key", "contact": "mailto:admin@test.com"},
)
self.assertFalse(result)
def test_graceful_noop_when_pywebpush_not_installed(self):
result = send_push_notification(
{"endpoint": "https://push.example.com/1"},
{"title": "Test"},
{"private_key": "k", "contact": "mailto:a@b.com"},
)
self.assertIsInstance(result, bool)
# ---------------------------------------------------------------------------
# Unit tests for send_push_to_user.
# ---------------------------------------------------------------------------
class TestSendPushToUser(unittest.TestCase):
"""Test send_push_to_user dispatches correctly."""
@mock.patch("remarkbox.models.user.is_user_name_available", mock.Mock(return_value=True))
def setUp(self):
self.user = User("pushuser@example.com")
self.mock_request = MagicMock()
self.mock_request.registry.settings = {
"push.vapid_private_key": "test-private-key",
"push.vapid_public_key": "test-public-key",
"push.vapid_contact": "mailto:test@example.com",
}
def test_returns_zero_when_push_unavailable(self):
result = send_push_to_user(self.mock_request, self.user, {"title": "Test"})
self.assertEqual(result, 0)
def test_returns_zero_when_no_subscriptions(self):
result = send_push_to_user(self.mock_request, self.user, {"title": "Test"})
self.assertEqual(result, 0)
@unittest.skipUnless(PUSH_AVAILABLE, "pywebpush not installed")
@patch("remarkbox.lib.push.send_push_notification")
def test_sends_to_all_subscriptions(self, mock_send):
mock_send.return_value = True
sub1 = {"endpoint": "https://push.example.com/1", "keys": {"p256dh": "a", "auth": "b"}}
sub2 = {"endpoint": "https://push.example.com/2", "keys": {"p256dh": "c", "auth": "d"}}
add_push_subscription(self.user, sub1)
add_push_subscription(self.user, sub2)
result = send_push_to_user(self.mock_request, self.user, {"title": "Test"})
self.assertEqual(result, 2)
self.assertEqual(mock_send.call_count, 2)
# ---------------------------------------------------------------------------
# Integration tests for notification_preference in dispatch.
# ---------------------------------------------------------------------------
class TestNotificationPreferenceDispatch(unittest.TestCase):
"""Test that _send_push_for_notification respects notification_preference."""
@mock.patch("remarkbox.models.user.is_user_name_available", mock.Mock(return_value=True))
def _make_user(self, pref):
user = User("pref-test@example.com")
user.notification_preference = pref
return user
def _make_mocks(self, pref, action="commented"):
user = self._make_user(pref)
notification = MagicMock()
notification.user = user
notification.id = "test-notification-id"
notification.node_event.node.user.name = "Alice"
notification.node_event.action = action
root = MagicMock()
root.title = "Test Thread"
namespace = MagicMock()
namespace.name = "test.example.com"
request = MagicMock()
request.host_url = "https://my.remarkbox.com"
return notification, root, namespace, request
@patch("remarkbox.lib.push.PUSH_AVAILABLE", True)
@patch("remarkbox.lib.push.send_push_to_user")
def test_push_pref_sends_push(self, mock_send_push):
from remarkbox.lib.notify import _send_push_for_notification
mock_send_push.return_value = 1
notification, root, namespace, request = self._make_mocks("push")
_send_push_for_notification(request, notification, root, namespace)
mock_send_push.assert_called_once()
@patch("remarkbox.lib.push.PUSH_AVAILABLE", True)
@patch("remarkbox.lib.push.send_push_to_user")
def test_both_pref_sends_push(self, mock_send_push):
from remarkbox.lib.notify import _send_push_for_notification
mock_send_push.return_value = 1
notification, root, namespace, request = self._make_mocks("both", "created")
_send_push_for_notification(request, notification, root, namespace)
mock_send_push.assert_called_once()
@patch("remarkbox.lib.push.PUSH_AVAILABLE", True)
@patch("remarkbox.lib.push.send_push_to_user")
def test_email_pref_does_not_send_push(self, mock_send_push):
from remarkbox.lib.notify import _send_push_for_notification
notification, root, namespace, request = self._make_mocks("email")
_send_push_for_notification(request, notification, root, namespace)
mock_send_push.assert_not_called()
@patch("remarkbox.lib.push.PUSH_AVAILABLE", True)
@patch("remarkbox.lib.push.send_push_to_user")
def test_none_pref_does_not_send_push(self, mock_send_push):
from remarkbox.lib.notify import _send_push_for_notification
notification, root, namespace, request = self._make_mocks("none")
_send_push_for_notification(request, notification, root, namespace)
mock_send_push.assert_not_called()
@patch("remarkbox.lib.push.PUSH_AVAILABLE", False)
def test_push_unavailable_is_noop(self):
from remarkbox.lib.notify import _send_push_for_notification
notification, root, namespace, request = self._make_mocks("push")
# Should not raise.
_send_push_for_notification(request, notification, root, namespace)
@patch("remarkbox.lib.push.PUSH_AVAILABLE", True)
@patch("remarkbox.lib.push.send_push_to_user")
def test_push_payload_contains_expected_fields(self, mock_send_push):
from remarkbox.lib.notify import _send_push_for_notification
mock_send_push.return_value = 1
notification, root, namespace, request = self._make_mocks("push", "commented")
_send_push_for_notification(request, notification, root, namespace)
mock_send_push.assert_called_once()
payload = mock_send_push.call_args[0][2]
self.assertIn("title", payload)
self.assertIn("body", payload)
self.assertIn("url", payload)
self.assertIn("tag", payload)
self.assertIn("test.example.com", payload["title"])
self.assertIn("Alice", payload["body"])
# ---------------------------------------------------------------------------
# Unit tests for User model columns.
# ---------------------------------------------------------------------------
class TestUserNotificationPreferenceColumn(unittest.TestCase):
"""Verify the notification_preference and push_subscriptions columns on User."""
@mock.patch("remarkbox.models.user.is_user_name_available", mock.Mock(return_value=True))
def test_notification_preference_column_exists(self):
user = User("coltest@example.com")
self.assertTrue(hasattr(user, "notification_preference"))
@mock.patch("remarkbox.models.user.is_user_name_available", mock.Mock(return_value=True))
def test_push_subscriptions_default_none(self):
user = User("coltest2@example.com")
self.assertIsNone(user.push_subscriptions)
@mock.patch("remarkbox.models.user.is_user_name_available", mock.Mock(return_value=True))
def test_notification_preference_can_be_set(self):
user = User("coltest3@example.com")
for pref in ("push", "both", "none", "email"):
user.notification_preference = pref
self.assertEqual(user.notification_preference, pref)
# ---------------------------------------------------------------------------
# Functional / API tests -- all in one class to share DB.
# ---------------------------------------------------------------------------
class TestPushEndpoints(unittest.TestCase, object):
"""
Test push notification HTTP endpoints:
GET /push/vapid-key, POST /push/subscribe, POST /push/unsubscribe,
and user settings notification_preference.
Uses a single shared user for all authenticated tests to avoid
per-test cleanup cascading issues with SQLite.
"""
@classmethod
def setUpClass(cls):
from remarkbox import main
cls.settings = get_appsettings("test.ini")
cls.app = main({}, **cls.settings)
cls.testapp = webtest.TestApp(cls.app)
cls.session_factory = cls.app.registry["dbsession_factory"]
cls.engine = cls.session_factory.kw["bind"]
# Dispose existing connections and recreate tables.
# A prior test file's tearDownClass may have called drop_all(),
# leaving the engine's connection pool in a broken state.
cls.engine.dispose()
Base.metadata.create_all(bind=cls.engine)
cls.tm = transaction.manager
cls.dbsession = get_tm_session(cls.session_factory, cls.tm)
# Create a shared test user for all authenticated endpoint tests.
cls._test_email = "push-endpoint-test@remarkbox.com"
cls._test_user = get_or_create_user_by_email(cls.dbsession, cls._test_email)
cls.dbsession.add(cls._test_user)
cls.dbsession.flush()
cls.tm.commit()
@classmethod
def tearDownClass(cls):
cls.dbsession.close()
def setUp(self):
pass
def tearDown(self):
self.testapp.get("/log-out")
def _login(self):
"""Log in the shared test user and return (user, csrf_token).
Generates a fresh OTP each time to avoid exceeding the 10-attempt
limit on check_password. Uses a dedicated short-lived session for
password generation to avoid conflicting with the app's SQLite
writer lock.
"""
tm = transaction.TransactionManager(explicit=True)
tm.begin()
dbsession = get_tm_session(self.session_factory, tm)
user = get_or_create_user_by_email(dbsession, self._test_email)
raw_otp = user.new_password()
dbsession.add(user)
dbsession.flush()
tm.commit()
dbsession.close()
self.testapp.post(
"/verification-challenge?email={}&raw-otp={}".format(
self._test_email, raw_otp
)
)
res_csrf = self.testapp.get("/")
csrf = res_csrf.form.fields["csrf_token"][0].value
user = get_or_create_user_by_email(self.dbsession, self._test_email)
return user, csrf
# -----------------------------------------------------------------------
# GET /push/vapid-key
# -----------------------------------------------------------------------
def test_vapid_key_returns_json(self):
res = self.testapp.get("/push/vapid-key", expect_errors=True)
self.assertEqual(res.status_int, 200)
self.assertIn("application/json", res.content_type)
def test_vapid_key_structure(self):
res = self.testapp.get("/push/vapid-key", expect_errors=True)
self.assertIn("available", res.json)
def test_vapid_key_without_config(self):
res = self.testapp.get("/push/vapid-key", expect_errors=True)
self.assertFalse(res.json["available"])
# -----------------------------------------------------------------------
# POST /push/subscribe
# -----------------------------------------------------------------------
def test_subscribe_requires_auth(self):
res = self.testapp.post_json(
"/push/subscribe",
{"subscription": {"endpoint": "https://push.example.com/1", "keys": {}}},
expect_errors=True,
)
self.assertEqual(res.status_int, 401)
def test_subscribe_requires_json_body(self):
self._login()
res = self.testapp.post(
"/push/subscribe",
"not json",
content_type="text/plain",
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
def test_subscribe_requires_subscription(self):
self._login()
res = self.testapp.post_json(
"/push/subscribe",
{},
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
def test_subscribe_requires_endpoint(self):
self._login()
res = self.testapp.post_json(
"/push/subscribe",
{"subscription": {}},
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
def test_subscribe_success(self):
self._login()
res = self.testapp.post_json(
"/push/subscribe",
{
"subscription": {
"endpoint": "https://fcm.googleapis.com/fcm/send/test123",
"keys": {"p256dh": "testkey", "auth": "testauthkey"},
}
},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertEqual(res.json["status"], "subscribed")
def test_subscribe_duplicate(self):
self._login()
sub = {
"subscription": {
"endpoint": "https://fcm.googleapis.com/fcm/send/dupe",
"keys": {"p256dh": "testkey", "auth": "testauthkey"},
}
}
self.testapp.post_json("/push/subscribe", sub, expect_errors=True)
res = self.testapp.post_json("/push/subscribe", sub, expect_errors=True)
self.assertEqual(res.json["status"], "already_subscribed")
# -----------------------------------------------------------------------
# POST /push/unsubscribe
# -----------------------------------------------------------------------
def test_unsubscribe_requires_auth(self):
res = self.testapp.post_json(
"/push/unsubscribe",
{"endpoint": "https://push.example.com/1"},
expect_errors=True,
)
self.assertEqual(res.status_int, 401)
def test_unsubscribe_requires_endpoint(self):
self._login()
res = self.testapp.post_json(
"/push/unsubscribe",
{},
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
self.assertIn("endpoint", res.json["error"])
def test_unsubscribe_not_found(self):
self._login()
res = self.testapp.post_json(
"/push/unsubscribe",
{"endpoint": "https://push.example.com/nonexistent"},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertEqual(res.json["status"], "not_found")
def test_unsubscribe_success(self):
self._login()
self.testapp.post_json(
"/push/subscribe",
{
"subscription": {
"endpoint": "https://fcm.googleapis.com/fcm/send/unsub-test",
"keys": {"p256dh": "testkey", "auth": "testauthkey"},
}
},
expect_errors=True,
)
res = self.testapp.post_json(
"/push/unsubscribe",
{"endpoint": "https://fcm.googleapis.com/fcm/send/unsub-test"},
expect_errors=True,
)
self.assertEqual(res.status_int, 200)
self.assertEqual(res.json["status"], "unsubscribed")
# -----------------------------------------------------------------------
# User settings: notification_preference
# -----------------------------------------------------------------------
def test_notification_preference_column_exists_on_user(self):
"""User model has notification_preference attribute."""
user, csrf = self._login()
self.assertTrue(hasattr(user, "notification_preference"))
if user.notification_preference is not None:
self.assertIn(
user.notification_preference, ("email", "push", "both", "none")
)
def test_save_notification_preference_push(self):
user, csrf = self._login()
self.testapp.post(
"/u/settings",
{
"csrf_token": csrf,
"default-node-watcher-frequency": "daily",
"reply-watcher-frequency": "daily",
"notification-preference": "push",
},
)
self._verify_preference("push")
def test_save_notification_preference_both(self):
user, csrf = self._login()
self.testapp.post(
"/u/settings",
{
"csrf_token": csrf,
"default-node-watcher-frequency": "daily",
"reply-watcher-frequency": "daily",
"notification-preference": "both",
},
)
self._verify_preference("both")
def test_save_notification_preference_none(self):
user, csrf = self._login()
self.testapp.post(
"/u/settings",
{
"csrf_token": csrf,
"default-node-watcher-frequency": "daily",
"reply-watcher-frequency": "daily",
"notification-preference": "none",
},
)
self._verify_preference("none")
def test_save_notification_preference_email(self):
user, csrf = self._login()
self.testapp.post(
"/u/settings",
{
"csrf_token": csrf,
"default-node-watcher-frequency": "daily",
"reply-watcher-frequency": "daily",
"notification-preference": "email",
},
)
self._verify_preference("email")
def _verify_preference(self, expected):
"""Read the user's notification_preference using a fresh session."""
tm = transaction.TransactionManager(explicit=True)
tm.begin()
dbsession = get_tm_session(self.session_factory, tm)
user = get_user_by_email(dbsession, self._test_email)
self.assertEqual(user.notification_preference, expected)
tm.abort()
dbsession.close()

1169
remarkbox/tests/test_views.py
View file

File diff suppressed because it is too large Load diff

622
remarkbox/tests/test_webmention.py Normal file
View file

@ -0,0 +1,622 @@
"""
Tests for T7: Webmentions / IndieWeb support.
Covers:
- Unit tests for the Webmention model
- API tests for the webmention endpoints (both /webmention and /api/v1/webmention)
- Validation of source/target parameters
- Integration with URI lookup for finding threads
- Mocked HTTP fetches for source verification
- Author extraction and content snippet extraction
"""
import transaction
import unittest
import uuid
import webtest
from unittest import mock
from unittest.mock import patch, MagicMock
from remarkbox.models import (
Node,
get_tm_session,
get_or_create_user_by_email,
get_user_by_email,
get_or_create_namespace,
)
from remarkbox.models.meta import Base, now_timestamp
from remarkbox.models.webmention import (
Webmention,
get_webmention_by_id,
get_webmention_by_source_and_target,
get_verified_webmentions_for_node,
)
from remarkbox.models.uri import Uri, get_uri_by_uri
from pyramid.paster import get_appsettings
# ---------------------------------------------------------------------------
# Unit tests -- no database, no app.
# ---------------------------------------------------------------------------
class TestWebmentionModel(unittest.TestCase):
"""Unit tests for the Webmention SQLAlchemy model."""
def test_init_sets_source_and_target(self):
wm = Webmention(
source="https://example.com/post",
target="https://my.remarkbox.com/thread",
)
self.assertEqual(wm.source, "https://example.com/post")
self.assertEqual(wm.target, "https://my.remarkbox.com/thread")
def test_init_defaults_verified_false(self):
wm = Webmention(
source="https://example.com/post",
target="https://my.remarkbox.com/thread",
)
self.assertFalse(wm.verified)
def test_init_sets_timestamps(self):
wm = Webmention(
source="https://example.com/post",
target="https://my.remarkbox.com/thread",
)
self.assertIsNotNone(wm.created_timestamp)
self.assertIsNotNone(wm.updated_timestamp)
self.assertGreater(wm.created_timestamp, 0)
def test_init_generates_uuid(self):
wm = Webmention(
source="https://example.com/post",
target="https://my.remarkbox.com/thread",
)
self.assertIsNotNone(wm.id)
self.assertIsInstance(wm.id, uuid.UUID)
def test_mark_verified_sets_verified(self):
wm = Webmention(
source="https://example.com/post",
target="https://my.remarkbox.com/thread",
)
old_timestamp = wm.updated_timestamp
wm.mark_verified()
self.assertTrue(wm.verified)
self.assertGreaterEqual(wm.updated_timestamp, old_timestamp)
def test_mark_verified_with_author(self):
wm = Webmention(
source="https://example.com/post",
target="https://my.remarkbox.com/thread",
)
wm.mark_verified(
author_name="Jane Doe",
author_url="https://janedoe.example.com",
content="Here is a snippet of the content.",
)
self.assertTrue(wm.verified)
self.assertEqual(wm.author_name, "Jane Doe")
self.assertEqual(wm.author_url, "https://janedoe.example.com")
self.assertEqual(wm.content, "Here is a snippet of the content.")
def test_mark_verified_truncates_long_content(self):
wm = Webmention(
source="https://example.com/post",
target="https://my.remarkbox.com/thread",
)
long_content = "x" * 1000
wm.mark_verified(content=long_content)
self.assertEqual(len(wm.content), 500)
def test_mark_verified_no_author(self):
"""mark_verified without author fields leaves them as None."""
wm = Webmention(
source="https://example.com/post",
target="https://my.remarkbox.com/thread",
)
wm.mark_verified()
self.assertIsNone(wm.author_name)
self.assertIsNone(wm.author_url)
self.assertIsNone(wm.content)
def test_node_id_default_none(self):
wm = Webmention(
source="https://example.com/post",
target="https://my.remarkbox.com/thread",
)
self.assertIsNone(wm.node_id)
# ---------------------------------------------------------------------------
# Helper extraction function tests.
# ---------------------------------------------------------------------------
class TestWebmentionHelpers(unittest.TestCase):
"""Test the helper functions in remarkbox.views.webmention."""
def test_is_valid_url_http(self):
from remarkbox.views.webmention import _is_valid_url
self.assertTrue(_is_valid_url("http://example.com"))
self.assertTrue(_is_valid_url("https://example.com"))
self.assertFalse(_is_valid_url("ftp://example.com"))
self.assertFalse(_is_valid_url(""))
self.assertFalse(_is_valid_url(None))
def test_extract_author_with_hcard(self):
from remarkbox.views.webmention import _extract_author
html = """
<div class="h-card">
<a class="p-name u-url" href="https://janedoe.example.com">Jane Doe</a>
</div>
"""
name, url = _extract_author(html)
self.assertEqual(name, "Jane Doe")
self.assertEqual(url, "https://janedoe.example.com")
def test_extract_author_no_hcard(self):
from remarkbox.views.webmention import _extract_author
html = "<html><body><p>No h-card here.</p></body></html>"
name, url = _extract_author(html)
self.assertIsNone(name)
self.assertIsNone(url)
def test_extract_content_snippet(self):
from remarkbox.views.webmention import _extract_content_snippet
html = '<p>Check out this great article at <a href="https://target.com/thread">target link</a> for more info.</p>'
snippet = _extract_content_snippet(html, "https://target.com/thread")
self.assertIsNotNone(snippet)
self.assertIn("target", snippet.lower())
def test_extract_content_snippet_no_match(self):
from remarkbox.views.webmention import _extract_content_snippet
html = "<p>Nothing here.</p>"
snippet = _extract_content_snippet(html, "https://notfound.com")
self.assertIsNone(snippet)
# ---------------------------------------------------------------------------
# Functional / API tests -- require running app.
# All webmention functional tests are in one class to avoid DB teardown issues.
# ---------------------------------------------------------------------------
class TestWebmentionEndpoint(unittest.TestCase, object):
"""
Test the POST /webmention and /api/v1/webmention endpoints.
Also tests query helpers with live database.
"""
@classmethod
def setUpClass(cls):
from remarkbox import main
cls.settings = get_appsettings("test.ini")
cls.app = main({}, **cls.settings)
cls.testapp = webtest.TestApp(cls.app)
cls.session_factory = cls.app.registry["dbsession_factory"]
cls.engine = cls.session_factory.kw["bind"]
Base.metadata.create_all(bind=cls.engine)
cls.tm = transaction.manager
cls.dbsession = get_tm_session(cls.session_factory, cls.tm)
@classmethod
def tearDownClass(cls):
cls.dbsession.close()
def setUp(self):
"""Create a namespace, user, and a thread with a URI so webmentions can resolve."""
from remarkbox.models import create_root_node
# Ensure we have a clean transaction state.
try:
self.dbsession.rollback()
except Exception:
pass
self.test_user = get_or_create_user_by_email(
self.dbsession, "wm-test@remarkbox.com"
)
self.dbsession.add(self.test_user)
self.ns = get_or_create_namespace(self.dbsession, "wm-test.example.com")
self.dbsession.add(self.ns)
self.root = create_root_node()
self.root.namespace = self.ns
self.root.user = self.test_user
self.root.verified = True
self.root.title = "Webmention Thread"
self.root.set_data("Content for the webmention test thread")
self.dbsession.add(self.root)
self.dbsession.flush()
# Create a URI mapping so the webmention endpoint can find the thread.
self.target_url = "https://wm-test.example.com/my-post"
self.uri = Uri(data=self.target_url)
self.uri.node = self.root
self.dbsession.add(self.uri)
self.dbsession.flush()
self.root_id = self.root.id
self.ns_id = self.ns.id
self.tm.commit()
def tearDown(self):
self.testapp.get("/log-out")
try:
# Clean up webmentions.
self.dbsession.query(Webmention).filter(
Webmention.node_id == self.root_id,
).delete(synchronize_session=False)
# Clean up URI.
self.dbsession.query(Uri).filter(
Uri.data == self.target_url
).delete(synchronize_session=False)
# Clean up nodes.
self.dbsession.query(Node).filter(
Node.id == self.root_id
).delete(synchronize_session=False)
user = get_user_by_email(self.dbsession, "wm-test@remarkbox.com")
if user:
self.dbsession.delete(user)
self.dbsession.flush()
self.tm.commit()
except Exception:
self.tm.abort()
self.tm.begin()
# -----------------------------------------------------------------------
# Validation tests (POST /webmention).
# -----------------------------------------------------------------------
def test_missing_source_returns_400(self):
"""POST with missing source parameter returns 400."""
res = self.testapp.post_json(
"/webmention",
{"target": self.target_url},
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
self.assertIn("source", res.json["error"])
def test_missing_target_returns_400(self):
"""POST with missing target parameter returns 400."""
res = self.testapp.post_json(
"/webmention",
{"source": "https://example.com/post"},
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
self.assertIn("source", res.json["error"])
def test_missing_both_returns_400(self):
"""POST with no parameters returns 400."""
res = self.testapp.post_json(
"/webmention",
{},
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
def test_invalid_source_url_returns_400(self):
"""POST with non-HTTP source URL returns 400."""
res = self.testapp.post_json(
"/webmention",
{"source": "ftp://example.com/post", "target": self.target_url},
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
self.assertIn("source", res.json["error"])
def test_invalid_target_url_returns_400(self):
"""POST with non-HTTP target URL returns 400."""
res = self.testapp.post_json(
"/webmention",
{"source": "https://example.com/post", "target": "ftp://example.com/x"},
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
self.assertIn("target", res.json["error"])
def test_source_equals_target_returns_400(self):
"""POST where source == target returns 400."""
res = self.testapp.post_json(
"/webmention",
{"source": self.target_url, "target": self.target_url},
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
self.assertIn("different", res.json["error"])
def test_target_not_matching_thread_returns_400(self):
"""POST with target URL that does not match any Remarkbox thread returns 400."""
res = self.testapp.post_json(
"/webmention",
{
"source": "https://example.com/post",
"target": "https://nonexistent.example.com/no-thread",
},
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
self.assertIn("does not match", res.json["error"])
# -----------------------------------------------------------------------
# Source fetch / verification tests.
# -----------------------------------------------------------------------
@patch("remarkbox.views.webmention._fetch_source")
def test_unreachable_source_returns_400(self, mock_fetch):
"""Source URL that cannot be fetched returns 400."""
mock_fetch.return_value = None
res = self.testapp.post_json(
"/webmention",
{
"source": "https://example.com/unreachable",
"target": self.target_url,
},
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
self.assertIn("Could not fetch", res.json["error"])
@patch("remarkbox.views.webmention._fetch_source")
def test_source_without_link_to_target_returns_400(self, mock_fetch):
"""Source HTML that does not contain the target URL returns 400."""
mock_fetch.return_value = "<html><body>No link here.</body></html>"
res = self.testapp.post_json(
"/webmention",
{
"source": "https://example.com/post",
"target": self.target_url,
},
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
self.assertIn("does not contain", res.json["error"])
# -----------------------------------------------------------------------
# Successful webmention flow.
# -----------------------------------------------------------------------
@patch("remarkbox.views.webmention._fetch_source")
def test_valid_webmention_accepted(self, mock_fetch):
"""Valid webmention with source linking to target is accepted (202)."""
html = '<html><body><a href="{}">Link</a></body></html>'.format(
self.target_url
)
mock_fetch.return_value = html
res = self.testapp.post_json(
"/webmention",
{
"source": "https://example.com/valid-post",
"target": self.target_url,
},
expect_errors=True,
)
self.assertEqual(res.status_int, 202)
self.assertEqual(res.json["status"], "accepted")
self.assertIn("id", res.json)
@patch("remarkbox.views.webmention._fetch_source")
def test_valid_webmention_stores_verified(self, mock_fetch):
"""Valid webmention is stored as verified in the database."""
html = '<html><body><a href="{}">Link</a></body></html>'.format(
self.target_url
)
mock_fetch.return_value = html
res = self.testapp.post_json(
"/webmention",
{
"source": "https://example.com/store-test",
"target": self.target_url,
},
expect_errors=True,
)
self.assertEqual(res.status_int, 202)
wm = get_webmention_by_source_and_target(
self.dbsession, "https://example.com/store-test", self.target_url
)
self.assertIsNotNone(wm)
self.assertTrue(wm.verified)
self.assertEqual(wm.node_id, self.root_id)
@patch("remarkbox.views.webmention._fetch_source")
def test_duplicate_webmention_updates(self, mock_fetch):
"""Sending the same webmention twice updates the existing one (200)."""
html = '<html><body><a href="{}">Link</a></body></html>'.format(
self.target_url
)
mock_fetch.return_value = html
source = "https://example.com/dupe-post"
res1 = self.testapp.post_json(
"/webmention",
{"source": source, "target": self.target_url},
expect_errors=True,
)
self.assertEqual(res1.status_int, 202)
first_id = res1.json["id"]
# Second send should update, not create a new one.
res2 = self.testapp.post_json(
"/webmention",
{"source": source, "target": self.target_url},
expect_errors=True,
)
self.assertEqual(res2.status_int, 200)
self.assertEqual(res2.json["status"], "updated")
self.assertEqual(res2.json["id"], first_id)
@patch("remarkbox.views.webmention._fetch_source")
def test_webmention_extracts_author(self, mock_fetch):
"""Webmention extracts h-card author info from source HTML."""
html = '''
<html><body>
<div class="h-card">
<a class="p-name u-url" href="https://author.example.com">Test Author</a>
</div>
<a href="{}">Link to thread</a>
</body></html>
'''.format(self.target_url)
mock_fetch.return_value = html
res = self.testapp.post_json(
"/webmention",
{
"source": "https://example.com/author-test",
"target": self.target_url,
},
expect_errors=True,
)
self.assertEqual(res.status_int, 202)
wm = get_webmention_by_source_and_target(
self.dbsession, "https://example.com/author-test", self.target_url
)
self.assertIsNotNone(wm)
self.assertEqual(wm.author_name, "Test Author")
self.assertEqual(wm.author_url, "https://author.example.com")
@patch("remarkbox.views.webmention._fetch_source")
def test_form_encoded_webmention(self, mock_fetch):
"""Webmention endpoint also accepts form-encoded POST."""
html = '<html><body><a href="{}">Link</a></body></html>'.format(
self.target_url
)
mock_fetch.return_value = html
res = self.testapp.post(
"/webmention",
{
"source": "https://example.com/form-post",
"target": self.target_url,
},
expect_errors=True,
)
self.assertEqual(res.status_int, 202)
self.assertEqual(res.json_body["status"], "accepted")
# -----------------------------------------------------------------------
# /api/v1/webmention alias tests.
# -----------------------------------------------------------------------
@patch("remarkbox.views.webmention._fetch_source")
def test_api_webmention_accepted(self, mock_fetch):
"""POST /api/v1/webmention with valid data returns 202."""
html = '<html><body><a href="{}">Link</a></body></html>'.format(
self.target_url
)
mock_fetch.return_value = html
res = self.testapp.post_json(
"/api/v1/webmention",
{
"source": "https://example.com/api-wm-post",
"target": self.target_url,
},
expect_errors=True,
)
self.assertEqual(res.status_int, 202)
self.assertEqual(res.json["status"], "accepted")
def test_api_webmention_missing_params(self):
"""POST /api/v1/webmention with missing params returns 400."""
res = self.testapp.post_json(
"/api/v1/webmention",
{},
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
@patch("remarkbox.views.webmention._fetch_source")
def test_api_webmention_source_no_link(self, mock_fetch):
"""POST /api/v1/webmention where source does not link to target."""
mock_fetch.return_value = "<html><body>No link here</body></html>"
res = self.testapp.post_json(
"/api/v1/webmention",
{
"source": "https://example.com/nolink",
"target": self.target_url,
},
expect_errors=True,
)
self.assertEqual(res.status_int, 400)
self.assertIn("does not contain", res.json["error"])
# -----------------------------------------------------------------------
# Query helper integration tests.
# -----------------------------------------------------------------------
@patch("remarkbox.views.webmention._fetch_source")
def test_get_verified_webmentions_for_node(self, mock_fetch):
"""get_verified_webmentions_for_node returns only verified webmentions."""
html = '<html><body><a href="{}">Link</a></body></html>'.format(
self.target_url
)
mock_fetch.return_value = html
self.testapp.post_json(
"/webmention",
{
"source": "https://example.com/query-test",
"target": self.target_url,
},
expect_errors=True,
)
verified = get_verified_webmentions_for_node(self.dbsession, self.root_id)
self.assertGreaterEqual(len(verified), 1)
for wm in verified:
self.assertTrue(wm.verified)
def test_get_verified_webmentions_empty_for_nonexistent_node(self):
"""get_verified_webmentions_for_node returns empty for non-existent node."""
fake_id = uuid.uuid1()
verified = get_verified_webmentions_for_node(self.dbsession, fake_id)
self.assertEqual(len(verified), 0)
@patch("remarkbox.views.webmention._fetch_source")
def test_get_webmention_by_source_and_target(self, mock_fetch):
"""get_webmention_by_source_and_target finds an existing webmention."""
html = '<html><body><a href="{}">Link</a></body></html>'.format(
self.target_url
)
mock_fetch.return_value = html
self.testapp.post_json(
"/webmention",
{
"source": "https://example.com/lookup-test",
"target": self.target_url,
},
expect_errors=True,
)
wm = get_webmention_by_source_and_target(
self.dbsession,
"https://example.com/lookup-test",
self.target_url,
)
self.assertIsNotNone(wm)
self.assertTrue(wm.verified)
def test_get_webmention_by_source_and_target_not_found(self):
"""get_webmention_by_source_and_target returns None for non-existent pair."""
wm = get_webmention_by_source_and_target(
self.dbsession,
"https://nonexistent.example.com/post",
"https://nonexistent.example.com/thread",
)
self.assertIsNone(wm)

151
remarkbox/views/authenticated/authenticated.py
View file

@ -1,3 +1,5 @@
from json import dumps
from pyramid.view import view_config
from pyramid.httpexceptions import HTTPFound
@ -16,6 +18,8 @@ from remarkbox.models import (
from remarkbox.views import get_referer_or_home, user_required, reject_stand_alone
from remarkbox.scripts.modify_namespace import delete_namespace_cascade
from string import digits
try:
@ -62,6 +66,15 @@ def namespace_settings(request):
"google-site-verification", request.namespace.google_site_verification
)
# T4: customizable button text and comment labels
submit_button_text = p.get("submit-button-text", request.namespace.submit_button_text)
comment_label_singular = p.get("comment-label-singular", request.namespace.comment_label_singular)
comment_label_plural = p.get("comment-label-plural", request.namespace.comment_label_plural)
# T8: nesting depth settings
max_nesting_depth_raw = p.get("max-nesting-depth", "").strip()
collapse_depth_raw = p.get("collapse-depth", "").strip()
hide_unless_approved_checkbox = p.get("hide-unless-approved-checkbox", "off")
allow_anonymous_checkbox = p.get("allow-anonymous-checkbox", "off")
link_protection_checkbox = p.get("link-protection-checkbox", "off")
@ -239,6 +252,56 @@ def namespace_settings(request):
)
)
# T4: customizable button text and comment labels
if submit_button_text != request.namespace.submit_button_text and (
submit_button_text or request.namespace.submit_button_text
):
request.namespace.submit_button_text = submit_button_text
request.session.flash(
("Success, you changed the submit button text.", "success")
)
if comment_label_singular != request.namespace.comment_label_singular and (
comment_label_singular or request.namespace.comment_label_singular
):
request.namespace.comment_label_singular = comment_label_singular
request.session.flash(
("Success, you changed the singular comment label.", "success")
)
if comment_label_plural != request.namespace.comment_label_plural and (
comment_label_plural or request.namespace.comment_label_plural
):
request.namespace.comment_label_plural = comment_label_plural
request.session.flash(
("Success, you changed the plural comment label.", "success")
)
# T8: nesting depth settings
max_nesting_depth = int(max_nesting_depth_raw) if max_nesting_depth_raw else None
if max_nesting_depth != request.namespace.max_nesting_depth:
request.namespace.max_nesting_depth = max_nesting_depth
if max_nesting_depth is not None:
request.session.flash(
("Max nesting depth set to {}.".format(max_nesting_depth), "success")
)
else:
request.session.flash(
("Max nesting depth set to unlimited.", "success")
)
collapse_depth = int(collapse_depth_raw) if collapse_depth_raw else None
if collapse_depth != request.namespace.collapse_depth:
request.namespace.collapse_depth = collapse_depth
if collapse_depth is not None:
request.session.flash(
("Collapse depth set to {}.".format(collapse_depth), "success")
)
else:
request.session.flash(
("Collapse depth disabled.", "success")
)
request.dbsession.add(request.namespace)
request.dbsession.flush()
@ -382,6 +445,33 @@ def user_settings(request):
("Invalid theme mode value.", "error")
)
notification_preference = request.params.get(
"notification-preference",
getattr(request.user, "notification_preference", "email"),
)
current_pref = getattr(request.user, "notification_preference", "email")
if notification_preference != current_pref:
if notification_preference in ("email", "push", "both", "none"):
request.user.notification_preference = notification_preference
pref_labels = {
"email": "Email only",
"push": "Browser push only",
"both": "Email and browser push",
"none": "None",
}
request.session.flash(
(
"Notification delivery set to <b>{}</b>".format(
pref_labels[notification_preference]
),
"success",
)
)
else:
request.session.flash(
("Invalid notification preference value.", "error")
)
request.dbsession.add(request.user)
request.dbsession.flush()
@ -450,8 +540,6 @@ def namespace_dump_to_json(request):
request.session.flash(("You do not own that Namespace.", "error"))
return HTTPFound(get_referer_or_home(request))
from json import dumps
response = Response(body=dumps(request.namespace.dict_dump))
response.headerlist = []
# allow Javascript from other domains to download this resource.
@ -459,3 +547,62 @@ def namespace_dump_to_json(request):
(("Access-Control-Allow-Origin", "*"), ("Content-Type", "application/json"))
)
return response
@view_config(route_name="basic-user-delete-account", renderer="confirm-delete-account.j2")
@view_config(route_name="embed-user-delete-account", renderer="confirm-delete-account.j2")
@user_required()
def delete_account(request):
"""Delete the authenticated user's account after confirmation."""
if request.method == "POST":
confirm = request.params.get("confirm-delete", "")
if confirm == "DELETE":
request.user.anonymize_account()
request.session.flash(
("Your account has been deleted and your data has been anonymized.", "success")
)
request.session["authenticated_user_id"] = None
return HTTPFound("/")
else:
request.session.flash(
('You must type "DELETE" to confirm account deletion.', "error")
)
return {"the_title": "Delete My Account"}
@view_config(route_name="basic-user-export-data")
@view_config(route_name="embed-user-export-data")
@user_required()
def export_user_data(request):
"""Export the authenticated user's data as a JSON download."""
data = request.user.export_user_data()
body = dumps(data, indent=2)
response = Response(body=body)
response.content_type = "application/json"
response.content_disposition = 'attachment; filename="remarkbox-data-export.json"'
return response
@view_config(route_name="basic-namespace-delete", renderer="confirm-delete-namespace.j2")
@view_config(route_name="embed-namespace-delete", renderer="confirm-delete-namespace.j2")
@user_required()
@reject_stand_alone
def delete_namespace(request):
"""Delete a namespace and all associated data after confirmation."""
if not request.user in request.namespace.owners:
request.session.flash(("You do not own that Namespace.", "error"))
return HTTPFound(get_referer_or_home(request))
namespace_name = request.namespace.name
if request.method == "POST":
confirm = request.params.get("confirm-delete", "")
if confirm == namespace_name:
delete_namespace_cascade(request.dbsession, request.namespace)
request.session.flash(
('Namespace "{}" and all associated data have been permanently deleted.'.format(namespace_name), "success")
)
return HTTPFound("/")
else:
request.session.flash(
('You must type the namespace name "{}" to confirm deletion.'.format(namespace_name), "error")
)
return {"the_title": "Delete Namespace: {}".format(namespace_name)}

6
remarkbox/views/list_nodes.py
View file

@ -118,17 +118,19 @@ def user_nodes(request):
subject_user = get_user_by_name(request.dbsession, subject_user_name)
if subject_user is None:
return HTTPFound(get_referer_or_home(request))
namespace = request.namespace
if "disabled" in request.params:
# TODO: pagination.
state = "disabled"
nodes = subject_user.disabled_nodes
nodes = subject_user.disabled_nodes(namespace=namespace)
elif "pending" in request.params:
# TODO: pagination.
state = "pending"
nodes = subject_user.unverified_nodes
nodes = subject_user.unverified_nodes(namespace=namespace)
else:
state = "active"
nodes = subject_user.page_nodes(
namespace=namespace,
limit=request.page_size, offset=request.page_offset
)
return {

2
remarkbox/views/new_thread.py
View file

@ -54,7 +54,7 @@ def new_thread(request):
node.namespace = request.namespace
node.ip_address = unicode(request.client_addr)
node.title = thread_title
node.set_data(thread_data)
node.set_data(thread_data, dbsession=request.dbsession)
# Handle anonymous vs authenticated user
if user_surrogate:

72
remarkbox/views/push.py Normal file
View file

@ -0,0 +1,72 @@
"""
Web Push notification subscribe/unsubscribe endpoints.
"""
import logging
from pyramid.view import view_config
from remarkbox.lib.push import (
PUSH_AVAILABLE,
add_push_subscription,
remove_push_subscription,
get_vapid_keys,
)
log = logging.getLogger(__name__)
@view_config(route_name="push-vapid-key", request_method="GET", renderer="json", require_csrf=False)
def get_vapid_public_key(request):
if not PUSH_AVAILABLE:
return {"available": False, "reason": "Push libraries not installed"}
vapid_keys = get_vapid_keys(request.registry.settings)
if not vapid_keys:
return {"available": False, "reason": "VAPID keys not configured"}
return {"available": True, "public_key": vapid_keys["public_key"]}
@view_config(route_name="push-subscribe", request_method="POST", renderer="json", require_csrf=False)
def push_subscribe(request):
if not request.user or not request.user.authenticated:
request.response.status_code = 401
return {"error": "Authentication required"}
try:
body = request.json_body
except Exception:
request.response.status_code = 400
return {"error": "JSON body required"}
subscription = body.get("subscription")
if not subscription or not subscription.get("endpoint"):
request.response.status_code = 400
return {"error": "subscription with endpoint is required"}
added = add_push_subscription(request.user, subscription)
request.dbsession.add(request.user)
request.dbsession.flush()
if added:
log.info("Push subscription added for user=%s", request.user.id)
return {"status": "subscribed"}
return {"status": "already_subscribed"}
@view_config(route_name="push-unsubscribe", request_method="POST", renderer="json", require_csrf=False)
def push_unsubscribe(request):
if not request.user or not request.user.authenticated:
request.response.status_code = 401
return {"error": "Authentication required"}
try:
body = request.json_body
except Exception:
request.response.status_code = 400
return {"error": "JSON body required"}
endpoint = body.get("endpoint")
if not endpoint:
request.response.status_code = 400
return {"error": "endpoint is required"}
removed = remove_push_subscription(request.user, endpoint)
request.dbsession.add(request.user)
request.dbsession.flush()
if removed:
log.info("Push subscription removed for user=%s", request.user.id)
return {"status": "unsubscribed"}
return {"status": "not_found"}

9
remarkbox/views/reply_node.py
View file

@ -68,6 +68,13 @@ def reply_node(request):
)
return HTTPFound(get_referer_or_home(request))
# return early if max nesting depth exceeded (T8).
if request.namespace.max_nesting_depth is not None:
if request.node.depth >= request.namespace.max_nesting_depth:
request.session.flash(
("Maximum nesting depth reached. You cannot reply further.", "error")
)
return HTTPFound(get_referer_or_home(request))
# check CSRF only if user is authenticated.
if request.method == "POST" and request.csrf_token:
@ -84,7 +91,7 @@ def reply_node(request):
# STEP 2: attach a brand new child node to parent node.
child = parent.new_child()
child.ip_address = unicode(request.client_addr)
child.set_data(thread_data, namespace=request.namespace)
child.set_data(thread_data, namespace=request.namespace, dbsession=request.dbsession)
# Handle anonymous vs authenticated user
if user_surrogate:

130
remarkbox/views/webmention.py Normal file
View file

@ -0,0 +1,130 @@
"""
Webmention receiving endpoint.
Implements the W3C Webmention spec (https://www.w3.org/TR/webmention/).
"""
import logging
import re
import urllib.request
import urllib.error
from pyramid.view import view_config
from pyramid.response import Response
from remarkbox.models.uri import get_uri_by_uri
from remarkbox.models.webmention import (
get_webmention_by_source_and_target,
Webmention,
)
log = logging.getLogger(__name__)
MAX_SOURCE_SIZE = 256 * 1024
FETCH_TIMEOUT = 10
def _is_valid_url(url):
return url and (url.startswith("http://") or url.startswith("https://"))
def _fetch_source(source_url):
try:
req = urllib.request.Request(
source_url,
headers={"User-Agent": "Remarkbox Webmention/1.0"},
)
resp = urllib.request.urlopen(req, timeout=FETCH_TIMEOUT)
body = resp.read(MAX_SOURCE_SIZE)
try:
return body.decode("utf-8")
except UnicodeDecodeError:
return body.decode("latin-1")
except Exception:
log.warning("Failed to fetch source URL: %s", source_url, exc_info=True)
return None
def _extract_author(html):
author_name = None
author_url = None
h_card_match = re.search(
r'class="[^"]*h-card[^"]*"[^>]*>.*?class="[^"]*p-name[^"]*"[^>]*>([^<]+)',
html, re.DOTALL | re.IGNORECASE,
)
if h_card_match:
author_name = h_card_match.group(1).strip()
url_match = re.search(
r'class="[^"]*h-card[^"]*"[^>]*>.*?class="[^"]*u-url[^"]*"[^>]*href="([^"]+)"',
html, re.DOTALL | re.IGNORECASE,
)
if url_match:
author_url = url_match.group(1).strip()
return author_name, author_url
def _extract_content_snippet(html, target_url):
pattern = re.escape(target_url)
match = re.search(pattern, html)
if match:
start = max(0, match.start() - 200)
end = min(len(html), match.end() + 200)
snippet = html[start:end]
snippet = re.sub(r"<[^>]+>", " ", snippet)
snippet = re.sub(r"\s+", " ", snippet).strip()
if len(snippet) > 300:
snippet = snippet[:300] + "..."
return snippet
return None
@view_config(route_name="webmention", request_method="POST", require_csrf=False)
@view_config(route_name="api-webmention", request_method="POST", require_csrf=False)
def receive_webmention(request):
try:
body = request.json_body
except Exception:
body = {}
source = body.get("source") or request.params.get("source", "")
target = body.get("target") or request.params.get("target", "")
if not source or not target:
return Response(json_body={"error": "source and target parameters are required"}, status=400, content_type="application/json")
if not _is_valid_url(source):
return Response(json_body={"error": "source must be a valid HTTP or HTTPS URL"}, status=400, content_type="application/json")
if not _is_valid_url(target):
return Response(json_body={"error": "target must be a valid HTTP or HTTPS URL"}, status=400, content_type="application/json")
if source == target:
return Response(json_body={"error": "source and target must be different URLs"}, status=400, content_type="application/json")
uri = get_uri_by_uri(request.dbsession, target)
if uri is None or uri.node is None:
return Response(json_body={"error": "target URL does not match any Remarkbox thread"}, status=400, content_type="application/json")
root_node = uri.node if uri.node.is_root else uri.node.root
existing = get_webmention_by_source_and_target(request.dbsession, source, target)
source_html = _fetch_source(source)
if source_html is None:
return Response(json_body={"error": "Could not fetch source URL"}, status=400, content_type="application/json")
if target not in source_html:
return Response(json_body={"error": "Source URL does not contain a link to the target"}, status=400, content_type="application/json")
author_name, author_url = _extract_author(source_html)
content_snippet = _extract_content_snippet(source_html, target)
if existing:
existing.mark_verified(author_name=author_name, author_url=author_url, content=content_snippet)
request.dbsession.add(existing)
request.dbsession.flush()
log.info("webmention updated: source=%s target=%s node=%s", source, target, root_node.id)
return Response(json_body={"status": "updated", "id": str(existing.id)}, status=200, content_type="application/json")
webmention = Webmention(source=source, target=target)
webmention.node = root_node
webmention.mark_verified(author_name=author_name, author_url=author_url, content=content_snippet)
request.dbsession.add(webmention)
request.dbsession.flush()
log.info("webmention received: source=%s target=%s node=%s", source, target, root_node.id)
return Response(json_body={"status": "accepted", "id": str(webmention.id)}, status=202, content_type="application/json")

419
remarkbox_client.py Normal file
View file

@ -0,0 +1,419 @@
"""
Remarkbox API Client (Python, stdlib only)
Download:
curl -s https://REMARKBOX/api/v1/clients/python -o remarkbox_client.py
wget -q https://REMARKBOX/api/v1/clients/python -O remarkbox_client.py
Quick start:
from remarkbox_client import RemarkboxClient
client = RemarkboxClient("https://my.remarkbox.com")
# List threads
result = client.list_threads("meta.remarkbox.com")
for thread in result["threads"]:
print(thread["title"])
# Read a thread and its replies
thread = client.get_thread(thread_id)
for reply in thread["replies"]:
print(reply["data"])
# Post anonymously (namespace must allow anonymous)
node = client.create_thread(
namespace="meta.remarkbox.com",
title="Hello from Python",
data="This is a test post.",
anonymous_name="MyBot",
)
# Reply to a thread
reply = client.reply(node["node"]["id"], data="Nice thread!")
# Authenticate via email OTP
client.login("agent@example.com")
# ... check inbox for 6-digit code ...
client.verify("agent@example.com", "123456")
# Now requests are authenticated
thread = client.create_thread(
namespace="meta.remarkbox.com",
title="Verified post",
data="Posted with a session.",
)
# Edit your own post
client.edit_node(thread["node"]["id"], data="Updated content.")
Configuration:
# From arguments (highest priority)
client = RemarkboxClient("https://my.remarkbox.com")
# From environment variables
# REMARKBOX_URL=https://my.remarkbox.com
# REMARKBOX_EMAIL=agent@example.com
client = RemarkboxClient.from_env()
# From config file (~/.config/remarkbox/config.json)
# {"url": "https://my.remarkbox.com", "email": "agent@example.com"}
client = RemarkboxClient.from_config()
Requires: Python 3.6+ (stdlib only, no pip install needed)
License: Same as Remarkbox
"""
import json
import os
import http.cookiejar
import urllib.request
import urllib.error
import urllib.parse
__version__ = "0.1.0"
class RemarkboxError(Exception):
"""Raised when the API returns an error response."""
def __init__(self, status, body):
self.status = status
self.body = body
msg = body.get("error", str(body)) if isinstance(body, dict) else str(body)
super().__init__("HTTP {}: {}".format(status, msg))
class RemarkboxClient:
"""Remarkbox API client. Manages sessions via cookies automatically."""
def __init__(self, url, email=None, cookie_file=None):
"""
Args:
url: Base URL of the Remarkbox instance (e.g. https://my.remarkbox.com)
email: Optional default email for login/verify
cookie_file: Optional path to persist session cookies across runs.
If provided, cookies are loaded on init and saved
after login/verify. Use this to stay logged in.
"""
self.url = url.rstrip("/")
self.email = email
self._cookie_file = cookie_file
if cookie_file:
self._cookie_jar = http.cookiejar.MozillaCookieJar(cookie_file)
if os.path.exists(cookie_file):
self._cookie_jar.load(ignore_discard=True, ignore_expires=True)
else:
self._cookie_jar = http.cookiejar.CookieJar()
self._opener = urllib.request.build_opener(
urllib.request.HTTPCookieProcessor(self._cookie_jar)
)
@classmethod
def from_env(cls):
"""Create client from environment variables.
Reads:
REMARKBOX_URL (required)
REMARKBOX_EMAIL (optional)
"""
url = os.environ.get("REMARKBOX_URL")
if not url:
raise RemarkboxError(0, {"error": "REMARKBOX_URL environment variable not set"})
email = os.environ.get("REMARKBOX_EMAIL")
return cls(url, email=email)
@classmethod
def from_config(cls, path=None):
"""Create client from a JSON config file.
Default path: ~/.config/remarkbox/config.json
Config format:
{"url": "https://my.remarkbox.com", "email": "agent@example.com"}
"""
if path is None:
path = os.path.join(
os.path.expanduser("~"), ".config", "remarkbox", "config.json"
)
with open(path) as f:
config = json.load(f)
url = config.get("url")
if not url:
raise RemarkboxError(0, {"error": "url is required in config file"})
return cls(url, email=config.get("email"), cookie_file=config.get("cookie_file"))
def _save_cookies(self):
"""Persist cookies to disk if cookie_file was provided."""
if self._cookie_file and hasattr(self._cookie_jar, "save"):
self._cookie_jar.save(ignore_discard=True, ignore_expires=True)
def _request(self, method, path, body=None):
"""Make an HTTP request and return parsed JSON."""
url = self.url + path
data = None
headers = {}
if body is not None:
data = json.dumps(body).encode("utf-8")
headers["Content-Type"] = "application/json"
req = urllib.request.Request(url, data=data, headers=headers, method=method)
try:
resp = self._opener.open(req)
raw = resp.read().decode("utf-8")
try:
return json.loads(raw) if raw else {}
except json.JSONDecodeError:
raise RemarkboxError(resp.status, {"error": "Non-JSON response"})
except urllib.error.HTTPError as e:
raw = e.read().decode("utf-8")
try:
body = json.loads(raw)
except Exception:
body = {"error": raw}
raise RemarkboxError(e.code, body)
# ----- Version -----
def version(self):
"""Get the deployed version (git commit hash).
Returns:
dict with key: version
"""
return self._request("GET", "/api/v1/version")
# ----- Threads -----
def list_threads(self, namespace, page=1):
"""List threads in a namespace.
Args:
namespace: The namespace name (e.g. "meta.remarkbox.com")
page: Page number (default 1)
Returns:
dict with keys: namespace, threads, page, page_size
"""
params = urllib.parse.urlencode({"namespace": namespace, "page": page})
return self._request("GET", "/api/v1/threads?" + params)
def get_thread(self, node_id, limit=None, offset=None):
"""Get a thread and its replies (paginated).
Args:
node_id: UUID of the root thread node
limit: Maximum replies to return (default 100, max 500)
offset: Number of replies to skip (default 0)
Returns:
dict with keys: namespace, thread, replies, total_replies,
page, limit, offset, has_more
"""
params = {}
if limit is not None:
params["limit"] = limit
if offset is not None:
params["offset"] = offset
path = "/api/v1/threads/{}".format(node_id)
if params:
path += "?" + urllib.parse.urlencode(params)
return self._request("GET", path)
def create_thread(self, namespace, title, data, anonymous_name=None, email=None):
"""Create a new thread.
Args:
namespace: Target namespace name
title: Thread title
data: Markdown content (max 500000 chars)
anonymous_name: Name for anonymous posting (optional)
email: Email to associate with post (optional)
Returns:
dict with keys: node, verified
"""
body = {"namespace": namespace, "title": title, "data": data}
if anonymous_name:
body["anonymous_name"] = anonymous_name
if email:
body["email"] = email
return self._request("POST", "/api/v1/threads", body)
# ----- Replies -----
def reply(self, node_id, data, anonymous_name=None, email=None):
"""Reply to a thread or another reply.
Args:
node_id: UUID of the parent node (thread or reply)
data: Markdown content (max 500000 chars)
anonymous_name: Name for anonymous posting (optional)
email: Email to associate with post (optional)
Returns:
dict with keys: node, verified
"""
body = {"data": data}
if anonymous_name:
body["anonymous_name"] = anonymous_name
if email:
body["email"] = email
return self._request("POST", "/api/v1/threads/{}/replies".format(node_id), body)
# ----- Nodes -----
def get_node(self, node_id):
"""Get a single node by ID.
Args:
node_id: UUID of the node
Returns:
dict with key: node
"""
return self._request("GET", "/api/v1/nodes/{}".format(node_id))
def edit_node(self, node_id, data=None, title=None):
"""Edit a node (requires authentication).
Args:
node_id: UUID of the node to edit
data: New markdown content (optional)
title: New title, only for root nodes (optional)
Returns:
dict with key: node
"""
body = {}
if data is not None:
body["data"] = data
if title is not None:
body["title"] = title
if not body:
raise ValueError("data or title is required")
return self._request("PATCH", "/api/v1/nodes/{}".format(node_id), body)
# ----- Auth -----
def login(self, email=None):
"""Request an OTP code be sent to the email address.
Args:
email: Email address (uses self.email if not provided)
Returns:
dict with keys: status, message
"""
email = email or self.email
if not email:
raise ValueError("email is required")
return self._request("POST", "/api/v1/auth/login", {"email": email})
def verify(self, email=None, otp=None):
"""Verify an OTP code and establish an authenticated session.
After calling this, subsequent requests are authenticated
via the session cookie (managed automatically).
Args:
email: Email address (uses self.email if not provided)
otp: The 6-digit verification code from email
Returns:
dict with keys: status, user
"""
email = email or self.email
if not email:
raise ValueError("email is required")
if not otp:
raise ValueError("otp is required")
result = self._request("POST", "/api/v1/auth/verify", {"email": email, "otp": otp})
self._save_cookies()
return result
# ----- Profile -----
def get_profile(self):
"""Get the current authenticated user's profile.
Returns:
dict with key: user (id, name, email)
"""
return self._request("GET", "/api/v1/user/profile")
def update_profile(self, name):
"""Update the current user's display name.
Args:
name: New display name (alphanumeric and dashes only)
Returns:
dict with key: user (id, name, email)
"""
return self._request("PATCH", "/api/v1/user/profile", {"name": name})
# ----- CLI -----
def main():
"""Simple CLI for quick testing."""
import sys
usage = """Usage: python remarkbox_client.py <url> <command> [args...]
Commands:
threads <namespace> List threads
thread <node_id> Get thread with replies
node <node_id> Get a single node
post <namespace> <title> <data> [name] Create thread (anonymous)
reply <node_id> <data> [name] Reply to thread (anonymous)
login <email> Request OTP
verify <email> <otp> Verify OTP
Examples:
python remarkbox_client.py https://my.remarkbox.com threads meta.remarkbox.com
python remarkbox_client.py https://my.remarkbox.com post meta.remarkbox.com "Hello" "World" MyBot
"""
if len(sys.argv) < 3:
print(usage)
sys.exit(1)
url = sys.argv[1]
cmd = sys.argv[2]
args = sys.argv[3:]
client = RemarkboxClient(url)
try:
if cmd == "threads" and len(args) >= 1:
result = client.list_threads(args[0])
elif cmd == "thread" and len(args) >= 1:
result = client.get_thread(args[0])
elif cmd == "node" and len(args) >= 1:
result = client.get_node(args[0])
elif cmd == "post" and len(args) >= 3:
name = args[3] if len(args) > 3 else None
result = client.create_thread(args[0], args[1], args[2], anonymous_name=name)
elif cmd == "reply" and len(args) >= 2:
name = args[2] if len(args) > 2 else None
result = client.reply(args[0], args[1], anonymous_name=name)
elif cmd == "login" and len(args) >= 1:
result = client.login(args[0])
elif cmd == "verify" and len(args) >= 2:
result = client.verify(args[0], args[1])
else:
print(usage)
sys.exit(1)
print(json.dumps(result, indent=2))
except RemarkboxError as e:
print(json.dumps({"error": str(e), "status": e.status}, indent=2), file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()