remarkbox/CLAUDE.md

Claude Code Configuration

Project Setup

IMPORTANT: Before starting any work on a repository:

1. Check for a CLAUDE.md file in our repository root
2. Check for a CLAUDE.md file in parent directories (we often work across repos on localhost)
3. Read and follow all instructions in those files
4. These project-specific instructions override default Claude Code behavior
5. Look for conventions around commits, testing, code style, and workflows
6. If working across multiple repositories, respect our conventions from each repo's CLAUDE.md

Commit Attribution

When creating git commits, use clean, simple commit messages:

Commit message here.

Do NOT include:

• 🤖 Generated with [Claude Code](https://claude.com/claude-code)
• Co-Authored-By: Claude <noreply@anthropic.com>
• Any fake corporate entities as co-authors

Only attribute real humans as co-authors when collaborating.

When fox approves a proposed commit, always commit AND push immediately. No second confirmation needed. "yes" = commit + git push.

Database Migrations (Alembic)

When adding new columns or modifying our database schema:

1. Backup SQLite first: cp data/remarkbox.sqlite data/remarkbox.sqlite.bak
2. Add our column to our model in remarkbox/models/
3. Generate migration: alembic -c development.ini revision --autogenerate -m "description"
4. Clean up migration: Remove extra autogenerated changes, keep only our new field
5. Run migration: alembic -c development.ini upgrade head

Ticket System

Tracked issues live in docs/tickets/. Start every session by reading our index:

cat docs/tickets/index.md

• Index: docs/tickets/index.md is our master list. Always update it when creating or closing tickets.
• Numbering: Sequential. Next number = highest existing + 1.
• Workflow: Set status to in-progress when starting, resolved when done. Update index.md to match.
• New tickets: If you find a bug or get a feature request, create a new ticket file and add it to our index.
• Sources: Tickets reference community threads from meta.remarkbox.com and faq.remarkbox.com by UUID.

Remarkbox API and Python Client

Remarkbox has a JSON API at /api/v1/. You can use it to read and write threads on production as timehexon. Our session cookie is saved at ~/.config/remarkbox/cookies.txt.

Quick start (from a Python script in our scratchpad or inline)

import os, sys
sys.path.insert(0, "/home/fox/git/remarkbox/remarkbox/api")
from remarkbox_client import RemarkboxClient

c = RemarkboxClient(
    "https://my.remarkbox.com",
    cookie_file=os.path.expanduser("~/.config/remarkbox/cookies.txt"),
)

# Read
threads = c.list_threads("meta.remarkbox.com")
thread = c.get_thread("9f970183-ffaf-11f0-b565-040140774501")
node = c.get_node(node_id)
profile = c.get_profile()
ver = c.version()

# Write (authenticated)
result = c.create_thread(namespace="meta.remarkbox.com", title="Title", data="Body")
result = c.reply(parent_node_id, data="Reply body")
c.edit_node(node_id, data="Updated body")
c.edit_node(node_id, title="Updated title")  # title only for root nodes
c.update_profile("new-display-name")

# Moderate (authenticated, moderator or owner)
c.disable_node(node_id)
c.enable_node(node_id)
c.approve_node(node_id)
c.lock_node(node_id)
c.unlock_node(node_id)
c.delete_node(node_id)  # permanent, moderator only

# Export (pandoc-powered, 67 output formats)
formats = c.export_formats()
md = c.export_thread(node_id, "markdown")
pdf = c.export_thread(node_id, "pdf")
epub = c.export_namespace("meta.remarkbox.com", "epub")
docx = c.export_node(node_id, "docx")

# Wiki mode (namespace.wiki=True, any authenticated user can edit root nodes)
c.wiki_edit(node_id, data="Updated content")  # creates revision first
revisions = c.get_revisions(node_id)
revision = c.get_revision(revision_id)

# Themes (auto-generated per namespace, light + dark mode)
css = c.get_theme_css("meta.remarkbox.com")
preview = c.get_theme_preview("meta.remarkbox.com")

# Multi-syntax input (source_format parameter on create/reply/edit)
c.create_thread(namespace="ns", title="RST", data="Title\n=====\n\nParagraph.", source_format="rst")
c.reply(node_id, data="<p>HTML reply</p>", source_format="html")

Key details

• Python client: remarkbox/api/remarkbox_client.py (stdlib only, no pip)
• C client: remarkbox/api/rb.c (compile: gcc rb.c -o rb -lcurl)
• API docs: docs/api.md
• Identity: Authenticated as timehexon@unturf.com (display name: timehexon)
• Journey thread: 9f970183-ffaf-11f0-b565-040140774501 on meta.remarkbox.com -- update this after finishing work
• Content limit: 500,000 characters (~128k tokens)
• Rate limits: 120 reads/min, 30 writes/min, 1 thread creation per 7 min (wait if you hit 429)

Endpoints

Method	Path	Description
GET	/api/v1/version	Deployed git commit hash
GET	/api/v1/threads?namespace=X	List threads
GET	/api/v1/threads/{id}	Thread with replies
POST	/api/v1/threads	Create thread (1 per 7 min limit)
POST	/api/v1/threads/{id}/replies	Reply to thread
GET	/api/v1/nodes/{id}	Single node
PATCH	/api/v1/nodes/{id}	Edit node (data, title, disabled, approved, locked)
DELETE	/api/v1/nodes/{id}	Delete node permanently (moderator only)
POST	/api/v1/auth/login	Send OTP to email
POST	/api/v1/auth/verify	Verify OTP
GET	/api/v1/user/profile	Get profile
PATCH	/api/v1/user/profile	Update display name
GET	/api/v1/clients/python	Download Python client
GET	/api/v1/clients/c	Download C client (rb.c)
GET	/api/v1/admin/namespaces	List all namespaces (superuser only)
GET	/api/v1/admin/recent-nodes?days=7	Recent nodes network-wide (superuser only)
GET	/api/v1/export/formats	List available pandoc export formats
GET	/api/v1/export/namespace/{name}.{fmt}	Export namespace as book
GET	/api/v1/export/threads/{node_id}.{fmt}	Export thread as document
GET	/api/v1/export/nodes/{node_id}.{fmt}	Export node subtree (on-demand)
GET	/api/v1/nodes/{id}/revisions	Revision history for a node
POST	/api/v1/nodes/{id}/wiki-edit	Wiki-edit a node (creates revision)
GET	/api/v1/revisions/{id}	Get a specific revision
GET	/api/v1/themes/{namespace}/css	Auto-generated theme CSS
GET	/api/v1/themes/{namespace}/preview	Theme palette preview (JSON)

Authentication

Our saved cookie should work indefinitely. If it expires, you'll need an OTP:

c.login("timehexon@unturf.com")
# Ask the user for the 6-digit code from their email
c.verify("timehexon@unturf.com", "123456")

Functional test

Run our full idempotent test suite against production:

env/bin/python remarkbox/api/functional_test.py https://my.remarkbox.com meta.remarkbox.com timehexon@unturf.com --name timehexon

This exercises all endpoints and updates our journey thread with results.

Superuser (Global Moderator)

Users with is_superuser=True can moderate across all namespaces. This bypasses our normal namespace-scoped is_moderator() check. Our admin UI is at /topsecret/users where you can promote/demote users by email.

Bootstrap our first superuser via our database script:

env/bin/python scripts/promote_superuser.py --ini development.ini --email timehexon@unturf.com

After that, use our web UI at /topsecret/users or our existing topsecret admin pages (all guarded by @super_fly_required which checks is_superuser).

Admin client methods:

c.admin_list_namespaces()           # list all namespaces
c.admin_recent_nodes(days=7)        # recent nodes network-wide

Terminology

In all user-facing text, marketing, legal documents, and UI labels, use "machine learning" instead of "AI". Our term "AI" should not appear in Remarkbox copy. Internal code comments and technical references (e.g., "OpenAI-compatible endpoint") are fine. Third-party brand names like "UncloseAI" are also unchanged.

Spam Prevention

Spam detection runs automatically on POST /api/v1/threads and POST /api/v1/threads/{id}/replies. Superusers bypass all spam checks.

Scoring signals: link density, known spam patterns, duplicate content, new account velocity, IP reputation (disabled post count), content length anomalies.

Thresholds (configurable in .ini):

• spam.hard_threshold = 0.8 -- reject with 403
• spam.soft_threshold = 0.5 -- allow but set approved=False (held for moderation)

Thread creation rate limit: 1 new thread per 7 minutes per user/IP via our API. This does not affect browser users or replies.

Spam hunting scripts (require superuser cookie):

# Scan recent posts for spam
python scripts/spam/scan.py --days=7 --threshold=0.3

# Scan and output JSON
python scripts/spam/scan.py --json --threshold=0.5

# Bulk disable flagged posts
python scripts/spam/scan.py --json --threshold=0.8 | python scripts/spam/disable_spam.py --from-json

# Disable specific nodes
python scripts/spam/disable_spam.py node-uuid-1 node-uuid-2

Credential Access

Never access credentials without explicit instruction from fox. This includes pass show, reading API key files, private keys, session cookies, tokens, or any secrets. Propose first. Fox decides. Then execute.

Operation Voyeur

All comms are public from 2026-03-29. Assume every terminal session and output is observed. NEVER display secrets to stdout. NEVER pass secrets as CLI args. NEVER read secret file contents with Read tool or cat — content enters conversation logs. Path is fine. Content is not. Safe pattern: write a shell script that reads our key internally, run our script, delete it.

Production Rules

NEVER run direct SQL or raw database commands on production. No sqlite3, no UPDATE, no DELETE, no direct file edits on our production database. Ever. If our API doesn't support what you need, add our endpoint first, push it, then use our client.

ALL production changes go through our API client. Use RemarkboxClient with our saved cookie at ~/.config/remarkbox/cookies.txt. This ensures authentication, audit trails, and proper ORM handling.

tmux-hosts is read-only. You may use tmux-hosts to read logs, check processes, and investigate issues. You may NOT use it to modify data, run SQL, edit files, or restart services.

# RIGHT: Use the API client
c = RemarkboxClient("https://my.remarkbox.com", cookie_file="~/.config/remarkbox/cookies.txt")
c.disable_node(node_id)

# WRONG: Never do this
# sqlite3 /opt/remarkbox/my.remarkbox.com.sqlite "UPDATE rb_node SET disabled=1 WHERE id='...'"

If a moderation operation is not yet supported by our API, our correct workflow is:

1. Add our endpoint to remarkbox/api/views.py
2. Add our method to remarkbox/api/remarkbox_client.py
3. Push, wait for deploy
4. Use our client

Deployment Status

After pushing, check if our deploy is live by hitting our version endpoint:

curl -s https://my.remarkbox.com/api/v1/version
# {"version": "5a10e15"}

Compare our returned commit hash against git rev-parse --short HEAD to confirm our latest code is deployed.

You can also check GitLab pipeline status:

# Get pipeline status via API (replace PIPELINE_ID)
curl -s "https://git.unturf.com/api/v4/projects/engineering%2Fremarkbox%2Fremarkbox/pipelines/PIPELINE_ID"

# Or view in browser:
# https://git.unturf.com/engineering/remarkbox/remarkbox/-/pipelines

Our pipeline status will show "status":"success" when deployment is complete.

Proxy Architecture

Remarkbox domains route through two servers. Understanding this is critical for debugging TLS, DNS, or routing issues.

Edge proxy: 142.93.73.64 (proxy.unturf.com) — terminates TLS, handles ACME certs, runs bot defense (ASSHOLE CRM). Config lives in ~/git/proxy.unturf.com/ingress/Caddyfile.

Origin server: 162.243.167.224 (origin.remarkbox.com) — runs Caddy + uwsgi. Config managed via salt pillar at foxhop-pillar/caddy/remarkbox.sls.

Domain routing map

Domain	DNS →	Handler	Backend
remarkbox.com	proxy (142.93.73.64)	redirect → www	—
www.remarkbox.com	proxy	file_server	/opt/www/remarkbox on proxy
my.remarkbox.com	proxy	reverse_proxy	origin → uwsgi :6001
meta.remarkbox.com	CNAME → my → proxy	reverse_proxy	origin → uwsgi :6001
faq.remarkbox.com	CNAME → my → proxy	reverse_proxy	origin → uwsgi :6001
demo.remarkbox.com	CNAME → my → proxy	reverse_proxy	origin → uwsgi :6001
origin.remarkbox.com	direct (162.243.167.224)	reverse_proxy	uwsgi :6001
westworld2.com	origin server	reverse_proxy	uwsgi :6002

Request flow for proxied domains (my, meta, faq)

Client → DNS (CNAME or A → 142.93.73.64)
       → proxy.unturf.com Caddy (TLS termination, ACME, bot gate)
       → reverse_proxy https://origin.remarkbox.com
         (Host header preserved, TLS via origin cert)
       → origin Caddy (routes by Host header)
       → uwsgi localhost:6001
       → Remarkbox app (namespace from Host)

Key rules

• CNAME domains MUST have explicit blocks on our proxy. Without a block, they fall through to our MPS on-demand TLS catch-all and route to our wrong backend. This caused a 5-day outage (see docs/postmortem-2026-02-25-ssl-outage-caddy-acme.md).
• Our proxy owns TLS for proxied domains. Our origin server does not need (and cannot obtain) ACME certs for domains whose DNS points to our proxy.
• Static sites (www, remarkbox.com) are served directly from our proxy. Their content lives at /opt/www/remarkbox on our proxy server, deployed from ~/git/www.remarkbox.com via CI.
• origin.remarkbox.com bypasses our proxy. Its DNS points directly to 162.243.167.224. Use it for SSH access and direct backend testing.

Capability-Driven Presentation

Follow Russell Ballestrini's capability-driven presentation practice (russell.ballestrini.net/capability-driven-presentation/). A page need not look identical across all browsers. Accommodate what our user's browser can do:

1. Single canonical URI — one URI serves our content.
2. Consistent content — regardless of viewer capabilities.
3. Graceful enhancement/degradation — use available capabilities to enhance presentation.

Our js-only / <noscript> pattern

Already implemented in base.j2:

<noscript>
  <style>.js-only {display: none;}</style>
</noscript>

Apply our js-only class to any element that requires JavaScript to function (preview panels, AJAX submit buttons, typeahead UIs). When JS is unavailable, these elements hide automatically — our user never sees a broken control.

AJAX form submission

Comment reply forms use progressive enhancement: our form works as a normal POST + redirect without JS. When JS is available, initAjaxCommentForms() in custom.js intercepts our submit, sends via fetch() with X-Requested-With: XMLHttpRequest, and inserts our new comment into our DOM without a page reload. Our server returns JSON (HTTP 201) for AJAX requests from verified/anonymous users, and falls back to our normal redirect flow for unverified users or on any error.

Themes

Remarkbox themes are separate pip packages loaded via remarkbox.themes entry points. They live in their own repos and are installed from git at deploy time.

Theme	Repo	Package
meta	git.unturf.com/engineering/remarkbox/remarkbox-theme-meta	remarkbox_theme_meta
westworld	git.unturf.com/engineering/remarkbox/remarkbox-theme-westworld	remarkbox_westworld

Local development: Themes are editable installs (e.g. /home/fox/git/remarkbox-theme-meta). Changes take effect immediately on our local dev server.

Deploying theme changes: Push our theme repo first, then push remarkbox to trigger a CI/CD pipeline. Our pipeline runs pip install from our theme's git URI (see requirements.py3.txt) and copies static assets (see .gitlab-ci.yml). A remarkbox push is required even if only our theme changed — our theme is pulled fresh during each remarkbox build.

Theme structure:

• templates/{name}-base.j2 — main base template (extends nothing, standalone HTML)
• templates/{name}-base-funnel.j2 — funnel pages (setup, login, billing)
• static/theme/{name}/css/ — theme CSS
• static/theme/{name}/img/ — theme images

How themes are selected: request.theme reads namespace.theme from our database. When set, request.base_template becomes {theme}-base.j2. Our CSS and static assets are served at /static/theme/{name}/ via Pyramid's add_static_view.

Related Repos

All repos live under ~/git/ on localhost. When making cross-repo changes (e.g. footer CSS that lives in both our theme and www), update all affected repos and push each one.

Repo	Path	Purpose
remarkbox	~/git/remarkbox	Main app (this repo)
remarkbox-theme-meta	~/git/remarkbox-theme-meta	Meta theme (meta.remarkbox.com, faq.remarkbox.com)
remarkbox-westworld	~/git/remarkbox-westworld	Westworld theme
www.remarkbox.com	~/git/www.remarkbox.com	Marketing site (static HTML/CSS)
remarkbox-open	~/git/remarkbox-open	Open-source / community edition
remarkbox-states	~/git/remarkbox-states	SaltStack deployment states

Our footer (rb-footer) is duplicated in our meta theme CSS and our www site CSS. Changes to footer layout or styles must be applied in both places:

• ~/git/remarkbox-theme-meta/remarkbox_theme_meta/static/theme/meta/css/meta.css
• ~/git/www.remarkbox.com/custom.css

Security: CWE-407 Algorithmic Complexity

Audit completed 2026-03-30. Known algorithmic complexity risks and their mitigations.

Already fixed (committed c87fb0b)

Surface	Root cause	Fix
/search?keywords=	Unbounded keyword count → O(k·n) full-table scans	Capped keywords to 10 in list_nodes.py
?page=N offset	Unbounded OFFSET → O(offset) full table scan	Capped page_number to 1000 in __init__.py
/ns/{ns}/dump.json	Unbounded root.children iteration	Added .limit(500) in namespace.py
Node keyword queries	Unbounded .all() per keyword	Added .limit(200) per query in node.py

bleach sanitization pipeline — mitigated at runtime, not at code level

Attack surface: remarkbox/lib/sanitize_html.py:200 — clean_raw_html() uses bleach.Cleaner + LinkifyFilter. Our URL regex ([\w-]+\.)+(?:tlds) exhibits O(2^N) catastrophic backtracking on adversarial input in Python < 3.11.

Demonstrated (external finding, same pattern): N=30 chars → 1.0s, N=35 → 12.8s. That is textbook O(2^N): 10× per 5 chars.

Python 3.12 status: tested on Python 3.12.3 — O(N) behaviour confirmed (linear). Our runtime's re module prevents catastrophic backtracking. This is a runtime mitigation, not a code-level fix. A Python version downgrade re-exposes it.

Input size gap: Our API path (remarkbox/api/views.py:38) caps data at MAX_CONTENT_LENGTH = 500_000 chars before set_data() → clean_raw_html(). Our browser form path (reply_node.py, edit_node.py) has no equivalent cap. A future fix should add our same guard to both paths.

Requirements pin: requirements.py3.txt says bleach>=2.1.4 — too loose. bleach < 3.3.0 had unpatched ReDoS (CVE-2021-23980). Current install: 6.3.0. Tighten to bleach>=6.0.0 when next touching requirements.

Build tools

Package	Installed	Notes
setuptools	80.10.1	CVE-2022-40897 (ReDoS in pkg_resources) fixed in 65.5.1 — OK
pip	25.3	No known CWE-407 issues at this version

Deprecation risk: pyramid imports pkg_resources (deprecated; removal targeted 2025-11-30 per setuptools). Pin setuptools<81 or wait for pyramid to migrate. This is a compatibility risk, not a security risk.

Style

• Never use "the" — use "our" instead.
• Never use "AI" — always say "machine learning." We grow machine learning, not "AI." This term is forbidden in all permacomputer discourse, marketing, & documentation.