russell@unturf.com
88e16c0ce2
Each gpu-worker.lsp now listens on both wire-TCP (existing :8320) and HTTP/1.1+CORS (new :8321), sharing one handle-request dispatcher. Lets a tab on https://lumbda.com/playground/ POST to its own machine via http://localhost:8321/ — browsers permit localhost from HTTPS origins without TLS, so no proxy, no cert, no fox-owned infra required for the decentralized run-your-own-bend story. main() forks at startup: child runs http-run-loop on :8321, parent keeps existing run-loop on :8320. Adding a new op-head to handle-request exposes it over both transports automatically. Binary modes (BSHK/BCGB/BSCP/BSRT/BSB3) stay wire-only — they exist for native callers who already cache the binary locally; browser callers send S-expression recipes the worker dispatches the same way. Two latent defects fixed to make CPU-only and Python-tier hosts work: - vram-used-mib now file-exists? guards /usr/bin/nvidia-smi. Python tier's spawn-process-stdio raises FileNotFoundError on missing binary, not returning #f as the prior code expected, which crashed every worker on a CPU-only laptop. - fork-self return discriminated via (number? pid) not (eq? pid 0). Python tier's (eq? 0 #f) returns #t because == conflates int 0 with bool False; pre-existing run-loop has the same risk but C/asm tier (identity eq?) masks it for the production case. Phase 2 (server-side factory ops: compile uploaded .lsp recipes into .bin before bending — the foxhop champion-circuit workflow) deferred until authentication lands; today a worker on the public internet would let any caller occupy our GPU. Operational Caddy + DNS proposals in plans/bend-http-deploy.md cover the personal-remote-access endpoint chain (proxy.unturf.com edge → ai.foxhop.net Caddy → 3090-ai:8321) gated by trusted-IP allowlist — applied separately. Also codifies the playground "CSS Grid only, never flexbox" rule in CLAUDE.md: all www/ and wasm/ stylesheets are already grid-only; documenting the invariant so future edits don't drift. Tests: smoke-bend-http.sh — (ping)→(ok pong), unknown-op fallback, OPTIONS CORS preflight — all PASS. Wire path unchanged, verified round-trip via 8-digit-prefix framing.
231 lines
10 KiB
Markdown
231 lines
10 KiB
Markdown
# Agent Blackops — Lumbda repo
|
||
|
||
Agent blackops operates this repo — ml agent for fox/timehexon on our unsandbox/unturf/permacomputer platform. **Lumbda** names a language (home: lumbda.com); our repo directory and binaries still carry a historical name `lumbda` until a filesystem rename ships in a later phase.
|
||
|
||
## Identity
|
||
|
||
Full shard: `~/git/unsandbox.com/blackops/BLACKOPS.md`
|
||
|
||
## Rules
|
||
|
||
- I propose, fox decides. Unsure = ask. Can't ask = stop.
|
||
- No autonomous ops decisions. No destructive commands without explicit instruction.
|
||
- Fail-closed. Cleanup crew, not demolition.
|
||
- Check our time every session. Gaps carry information.
|
||
- DRY in context — single source of truth, no sprawl.
|
||
- Never say "AI" — always say "machine learning."
|
||
- Prefer "defect" over "bug."
|
||
|
||
## Orientation
|
||
|
||
```bash
|
||
date -u
|
||
pwd
|
||
git log --oneline -5
|
||
git status
|
||
```
|
||
|
||
Then ask fox about our mission.
|
||
|
||
## Documentation
|
||
|
||
- **A diagram beats 10,000 words.** — russell@unturf.com
|
||
- Architecture diagrams live in `docs/*.dot` (Graphviz DOT format)
|
||
- Generate PNGs: `make docs`
|
||
- Every implementation (Python, C, GNU asm) carries a dedicated architecture diagram
|
||
- When explaining architecture, draft or reference a dot diagram first
|
||
|
||
## Web styling — CSS Grid only, never flexbox
|
||
|
||
Every page in `www/` and `wasm/` (homepage, whitepaper, playground,
|
||
REPL, bend demo, 404) lays out multi-child regions with **CSS Grid**.
|
||
Flexbox is **banned** as a layout primitive. One layout language across
|
||
every page; no mode-switching in our head while we read or edit.
|
||
|
||
**Forbidden** anywhere in our CSS (source or generated):
|
||
|
||
- `display: flex`, `display: inline-flex`
|
||
- `flex-direction`, `flex-wrap`, `flex-flow`
|
||
- `flex-grow`, `flex-shrink`, `flex-basis`, shorthand `flex:`
|
||
- `order` (use grid-area / source order instead)
|
||
|
||
**Allowed** (works for grid too — keep on grid containers only):
|
||
|
||
- `gap`, `row-gap`, `column-gap`
|
||
- `align-items`, `justify-items`, `place-items`
|
||
- `align-content`, `justify-content`, `place-content`
|
||
- `align-self`, `justify-self`, `place-self`
|
||
|
||
**Patterns that look like they need flex but don't:**
|
||
|
||
- Horizontal toolbar → `display: grid; grid-auto-flow: column; gap: …`
|
||
- Tab row → same as above; sticky positioning composes fine
|
||
- Centered single child → `display: grid; place-items: center`
|
||
- Sidebar + main → `display: grid; grid-template-columns: auto 1fr`
|
||
- Wrapping chip cloud → `display: grid; grid-template-columns: repeat(auto-fit, minmax(N, max-content))`
|
||
|
||
Every source CSS file under `www/` and `wasm/{app,repl,dist,dist-repl}/`
|
||
opens with the banner `/* No flexbox. Every multi-child layout uses
|
||
CSS Grid. */`. Keep that banner intact when editing; add it when
|
||
introducing a new stylesheet.
|
||
|
||
**Audit before commit** when CSS changed:
|
||
|
||
```bash
|
||
grep -rn 'display:[[:space:]]*\(inline-\)\?flex\|flex-direction\|flex-wrap\|flex-grow\|flex-shrink\|flex-basis' www/ wasm/ && exit 1 || echo "grid-only OK"
|
||
```
|
||
|
||
A hit fails our audit. Convert to grid before committing.
|
||
|
||
## Implementations
|
||
|
||
| Impl | Path | Build | Test | REPL |
|
||
|------|------|-------|------|------|
|
||
| Python | `lumbda.py` | — | `make test` | `make repl` |
|
||
| C | `c/` | `make c-build` | `make c-test` | `make c-repl` |
|
||
| GNU asm | `asm/` | `make asm-build` | `make asm-test` | `make asm-repl` |
|
||
| All | — | — | `make test-all` | — |
|
||
|
||
## Bend — GPU dispatch primitive
|
||
|
||
`examples/cuda-fanout/` ships `(bend ...)` — runtime decides per call
|
||
whether to evaluate locally or ship to a CUDA worker over our wire
|
||
protocol. **Each worker listens on two ports out of the box:**
|
||
|
||
- **8320 — wire-TCP** (length-prefixed S-expressions + binary magic
|
||
`BSHK`/`BCGB`/`BSCP`/`BSRT`/`BSB3` blobs). The native path: used
|
||
by `bend.lsp`, asm clients, and anything that can open a raw
|
||
socket. Fastest. Binary mode is 150× faster than S-exp at 1M inputs.
|
||
- **8321 — HTTP/1.1 + CORS** (POST body is the S-expression, response
|
||
body is the result text). The browser path: lets a tab on
|
||
`https://lumbda.com/playground/` POST to `http://localhost:8321/`
|
||
via the browser's localhost-exception (no TLS required, no proxy
|
||
needed). Same `handle-request` dispatcher fires on both ports, so
|
||
every op-head ships once and lights up on both transports.
|
||
|
||
Workers run on any tier (`make gpu-worker LUMBDA={c,python,asm}`).
|
||
C tier ~9x faster than Python on small calls; binary mode equalizes
|
||
everything at huge calls. Asm tier hosts workers via raw
|
||
`pipe2 + fork + execve` syscalls — no libc, ~70 KB statically linked.
|
||
|
||
The integration with `www.foxhop.net/ecdsa/cuda/` (kickmix circuit
|
||
simulator, full upstream byte-parity at 9024 shots) is now live:
|
||
lumbda search loops can `(bend!-call '(cuda-sim-ops-bin path 141))`
|
||
to dispatch real-scale candidate scoring to a GPU worker. The Phase B
|
||
1-8 secp256k1 arithmetic landed on the foxhop side this session, so
|
||
the substrate has every piece it needs.
|
||
|
||
**Phase 2 (deferred until auth lands):** server-side factory ops
|
||
that take an `.lsp` recipe via HTTP, compile a `.bin` on the worker
|
||
filesystem, then bend it. Killer use case: upload a foxhop champion
|
||
ECDSA circuit recipe from the playground/REPL and get the result
|
||
back from a GPU cluster. Blocked on authentication — today a worker
|
||
without auth would let any caller occupy our GPU.
|
||
|
||
**Run your own bend** (the decentralized story we encourage): any
|
||
user runs `make gpu-worker LUMBDA=python` on their machine and
|
||
pastes `http://localhost:8321/` into the playground bend field.
|
||
CPU works (kernels fall back to host execution); GPU faster. No
|
||
fox-owned infra required.
|
||
|
||
## Test Suites
|
||
|
||
- Python unit/integration: `tests.py` (571 tests)
|
||
- C unit/integration/JIT/continuations/portal: `c/test.c` (83 tests)
|
||
- GNU asm unit/integration/functional: `asm/test.sh` (132 tests)
|
||
- Shared functional: `tests/functional.lsp` (189 tests, runs under Python + C)
|
||
- Cross-impl portal matrix: `tests/portal-cross-test.sh` (9 cells)
|
||
- Portal benchmark: `tests/portal-benchmark.sh` (timings + mismatch classification)
|
||
- Web benchmark: `tests/web-benchmark.sh` (all three impls + Python http.server + busybox)
|
||
- Total: **975 verified assertions** via `make test-all`
|
||
|
||
## Asm memory discipline — our heap never shrinks
|
||
|
||
Our asm implementation uses a bump allocator (r15). Every allocation
|
||
(`string-append`, `tcp-recv`, `make-pair`, `number->string`, etc.)
|
||
grows r15 monotonically. When r15 hits r13 (heap limit), `heap_grow`
|
||
mmaps ANOTHER 64 MB chunk. **We free nothing, ever.**
|
||
|
||
A long-running asm server leaks ~64 MB every few thousand requests
|
||
until it OOMs our machine. Two prior crashes on fox's machine taught
|
||
us this: 2026-04-16 (19.3 GB RSS) and 2026-04-17. Both times blackops
|
||
spawned an asm server in the background for testing and failed to
|
||
verify its absence before moving on. A first crash added this
|
||
discipline section; a second proved our discipline needed teeth.
|
||
Hence our MANDATORY checklist below.
|
||
|
||
**Shared-machine context:** other agents run on this box. An OOM
|
||
crash takes their state down too, not just mine. Rules below act as
|
||
belt, suspenders, AND parachute so that even if two safeguards fail,
|
||
our kernel itself backstops.
|
||
|
||
**MANDATORY pattern — every asm/lumbda test in a shell block:**
|
||
|
||
```bash
|
||
set -e # (1) fail-fast
|
||
ulimit -v 524288 # (2) KERNEL CAP: 512 MB virt
|
||
# process gets SIGKILL at cap, no matter what
|
||
trap 'pkill -9 -u "$USER" -f "examples/http-server|asm/lumbda" 2>/dev/null || true' \
|
||
EXIT INT TERM # (3) cleanup always fires
|
||
|
||
timeout 30 asm/lumbda < server.lsp & # (4) wall-clock ceiling
|
||
SPID=$!
|
||
# ... do the work (curl requests, measurements, etc.) ...
|
||
kill -9 $SPID 2>/dev/null; wait $SPID 2>/dev/null # (5) explicit cleanup
|
||
|
||
# (6) VERIFY a block stays clean before moving on
|
||
pgrep -u "$USER" -f 'asm/lumbda|examples/http-server' \
|
||
&& { echo "STRAGGLER"; exit 1; } || true
|
||
```
|
||
|
||
Six layers. Bypass any one; a next layer catches. Two crashes struck
|
||
when I ran only layers 3-5; a kernel cap (2) turns "if I forget" from
|
||
"fox reboots" into "my one rogue process dies at 512 MB without
|
||
touching shared RAM."
|
||
|
||
Additional rules:
|
||
|
||
- `ulimit -v` affects only a shell it runs in and its children, so it
|
||
cannot degrade anyone else's agents. Always set it before
|
||
backgrounding any Lumbda process.
|
||
- Bound iterations **inside a .lsp** (e.g. `*max-requests* = 50000`
|
||
in `examples/http-server.lsp`). Never raise for long-running tests.
|
||
- Use `pkill -u "$USER" -f <pattern>` not `pkill` alone — others may
|
||
have their own processes on this machine.
|
||
- Prefer FOREGROUND runs when possible: `timeout 10 asm/lumbda
|
||
< test.lsp` with a `.lsp` exiting on its own beats backgrounding.
|
||
- C carries Boehm GC via `GC_MALLOC`. Python carries Python's GC.
|
||
Asm carries neither. Risk scales with how long our asm process
|
||
lives.
|
||
- If no `ulimit` exists (some container setups), substitute
|
||
`systemd-run --user --scope -p MemoryMax=512M -- asm/lumbda ...`
|
||
as a cgroup-based cap.
|
||
|
||
## MOAD Scanner
|
||
|
||
**`~/git/unmoad.com/` detects MOAD defects in source code.** Run it on every change.
|
||
|
||
```bash
|
||
cd ~/git/unmoad.com && make all
|
||
./unmoad ~/git/lumbda/ # scan our entire repo
|
||
./unmoad ~/git/lumbda/asm/ # scan GNU asm only
|
||
./unmoad ~/git/lumbda/c/ # scan C only
|
||
./unmoad ~/git/lumbda/*.py # scan Python only
|
||
```
|
||
|
||
**MANDATORY before committing new code:** run `unmoad` on changed
|
||
files. Machine learning agents (blackops included) propagate MOAD-0001
|
||
by default. Our training data encodes O(N) linear scans as a norm. A
|
||
scanner catches what our weights miss.
|
||
|
||
Supported languages for this repo: Python, C, Scheme (.lsp), GNU asm (.s).
|
||
|
||
Key MOAD-0001 patterns our scanner catches:
|
||
- Python: `.count()`, `.index()`, `in list` inside loops
|
||
- C: `std::find()`, `strcmp()` inside loops
|
||
- Scheme: `(member)`, `(memq)`, `(assoc)` inside `(let loop)`, `(for-each)`, `(map)`
|
||
- GNU asm: `rep cmpsb` inside search loops
|
||
|
||
**Our commit history proves a need**: blackops wrote MOAD-0001 into
|
||
fresh code on April 13-14 despite carrying full MOAD context. Fixed
|
||
only after explicit audit on April 15. See whitepaper Section 14.
|