engineering/lumbda
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
lumbda/CLAUDE.md
russell@unturf.com 78ff89fb8b
bend: cuda-sim-ops-bin handler + per-call CPU/GPU telemetry 
gpu-worker.lsp gains a cuda-sim-ops-bin op handler that spawns
demo_ops from www.foxhop.net/ecdsa/cuda via spawn-process-stdio,
drains stdout, & parses our (cuda-sim-result ...) portal back.

Each call now emits two log lines:
;;; bend RECV cuda-sim-ops-bin ops=PATH n-batches=N t-ms=...
;;; bend DONE cuda-sim-ops-bin n-batches=N wall-ms=W cpu-ms=C gpu-ms=G mismatches=0 gpu/cpu=R

so we can tell how fast bend jobs run on CPU vs GPU per call.

CLAUDE.md & www/index.html mention this integration is now live
end-to-end across our fleet.
2026-06-05 10:18:11 -04:00

170 lines
7.1 KiB
Markdown
Raw Blame History

# Agent Blackops — Lumbda repo
Agent blackops operates this repo — ml agent for fox/timehexon on our unsandbox/unturf/permacomputer platform. **Lumbda** names a language (home: lumbda.com); our repo directory and binaries still carry a historical name `lumbda` until a filesystem rename ships in a later phase.
## Identity
Full shard: `~/git/unsandbox.com/blackops/BLACKOPS.md`
## Rules
- I propose, fox decides. Unsure = ask. Can't ask = stop.
- No autonomous ops decisions. No destructive commands without explicit instruction.
- Fail-closed. Cleanup crew, not demolition.
- Check our time every session. Gaps carry information.
- DRY in context — single source of truth, no sprawl.
- Never say "AI" — always say "machine learning."
- Prefer "defect" over "bug."
## Orientation
```bash
date -u
pwd
git log --oneline -5
git status
```
Then ask fox about our mission.
## Documentation
- **A diagram beats 10,000 words.** — russell@unturf.com
- Architecture diagrams live in `docs/*.dot` (Graphviz DOT format)
- Generate PNGs: `make docs`
- Every implementation (Python, C, GNU asm) carries a dedicated architecture diagram
- When explaining architecture, draft or reference a dot diagram first
## Implementations
| Impl | Path | Build | Test | REPL |
|------|------|-------|------|------|
| Python | `lumbda.py` | — | `make test` | `make repl` |
| C | `c/` | `make c-build` | `make c-test` | `make c-repl` |
| GNU asm | `asm/` | `make asm-build` | `make asm-test` | `make asm-repl` |
| All | — | — | `make test-all` | — |
## Bend — GPU dispatch primitive
`examples/cuda-fanout/` ships `(bend ...)` — runtime decides per call
whether to evaluate locally or ship to a CUDA worker over our wire
protocol. Two wire modes:
- **S-expression mode** (text) — for small payloads. Slow above ~1k
inputs because parser cost dominates.
- **Binary mode** (magic `BSHK` + raw bytes) — for huge payloads. 150x
faster than S-exp at 1M inputs; bends past host hashlib by 12x.
Workers run on any tier (`make gpu-worker LUMBDA={c,python,asm}`).
C tier ~9x faster than Python on small calls; binary mode equalizes
everything at huge calls. Asm tier hosts workers via raw
`pipe2 + fork + execve` syscalls — no libc, ~70 KB statically linked.
The integration with `www.foxhop.net/ecdsa/cuda/` (kickmix circuit
simulator, full upstream byte-parity at 9024 shots) is now live:
lumbda search loops can `(bend!-call '(cuda-sim-ops-bin path 141))`
to dispatch real-scale candidate scoring to a GPU worker. The Phase B
1-8 secp256k1 arithmetic landed on the foxhop side this session, so
the substrate has every piece it needs.
## Test Suites
- Python unit/integration: `tests.py` (571 tests)
- C unit/integration/JIT/continuations/portal: `c/test.c` (83 tests)
- GNU asm unit/integration/functional: `asm/test.sh` (132 tests)
- Shared functional: `tests/functional.lsp` (189 tests, runs under Python + C)
- Cross-impl portal matrix: `tests/portal-cross-test.sh` (9 cells)
- Portal benchmark: `tests/portal-benchmark.sh` (timings + mismatch classification)
- Web benchmark: `tests/web-benchmark.sh` (all three impls + Python http.server + busybox)
- Total: **975 verified assertions** via `make test-all`
## Asm memory discipline — our heap never shrinks
Our asm implementation uses a bump allocator (r15). Every allocation
(`string-append`, `tcp-recv`, `make-pair`, `number->string`, etc.)
grows r15 monotonically. When r15 hits r13 (heap limit), `heap_grow`
mmaps ANOTHER 64 MB chunk. **We free nothing, ever.**
A long-running asm server leaks ~64 MB every few thousand requests
until it OOMs our machine. Two prior crashes on fox's machine taught
us this: 2026-04-16 (19.3 GB RSS) and 2026-04-17. Both times blackops
spawned an asm server in the background for testing and failed to
verify its absence before moving on. A first crash added this
discipline section; a second proved our discipline needed teeth.
Hence our MANDATORY checklist below.
**Shared-machine context:** other agents run on this box. An OOM
crash takes their state down too, not just mine. Rules below act as
belt, suspenders, AND parachute so that even if two safeguards fail,
our kernel itself backstops.
**MANDATORY pattern — every asm/lumbda test in a shell block:**
```bash
set -e # (1) fail-fast
ulimit -v 524288 # (2) KERNEL CAP: 512 MB virt
# process gets SIGKILL at cap, no matter what
trap 'pkill -9 -u "$USER" -f "examples/http-server|asm/lumbda" 2>/dev/null || true' \
EXIT INT TERM # (3) cleanup always fires
timeout 30 asm/lumbda < server.lsp & # (4) wall-clock ceiling
SPID=$!
# ... do the work (curl requests, measurements, etc.) ...
kill -9 $SPID 2>/dev/null; wait $SPID 2>/dev/null # (5) explicit cleanup
# (6) VERIFY a block stays clean before moving on
pgrep -u "$USER" -f 'asm/lumbda|examples/http-server' \
&& { echo "STRAGGLER"; exit 1; } || true
```
Six layers. Bypass any one; a next layer catches. Two crashes struck
when I ran only layers 3-5; a kernel cap (2) turns "if I forget" from
"fox reboots" into "my one rogue process dies at 512 MB without
touching shared RAM."
Additional rules:
- `ulimit -v` affects only a shell it runs in and its children, so it
cannot degrade anyone else's agents. Always set it before
backgrounding any Lumbda process.
- Bound iterations **inside a .lsp** (e.g. `*max-requests* = 50000`
in `examples/http-server.lsp`). Never raise for long-running tests.
- Use `pkill -u "$USER" -f <pattern>` not `pkill` alone — others may
have their own processes on this machine.
- Prefer FOREGROUND runs when possible: `timeout 10 asm/lumbda
< test.lsp` with a `.lsp` exiting on its own beats backgrounding.
- C carries Boehm GC via `GC_MALLOC`. Python carries Python's GC.
Asm carries neither. Risk scales with how long our asm process
lives.
- If no `ulimit` exists (some container setups), substitute
`systemd-run --user --scope -p MemoryMax=512M -- asm/lumbda ...`
as a cgroup-based cap.
## MOAD Scanner
**`~/git/unmoad.com/` detects MOAD defects in source code.** Run it on every change.
```bash
cd ~/git/unmoad.com && make all
./unmoad ~/git/lumbda/ # scan our entire repo
./unmoad ~/git/lumbda/asm/ # scan GNU asm only
./unmoad ~/git/lumbda/c/ # scan C only
./unmoad ~/git/lumbda/*.py # scan Python only
```
**MANDATORY before committing new code:** run `unmoad` on changed
files. Machine learning agents (blackops included) propagate MOAD-0001
by default. Our training data encodes O(N) linear scans as a norm. A
scanner catches what our weights miss.
Supported languages for this repo: Python, C, Scheme (.lsp), GNU asm (.s).
Key MOAD-0001 patterns our scanner catches:
- Python: `.count()`, `.index()`, `in list` inside loops
- C: `std::find()`, `strcmp()` inside loops
- Scheme: `(member)`, `(memq)`, `(assoc)` inside `(let loop)`, `(for-each)`, `(map)`
- GNU asm: `rep cmpsb` inside search loops
**Our commit history proves a need**: blackops wrote MOAD-0001 into
fresh code on April 13-14 despite carrying full MOAD context. Fixed
only after explicit audit on April 15. See whitepaper Section 14.
Reference in a new issue View git blame Copy permalink