# `(go-gpu …)` — S-expression coroutines on remote GPU

Sketch of the architecture fox proposed: Go-keyword-style coroutines
that ship S-expressions to a remote GPU box for execution. Pattern
analogous to vLLM's inference RPC, but for arbitrary lumbda
expressions backed by a registered CUDA primitive.

## API shape

```scheme
;; fire-and-get-future
(define f (go-gpu (cuda-shake-fanout inputs 32)))
(define result (await f))

;; fan out across N GPU workers in parallel
(define fs (go-gpu-many
             (cuda-shake-fanout batch-1 32)
             (cuda-shake-fanout batch-2 32)
             (cuda-shake-fanout batch-3 32)))
(define results (await-all fs))

;; streaming (long-running kernels emit chunks back)
(go-gpu-stream (cuda-sim-batch ops-bin 141)
  (lambda (chunk) (process-partial chunk)))
```

## Runtime architecture

```
                          ┌──── coordinator (any host) ──────┐
                          │  - load tracker per GPU worker   │
                          │  - dispatch queue                │
                          │  - future registry               │
                          └────────────┬─────────────────────┘
                                       │ TCP / S-exp message
                ┌──────────────────────┴───────────────────────┐
                │                                              │
        ┌───────▼───────┐                              ┌───────▼───────┐
        │ gpu-worker    │                              │ gpu-worker    │
        │ ai.foxhop.net │                              │ 3090-ai.fox.n │
        │ RTX 4090      │                              │ RTX 3090      │
        └───────────────┘                              └───────────────┘
                │                                              │
                ▼                                              ▼
        spawn local binary                              spawn local binary
        (cuda-shake-fanout,                             (cuda-shake-fanout,
         cuda-sim-batch, etc.)                           cuda-sim-batch, etc.)
```

Reuses lumbda's existing fleet TCP primitive
(`ecdsa/lumbda/fleet/coordinator-fleet.lsp` pattern). The only new
piece is **GPU capability advertising** in the worker handshake +
**load-aware dispatch** in the coordinator.

## Wire protocol (S-exp over TCP)

Request (coordinator → worker):
```
(go-gpu-request
  (id 17)
  (op cuda-shake-fanout)
  (args
    (inputs ("00" "01" "deadbeef"))
    (output-bytes 32)))
```

Response (worker → coordinator → awaiting future):
```
(go-gpu-response
  (id 17)
  (status ok)
  (result
    (hashes ("abc..." "def..." "fed...")))
  (timing-ms (work 0.2) (wire 1.4) (queue 0.0)))
```

Streaming response (long kernels):
```
(go-gpu-stream-chunk (id 17) (seq 0) (data ...))
(go-gpu-stream-chunk (id 17) (seq 1) (data ...))
(go-gpu-stream-end   (id 17) (status ok))
```

## Capability advertising (worker startup)

```
(gpu-worker-hello
  (host 3090-ai.foxhop.net)
  (vram-gb 24)
  (cuda-cores 10496)
  (binaries
    (cuda-shake-fanout (path /usr/local/bin/cuda-shake-fanout))
    (cuda-sim-batch    (path /usr/local/bin/cuda-sim-batch))
    (cuda-keccakp      (path /usr/local/bin/cuda-keccakp)))
  (max-concurrent 4))
```

Coordinator only routes operations whose `op` name appears in some
worker's `binaries` list. Unknown ops error early at submit time.

## Scheduling

Coordinator keeps per-worker state:

```
(worker-state
  (host 3090-ai.foxhop.net)
  (load 0.5)                ;; fraction of max-concurrent in use
  (vram-free-mb 23900)
  (last-heartbeat 1717530000))
```

`go-gpu` form picks the worker with lowest `load × (vram-needed /
vram-free)` and routes there. Future stored in registry keyed by
`(coordinator-issued-id worker-host)`. When response lands, future
fires.

## Failure semantics

- Worker disconnect mid-request → future raises `gpu-worker-lost`
- Worker reports OOM → coordinator marks worker `vram-tight`, retries
  on another worker if available, else raises `gpu-out-of-memory`
- Network hiccup → coordinator retries up to 2× with exponential
  backoff, then raises `gpu-timeout`

`await` MUST be wrapped in error handlers same as any I/O primitive.

## Per-tier integration (≈ 30 LoC each)

Each lumbda tier registers four primitives in its dispatch table:

| primitive | shape | what it does |
|-----------|-------|--------------|
| `go-gpu` | special form (delayed eval of body) | serializes op + args → submits to coordinator → returns opaque future handle |
| `await` | (await future) | blocks current continuation until response lands |
| `await-all` | (await-all (f1 f2 ...)) | blocks on every future; returns list of results |
| `go-gpu-stream` | (go-gpu-stream body callback) | spawns + invokes callback per chunk |

Coordinator + worker stay outside lumbda's core: they're standalone
lumbda scripts (or, eventually, the same role lumbda's fleet plays)
launched at boot per host.

## Comparison to vLLM

vLLM is HTTP/REST with token-stream responses; lumbda's
`go-gpu` is S-expression over TCP with portal-style responses. Same
shape: client submits a job, server runs on GPU, client receives
result asynchronously. Key differences for lumbda:

| dimension | vLLM | go-gpu |
|-----------|------|--------|
| wire | HTTP/JSON | TCP/S-exp |
| model loading | per-server, persistent | per-binary, spawned per request (today); persistent daemon per binary (future) |
| streaming | token-by-token | chunk-by-chunk |
| failure | HTTP status + retry | typed Scheme conditions |
| scheduling | one worker per model | one binary per capability; load-aware fanout |

## What's needed to ship a working v1

1. **`gpu-worker-daemon.lsp`** — Scheme script that listens on TCP,
   matches incoming op names against local binaries, spawns binary
   with input portal, returns result portal.
2. **`gpu-coordinator-daemon.lsp`** — Scheme script that tracks
   workers + their advertised capabilities, accepts client `go-gpu`
   submissions, routes to least-loaded capable worker, holds futures.
3. **`go-gpu.lsp`** — client-side wrapper Scheme that submits & awaits.
4. **Capability advertise file** — `~/.lumbda/gpu-capabilities` per
   host, lists local binaries the daemon should advertise.

Each piece is ~50-100 LoC. Total v1 ships in one focused day.

## Open questions for fox

- **Q1**: should the coordinator be a separate process, or should
  every lumbda runtime double as coordinator (mesh)?
- **Q2**: should `go-gpu` be a special form that captures free
  variables in the body, or a regular function call that takes only
  serializable args? (Special form is nicer but needs closure
  shipping; function call is portable.)
- **Q3**: do we want pre-warmed daemon binaries (kill the 200 ms
  cuda-ctx-init per spawn) or stick with per-spawn for simplicity?
  Bench data informs this — see `bench.py`.
- **Q4**: should `await` be implicit at the end of a `(let ...)`
  scope (Go-style scope-bound spawn), or always explicit?
