whitepaper: §11.5 Portal over HTTP — state transfer between machines

New subsection documenting the portal-over-HTTP demo shipped in
commit 06b93c5. Closes the Future Work item about continuation-
style payloads traversing HTTP endpoints.

Covers:
- The server endpoint that returns an S-expression portal body as
  HTTP/1.0 content-type application/scheme.
- The client that strips HTTP headers, splits by newline, evals
  each form. ~90 lines of portable Scheme on each side.
- 3×3 server/client matrix: every runtime hosts, every runtime
  consumes. Nine cells green.
- The eval-to-global-env semantic alignment (a two-line fix in
  Python and C that matches asm's long-standing bi_eval behavior).

Closing passage frames the four scopes of feedback as now all
running demos:
- within process: call/cc
- across process: portal files
- across implementations: S-expression serialization
- across machines: sockets (HTTP, RPC, raw TCP)

Removed the now-redundant "Portal over HTTP" item from §13 Future
Work. Added "Continuation-passing over HTTP" as its successor —
moving live continuations (not just bindings) via call/cc + JSON
portal + TCP.

The existing §11.5 "heap-snapshot" moves to §11.6.
This commit is contained in:
russell@unturf.com 2026-04-17 14:05:01 -04:00
parent 06b93c588a
commit 18e68f8838
2 changed files with 707 additions and 475 deletions

File diff suppressed because one or more lines are too long

View file

@ -839,7 +839,44 @@ Each relay hop costs ~650 µs (one full TCP round-trip + context switches on the
The result is not a performance story — it is a composition story. S-expressions are the envelope and the payload. A 22 KB binary can be a backend, a relay, a client, or any point in a chain; the protocol needs no separate definition because the protocol IS the language.
11.5 heap-snapshot: The Arena Escape Hatch
11.5 Portal over HTTP: State Transfer Between Machines
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The HTTP server carries request bytes. The portal serializes machine state to bytes. Combine them: a node exposes its state as an HTTP endpoint, another node pulls that endpoint down and resumes. This was the first item in §13 Future Work — it is now a running demo.
``examples/portal-http-server.lsp`` holds some state (integer counter, list, string, the result of fib(30)) and answers ``GET /portal`` with an S-expression portal body — literally a sequence of ``(define ...)`` forms.
``examples/portal-http-client.lsp`` dials the endpoint, strips the HTTP headers, splits the body by newline, and for each non-empty, non-comment line calls ``(eval (read-from-string line))``. The remote bindings become local.
::
;; server-side (excerpt)
(define (portal-body)
(string-append
"(define counter " (number->string counter) ")\n"
"(define my-fib " (number->string my-fib) ")\n" ...))
;; client-side (excerpt)
(define resp (tcp-recv sock 65536))
(define body (body-of resp))
(eval-all-lines body) ; iterates, calls (eval (read-from-string line))
(display counter) ; => the server's counter value
(display my-fib) ; => 832040
**3×3 matrix green:** Python/C/asm in any role — server or client or both — exchange state correctly. All nine combinations verified. ``examples/portal-http-client.lsp`` is 90 lines of portable Scheme; the full round-trip uses only the six ``tcp-*`` primitives plus ``read-from-string``, ``eval``, and standard string manipulation.
The client needs ``eval`` semantics that install define-bindings in the global env regardless of the dynamic scope of the ``eval`` call. Python and C originally evaluated the ``eval`` result in the caller's env, which worked at top level but silently failed inside a helper function. A two-line fix in each (``env = env.g`` in Python's ``leval``, ``env = env->global`` in C's SYM_EVAL branch) aligns both with asm's long-standing ``bi_eval`` behavior.
This is the closing demonstration of "feedback is all you need" extended across all four scopes. A continuation-style value — here, a set of bindings — travels:
- within a process via ``call/cc``
- across processes on the same host via portal files
- across implementations that share no binary compatibility via S-expression serialization
- across machines via sockets (HTTP, RPC, raw TCP)
The wire protocol and the language are the same artifact. A 22 KB binary can be any node in any chain of any scope.
11.6 heap-snapshot: The Arena Escape Hatch
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The asm bump allocator has no GC. Every ``string-append``, ``tcp-recv``, ``make-pair``, or similar per-request allocation grows r15. Over a long-running server that is an unbounded leak — an incident on 2026-04-16 drove an asm server to 19.3 GB RSS before being killed.
@ -937,8 +974,8 @@ This is the permacomputer obligation: infrastructure that renews itself. Code th
13. Future Work
----------------
- **Portal over HTTP**: With sockets now in place, a portal file becomes a payload. Serve ``portal-save`` output at an HTTP endpoint, have another node ``portal-resume`` it. Continuations as messages, permacomputer nodes as consumers.
- **WebSocket / bidirectional**: HTTP/1.0 is request-response; a persistent socket loop with framing brings full-duplex feedback.
- **Continuation-passing over HTTP**: §11.5 moves *bindings* over HTTP. The next step is moving a live *continuation* — serialize it via ``call/cc`` + JSON portal, transmit, resume on the remote VM. Makes any TCP endpoint a trampoline target.
- **GPU lambda execution**: Map/reduce on CUDA for data-parallel Scheme (Phase 1), trampolining for recursive lambdas (Phase 2), interaction combinators for massive parallelism (Phase 3)
- **Copying GC in asm**: ``heap-snapshot`` is an escape hatch. A mark-and-copy collector would remove the sharp edge for general programs without forcing the programmer to reason about lifetimes.
- **Concurrent accept loop (asm)**: Currently single-threaded. A pre-forked worker model or ``SO_REUSEPORT`` pool would multiply throughput without changing the Scheme code.