ecdsa's Phase B emit at full secp256k1 width writes a 4–6 GB QECCOPS1
ops binary. The existing path — accumulate in a string-output port,
materialize via get-output-string, write — peaks RAM at 3× body
size (port internal buffer + Scheme string copy + write-binary-file
concat). A 6 GB body needs ~18 GB transient; OOMs a 16 GB QEMU guest.
This commit shifts emit-stream onto a constant-RAM file-port path
and fixes binary-correctness defects in the supporting primitives.
New primitives (mirrored across c/builtins.c + lumbda.py):
- open-binary-output-file path
Opens in "w+b" so the caller can seek back to rewrite a header.
- port-set-position! port offset
fseek absolute offset on a file port. emit-stream reserves a
16-byte placeholder header, streams the body, then seeks back to
byte 0 to rewrite the QECCOPS1 + n_ops u64 LE once n_ops is known.
- append-binary-file path data
Opens in "ab" and fwrite's the bytes through. Pairs with
write-binary-file so callers can land header + body in two writes
instead of (string-append header body).
- append-port-to-binary-file path port
Streams a string-output port's buffer to disk via fwrite without
materializing (get-output-string port). Lets callers keep their
existing string-output sink and avoid the body-size string copy
if they stay on string-port emit.
Binary-correctness fixes:
- bi_write_string to a file port used fputs, which calls strlen.
Binary payloads containing 0x00 truncated at the first null byte.
Switched the file-port branch to fwrite with the string's known
->len (same fix family as the earlier bi_get_output_string
strlen defect).
- bi_write_char per-byte fflush guarded to stdout only. With
millions of gate-bytes per second, flushing after every fputc to
a file port was a 100× slowdown. File ports buffer until close
or explicit flush-port — keep stdout's per-byte feedback path,
drop fflush on every file-port byte.
- port_write_str grows 1.5× past 256 MB instead of 2× throughout.
At realloc time the transient peak is old + new; 2× at 8 GB →
16 GB transient needs 24 GB. 1.5× bounds peak at 2.5× and keeps
multi-GB string-port workloads inside a 16 GB VM.
Tests: 88/88 c-test, 4/4 regression-named-let-leak, 205/205
functional, zoe-favorites all tiers. Binary roundtrip with embedded
nulls at 10/1000/100000 bytes passes byte-for-byte.
End-to-end: foxhop ecdsa DIALOG_GCD secp256k1 emit lands a 4.7 GB
binary at 322 MB peak RSS in 7:41 wall on a 16 GB QEMU guest.
russell@unturf.com
1731304ed8