whitepaper §6.6.4: Fix 3 soak results — GC bounded at 20k HTTP requests

Extended the HTTP-under-GC validation from the original 5,000
requests to 20,000 requests per cell. All four cells still hold:

  asm no-GC + snapshot   ~240 req/s   peak 96 KB    growth 0 KB
  asm GC    + snapshot   ~235 req/s   peak 120 KB   growth 0 KB
  asm no-GC + no snap    ~470 req/s   peak 185 MB   growth 185 MB
  asm GC    + no snap    ~450 req/s   peak 1,084 KB growth 972 KB

Cell 4 (GC + no snapshot) is the real validation target. 972 KB of
growth over 20,000 requests = one chunk filled once, after which
the collector cycles through reclaimed space. No monotonic leak,
no OOM, no crash. Naive mark-sweep is now a correct (not optimal)
allocator for long-running asm servers that don't manage arenas.

The paper's soak commentary explicitly calls this out and also
notes two remaining rough edges we've observed but not yet
debugged: (a) at 50k requests with the no-GC + no-snap case
saturating the 512 MB vcap right before cell 4 starts, cell 4's
server sometimes fails to initialize — looks process-environment
rather than GC, no clean explanation yet; (b) the hash-set
benchmark on the GC build still surfaces an occasional
unbound-variable error at ~1 MB/iter workloads.

Reproduce: make bench-gc-http defaults to 5k now; soak uses
`REQUESTS=20000 CONCURRENCY=16 bash tests/bench-gc-http.sh`.
This commit is contained in:
russell@unturf.com 2026-04-18 19:52:38 -04:00
parent 58025a37bc
commit d8dd4fd393
2 changed files with 613 additions and 653 deletions

File diff suppressed because it is too large Load diff

View file

@ -628,17 +628,17 @@ The reason we built the GC at all was to let long-running asm HTTP servers not l
.. table::
:widths: 38 14 14 14 20
============================== ========= ========= ========== =============
Config req/s baseline peak RSS growth KB
RSS KB KB
============================== ========= ========= ========== =============
asm no-GC + ``heap-snapshot`` ~300 100 104 **4**
asm GC + ``heap-snapshot`` ~330 120 124 **4**
asm no-GC + no snapshot ~360 96 45,812 **46,096**
asm GC + no snapshot ~410 116 1,088 **972**
============================== ========= ========= ========== =============
============================== ========= ========= =========== =============
Config req/s baseline peak RSS growth KB
RSS KB KB (20 000 req)
============================== ========= ========= =========== =============
asm no-GC + ``heap-snapshot`` ~240 96 96 **0**
asm GC + ``heap-snapshot`` ~235 120 120 **0**
asm no-GC + no snapshot ~470 264 185,000 **185,000**
asm GC + no snapshot ~450 112 1,084 **972**
============================== ========= ========= =========== =============
All four cells validate cleanly now:
All four cells validate cleanly now at **20,000 requests per cell** (up from the original 5,000):
- **Cells 1 and 2** show that on idiomatic code using ``heap-snapshot``, both binaries hold memory absolutely flat (~4 KB growth over 5,000 requests is normal VM noise). The GC build costs a small throughput overhead for a feature the snapshot pattern doesn't need.
@ -648,7 +648,11 @@ All four cells validate cleanly now:
**Precise-type dispatch was the fix.** Cell 4 was crashing at first GC until we replaced the conservative-scan-plus-sentinel-checks walker with a precise one. Every heap block's 8-byte header now carries an explicit type byte at bits 815 (see §6.6.5 below for the redesign), so ``gc_mark_drain``, ``gc_mark_env``, and the arena-escape scan dispatch on the type byte instead of guessing from block size. This eliminated the entire class of "24-byte env vs string" / "40-byte vector vs string" type-confusion bugs we'd been patching one-by-one.
**Honest read.** The GC build now succeeds at the "GC instead of snapshots" use case. ``heap-snapshot`` + ``heap-restore`` remain the idiomatic production pattern (they're cheaper per-request and portable across all tiers), but the GC is finally a correct fallback for code that doesn't manage arenas explicitly. Remaining rough edge: on very heavy sustained allocation workloads (hash-set benchmark at the ~1 MB/iter scale under the GC build) we still see the occasional unbound-variable error that points to a root-scan edge case the precise-type fix didn't completely close. Tracked as a follow-up.
**Honest read.** The GC build now succeeds at the "GC instead of snapshots" use case. ``heap-snapshot`` + ``heap-restore`` remain the idiomatic production pattern (they're cheaper per-request and portable across all tiers), but the GC is finally a correct fallback for code that doesn't manage arenas explicitly.
**Soak result.** At 20,000 HTTP requests × 16 concurrent clients, the GC build serves ~450 req/s with peak RSS of 1,084 KB — one heap chunk, steady state. 972 KB of growth represents the heap filling up exactly once after which the collector keeps reusing reclaimed space. This is the production validation we needed: naive mark-sweep is a correct, if not optimal, allocator for long-running asm servers that don't want to think about arena discipline.
Remaining rough edges: at larger scales (50k+ requests with 512 MB ``ulimit -v`` and the bump-only no-GC case saturating the cap right before cell 4 starts) we've seen cell 4 die at server startup; this appears to be a process-environment issue rather than a GC bug, but we don't have a clean explanation yet. The hash-set benchmark on the GC build under the ~1 MB-per-iteration scale also still surfaces an occasional unbound-variable error pointing to a root-scan edge case the precise-type fix didn't fully close. Both are tracked as follow-ups.
**Reproduce:** ``make bench-gc-http``. Tuning: ``REQUESTS=10000 CONCURRENCY=16 VCAP=524288 bash tests/bench-gc-http.sh``.