whitepaper §6.5: asm native hash-set vs portable — 15-21x speedup
New subsection documents the intra-asm benchmark: same chained-hash algorithm, same 64 buckets, same hash function; only difference is whether the bucket walk runs in Scheme (tree-walker) or in asm (straight-line machine code). Numbers (N=5,000 integers, i5-8350U): insert 129 ms -> 6 ms (21x) hit-lookup 124 ms -> 8 ms (15x) miss-lookup 238 ms -> 12 ms (19x) Also registers the bench in §6 reproducibility list.
This commit is contained in:
parent
afb5616843
commit
fc92743b8b
2 changed files with 1337 additions and 1031 deletions
File diff suppressed because one or more lines are too long
|
|
@ -350,6 +350,7 @@ Folding only applies when all operands are compile-time constants & the function
|
|||
|
||||
- §6.1-§6.3 Python internal (tree-walker vs bytecode): ``make bench``
|
||||
- §6.4 Three impls head-to-head: ``make bench-3way``
|
||||
- §6.5 asm native hash-set vs portable Scheme hash-set: ``make bench-hashset``
|
||||
- §7.2 Cross-impl portal matrix: ``make bench-portal-cross``
|
||||
- §7.5 Portal save+load timings: ``make bench-portal``
|
||||
- §11.3 HTTP server vs busybox / python http.server: ``make bench-web``
|
||||
|
|
@ -442,6 +443,32 @@ C wins every workload on this hardware. Its bytecode compiler + explicit frame s
|
|||
|
||||
**asm's remaining bottleneck is allocation, not lookup.** Profiling ``sum-to(1M)`` shows ~170 MB RSS — each tail call through ``apply_closure`` + ``env_define`` allocates 24 bytes per parameter (sym / val / parent), twice per ``loop`` iteration, for 48 MB total before the three ``heap_grow`` events that follow. A future optimization candidate is per-frame batched allocation (``8 + 16N`` bytes once per call instead of ``24N``), or env-cell in-place reuse for self-tail-calls. An inline cache for env lookups (ported from Python's VM) turns out to help less than anticipated because asm's env chains are typically only 2 deep and each step is a pointer dereference; measured upper bound is ~5%. The ~3× gap to C is mostly the absence of a bytecode layer and the per-call env allocation — not a lookup-path problem.
|
||||
|
||||
6.5 Native Container Primitives: Moving Hot Loops Into Asm
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
The proof-netspace server (§7) needs a hash-set to dedupe proof hashes. asm had no native hash-set, so d88149a shipped a portable one in Lumbda itself: a fixed-size vector of alists, with the bucket walk written in Scheme. It works in every tier and matches the usual chained-hash-table algorithm. But the entire inner loop — ``modulo`` for bucket index, ``car`` / ``cdr`` / ``=`` to walk the chain, ``cons`` to insert — runs through asm's tree-walker, one Scheme AST node at a time.
|
||||
|
||||
A native hash-set moves the same algorithm into asm: ``bi_hash_set_add`` is one ``hash_value`` call, one ``AND`` mask, one bucket-slot load, one ``hs_chain_find`` loop over raw tagged pairs, and one ``make_pair`` on miss. No environment lookups, no frame allocation, no tree-walk dispatch — the entire hot path is straight-line machine code.
|
||||
|
||||
.. table::
|
||||
:widths: 32 22 22 24
|
||||
|
||||
============================= =============== =============== ===========
|
||||
Phase (N=5,000 integers) portable (ms) native (ms) Speedup
|
||||
============================= =============== =============== ===========
|
||||
insert-N 129 6 21×
|
||||
hit-lookup-N 124 8 15×
|
||||
miss-lookup-N 238 12 19×
|
||||
============================= =============== =============== ===========
|
||||
|
||||
Same hash function (key value for ints, djb2 for strings, identity for everything else), same 64-bucket chaining, same load factor. The 15–21× factor is **purely the cost of interpretation on the bucket walk**: each Scheme-level ``car`` pays a tag check plus an eval dispatch, while the asm-level loop is two memory loads and a compare per step.
|
||||
|
||||
This matters beyond one data structure. Every container that lives in Lumbda-the-language-on-asm-the-interpreter pays the same overhead. For a server that processes thousands of proofs per minute, pulling the hash-set into the asm primitive layer erases a dominant cost. The same pattern applies to any structure with a tight inner loop: vector sort, string search, rolling checksum. Asm's tree-walker is slow on user code; asm's builtins run at the speed of the host CPU.
|
||||
|
||||
The layout reuses vector tag 7 with a negative sentinel at offset 0 (hash-table: ``-1``, hash-set: ``-2``). ``vector?`` and the printer check the sentinel to stay disjoint from real vectors. No additional tag bits were consumed — the seven-way tag mask (``TAG_MASK = 7``) still has room for only heap-level containers, and nothing new needs tagging because the sentinel carries the discrimination.
|
||||
|
||||
**Reproduce:** ``make bench-hashset`` (source: ``tests/bench-hashset.sh``). Asm-only: C and Python tiers supply ``hash-table`` but not ``hash-set`` — the speedup shown here is intra-asm, native primitive vs Scheme-level implementation, not cross-tier.
|
||||
|
||||
7. Portal: Feedback Across Time
|
||||
----------------------------------------
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue