Update UN Inception blog post with 4.2.5 & 4.2.6 data

Now analyzing 5 releases instead of 3 (4.2.0, 4.2.3, 4.2.4, 4.2.5, 4.2.6)

Updated variance examples:
- Haskell: 509.5% variance (21s → 128s) - new worst offender
- V: 418.2% variance (22s → 114s peak)
- Elixir: 425% variance (20s → 105s peak)
- C: 365% variance (23s → 107s peak)

Shows even more dramatic chaos with 5 data points demonstrating
non-deterministic scheduling patterns across releases.

Updated all 3 aggregated charts to reflect 5-release analysis.
This commit is contained in:
russell@unturf.com 2026-01-19 15:30:11 -05:00
parent b60c1451b7
commit 842daaec5c
4 changed files with 29 additions and 7 deletions

View file

@ -9,7 +9,7 @@ un inception: performance variance & chaos engineering in ci/cd
**The full** `AGGREGATED-PERFORMANCE.md report <https://git.unturf.com/engineering/unturf/un-inception/-/blob/main/AGGREGATED-PERFORMANCE.md>`_ **lives in the un-inception repo.**
I recently analyzed performance variance across 3 releases of `UN Inception <https://git.unturf.com/engineering/unturf/un-inception>`_ (4.2.0, 4.2.3, 4.2.4) & discovered something fascinating: the same test suite running the same workload shows 2-3x performance variance. Not random noise. A symptom of architectural misplacement.
I recently analyzed performance variance across 5 releases of `UN Inception <https://git.unturf.com/engineering/unturf/un-inception>`_ (4.2.0, 4.2.3, 4.2.4, 4.2.5, 4.2.6) & discovered something fascinating: the same test suite running the same workload shows 2-5x performance variance. Not random noise. A symptom of architectural misplacement.
**What is UN?** `UN (unsandbox.com) <https://unsandbox.com>`_ is a remote code execution platform supporting 42+ programming languages via REST API. Think of it as a computational sandbox for running arbitrary code securely & retrieving artifacts.
@ -75,25 +75,47 @@ Sounds deterministic, right? Same code, same tests, same infrastructure. Should
the variance
============
**Elixir's journey:**
**Haskell's journey across 5 releases:**
- 4.2.0: 21 seconds (fast, consistent)
- 4.2.3: 58 seconds (2.8x slower)
- 4.2.4: 66 seconds (3.1x slower)
- 4.2.5: 106 seconds (5x slower)
- 4.2.6: 128 seconds (6.1x slower than baseline)
**509.5% variance.** For identical tests. The winner becomes the loser.
**V's wild ride:**
- 4.2.0: 22 seconds
- 4.2.3: 40 seconds
- 4.2.4: 49 seconds
- 4.2.5: 114 seconds (5.2x slower than baseline)
- 4.2.6: 58 seconds (back down, but still 2.6x baseline)
**418.2% variance.** V is a roller coaster.
**Elixir's chaos:**
- 4.2.0: 20 seconds (fast, efficient)
- 4.2.3: 69 seconds (3.5x slower)
- 4.2.4: 105 seconds (5x slower than baseline)
- 4.2.4: 105 seconds (5.3x slower)
- 4.2.5: 55 seconds (back to reasonable)
- 4.2.6: 41 seconds (2x baseline)
**425% variance.** For identical tests.
**425% variance.** Up, down, all around.
**C's performance:**
- 4.2.0: 23 seconds
- 4.2.3: 107 seconds (4.7x slower)
- 4.2.4: 23 seconds (back to baseline)
- 4.2.5: 68 seconds (3x baseline)
- 4.2.6: 74 seconds (3.2x baseline)
**365% variance.** C didn't change. The infrastructure did.
**Rust, Clojure, Scheme:** All showing 120-360% variance.
No consistent pattern. Different languages win & lose in each run. The fastest language in one release becomes the slowest in the next.
No consistent pattern. Different languages win & lose in each run. The fastest language in one release becomes the slowest in the next. Haskell went from 21s to 128s. V spiked to 114s then dropped. Pure chaos.
.. image:: /uploads/2026/01/aggregated-duration-trend.png
:alt: Duration degradation over releases

Binary file not shown.

Before

Width:  |  Height:  |  Size: 60 KiB

After

Width:  |  Height:  |  Size: 68 KiB

Before After
Before After

Binary file not shown.

Before

Width:  |  Height:  |  Size: 80 KiB

After

Width:  |  Height:  |  Size: 82 KiB

Before After
Before After

Binary file not shown.

Before

Width:  |  Height:  |  Size: 60 KiB

After

Width:  |  Height:  |  Size: 63 KiB

Before After
Before After