Update UN Inception blog post with 4.2.5 & 4.2.6 data
Now analyzing 5 releases instead of 3 (4.2.0, 4.2.3, 4.2.4, 4.2.5, 4.2.6) Updated variance examples: - Haskell: 509.5% variance (21s → 128s) - new worst offender - V: 418.2% variance (22s → 114s peak) - Elixir: 425% variance (20s → 105s peak) - C: 365% variance (23s → 107s peak) Shows even more dramatic chaos with 5 data points demonstrating non-deterministic scheduling patterns across releases. Updated all 3 aggregated charts to reflect 5-release analysis.
This commit is contained in:
parent
b60c1451b7
commit
842daaec5c
4 changed files with 29 additions and 7 deletions
|
|
@ -9,7 +9,7 @@ un inception: performance variance & chaos engineering in ci/cd
|
|||
|
||||
**The full** `AGGREGATED-PERFORMANCE.md report <https://git.unturf.com/engineering/unturf/un-inception/-/blob/main/AGGREGATED-PERFORMANCE.md>`_ **lives in the un-inception repo.**
|
||||
|
||||
I recently analyzed performance variance across 3 releases of `UN Inception <https://git.unturf.com/engineering/unturf/un-inception>`_ (4.2.0, 4.2.3, 4.2.4) & discovered something fascinating: the same test suite running the same workload shows 2-3x performance variance. Not random noise. A symptom of architectural misplacement.
|
||||
I recently analyzed performance variance across 5 releases of `UN Inception <https://git.unturf.com/engineering/unturf/un-inception>`_ (4.2.0, 4.2.3, 4.2.4, 4.2.5, 4.2.6) & discovered something fascinating: the same test suite running the same workload shows 2-5x performance variance. Not random noise. A symptom of architectural misplacement.
|
||||
|
||||
**What is UN?** `UN (unsandbox.com) <https://unsandbox.com>`_ is a remote code execution platform supporting 42+ programming languages via REST API. Think of it as a computational sandbox for running arbitrary code securely & retrieving artifacts.
|
||||
|
||||
|
|
@ -75,25 +75,47 @@ Sounds deterministic, right? Same code, same tests, same infrastructure. Should
|
|||
the variance
|
||||
============
|
||||
|
||||
**Elixir's journey:**
|
||||
**Haskell's journey across 5 releases:**
|
||||
|
||||
- 4.2.0: 21 seconds (fast, consistent)
|
||||
- 4.2.3: 58 seconds (2.8x slower)
|
||||
- 4.2.4: 66 seconds (3.1x slower)
|
||||
- 4.2.5: 106 seconds (5x slower)
|
||||
- 4.2.6: 128 seconds (6.1x slower than baseline)
|
||||
|
||||
**509.5% variance.** For identical tests. The winner becomes the loser.
|
||||
|
||||
**V's wild ride:**
|
||||
|
||||
- 4.2.0: 22 seconds
|
||||
- 4.2.3: 40 seconds
|
||||
- 4.2.4: 49 seconds
|
||||
- 4.2.5: 114 seconds (5.2x slower than baseline)
|
||||
- 4.2.6: 58 seconds (back down, but still 2.6x baseline)
|
||||
|
||||
**418.2% variance.** V is a roller coaster.
|
||||
|
||||
**Elixir's chaos:**
|
||||
|
||||
- 4.2.0: 20 seconds (fast, efficient)
|
||||
- 4.2.3: 69 seconds (3.5x slower)
|
||||
- 4.2.4: 105 seconds (5x slower than baseline)
|
||||
- 4.2.4: 105 seconds (5.3x slower)
|
||||
- 4.2.5: 55 seconds (back to reasonable)
|
||||
- 4.2.6: 41 seconds (2x baseline)
|
||||
|
||||
**425% variance.** For identical tests.
|
||||
**425% variance.** Up, down, all around.
|
||||
|
||||
**C's performance:**
|
||||
|
||||
- 4.2.0: 23 seconds
|
||||
- 4.2.3: 107 seconds (4.7x slower)
|
||||
- 4.2.4: 23 seconds (back to baseline)
|
||||
- 4.2.5: 68 seconds (3x baseline)
|
||||
- 4.2.6: 74 seconds (3.2x baseline)
|
||||
|
||||
**365% variance.** C didn't change. The infrastructure did.
|
||||
|
||||
**Rust, Clojure, Scheme:** All showing 120-360% variance.
|
||||
|
||||
No consistent pattern. Different languages win & lose in each run. The fastest language in one release becomes the slowest in the next.
|
||||
No consistent pattern. Different languages win & lose in each run. The fastest language in one release becomes the slowest in the next. Haskell went from 21s to 128s. V spiked to 114s then dropped. Pure chaos.
|
||||
|
||||
.. image:: /uploads/2026/01/aggregated-duration-trend.png
|
||||
:alt: Duration degradation over releases
|
||||
|
|
|
|||
Binary file not shown.
|
Before Width: | Height: | Size: 60 KiB After Width: | Height: | Size: 68 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 80 KiB After Width: | Height: | Size: 82 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 60 KiB After Width: | Height: | Size: 63 KiB |
Loading…
Add table
Add a link
Reference in a new issue