diff --git a/AGGREGATED-PERFORMANCE.md b/AGGREGATED-PERFORMANCE.md index 9cf6563..d45ce69 100644 --- a/AGGREGATED-PERFORMANCE.md +++ b/AGGREGATED-PERFORMANCE.md @@ -1,22 +1,13 @@ # UN Inception: Aggregated Performance Analysis -<<<<<<< Updated upstream -**Analysis Date:** 1769282055.7116168 -**Reports Analyzed:** 4.2.0, 4.2.10, 4.2.11, 4.2.12, 4.2.13, 4.2.14, 4.2.15, 4.2.16, 4.2.17, 4.2.18, 4.2.19, 4.2.20, 4.2.21, 4.2.22, 4.2.24, 4.2.3, 4.2.4, 4.2.5, 4.2.6, 4.2.7, 4.2.8, 4.2.9 -======= -**Analysis Date:** 1769282070.5510108 +**Analysis Date:** 1769287519.037188 **Reports Analyzed:** 4.2.0, 4.2.10, 4.2.11, 4.2.12, 4.2.13, 4.2.14, 4.2.15, 4.2.16, 4.2.17, 4.2.18, 4.2.19, 4.2.20, 4.2.21, 4.2.22, 4.2.23, 4.2.24, 4.2.3, 4.2.4, 4.2.5, 4.2.6, 4.2.7, 4.2.8, 4.2.9 ->>>>>>> Stashed changes --- ## Executive Summary -<<<<<<< Updated upstream -Analysis of 22 performance reports reveals **significant variance** in execution metrics across releases. Different languages rank as slowest/fastest in different runs, indicating **non-deterministic execution patterns** likely caused by: -======= Analysis of 23 performance reports reveals **significant variance** in execution metrics across releases. Different languages rank as slowest/fastest in different runs, indicating **non-deterministic execution patterns** likely caused by: ->>>>>>> Stashed changes 1. **Orchestrator placement on CPU-bound pool** (not an SRE best practice) 2. **Resource contention** between the orchestrator & test jobs @@ -45,12 +36,8 @@ Analysis of 23 performance reports reveals **significant variance** in execution | 4.2.20 | 114s | java (272s) | crystal (49s) | -37s (-24.5%) | | 4.2.21 | 102s | nim (215s) | erlang (22s) | -12s (-10.5%) | | 4.2.22 | 300s | javascript (2173s) | clojure (14s) | +198s (+194.1%) | -<<<<<<< Updated upstream -| 4.2.24 | 129s | python (494s) | clojure (8s) | -171s (-57.0%) | -======= | 4.2.23 | 373s | zig (1058s) | perl (8s) | +73s (+24.3%) | | 4.2.24 | 129s | python (494s) | clojure (8s) | -244s (-65.4%) | ->>>>>>> Stashed changes | 4.2.3 | 63s | rust (142s) | v (40s) | -66s (-51.2%) | | 4.2.4 | 70s | python (110s) | c (23s) | +7s (+11.1%) | | 4.2.5 | 67s | v (114s) | erlang (44s) | -3s (-4.3%) | @@ -88,10 +75,7 @@ The same language changes dramatically in rank between runs: - 4.2.20: 87s - 4.2.21: 42s - 4.2.22: 2173s -<<<<<<< Updated upstream -======= - 4.2.23: 425s ->>>>>>> Stashed changes - 4.2.24: 242s - 4.2.3: 75s - 4.2.4: 109s @@ -117,10 +101,7 @@ The same language changes dramatically in rank between runs: - 4.2.20: 57s - 4.2.21: 24s - 4.2.22: 1834s -<<<<<<< Updated upstream -======= - 4.2.23: 9s ->>>>>>> Stashed changes - 4.2.24: 434s - 4.2.3: 64s - 4.2.4: 74s @@ -146,10 +127,7 @@ The same language changes dramatically in rank between runs: - 4.2.20: 103s - 4.2.21: 215s - 4.2.22: 1217s -<<<<<<< Updated upstream -======= - 4.2.23: 286s ->>>>>>> Stashed changes - 4.2.24: 8s - 4.2.3: 52s - 4.2.4: 76s @@ -175,10 +153,7 @@ The same language changes dramatically in rank between runs: - 4.2.20: 224s - 4.2.21: 192s - 4.2.22: 1014s -<<<<<<< Updated upstream -======= - 4.2.23: 1058s ->>>>>>> Stashed changes - 4.2.24: 8s - 4.2.3: 61s - 4.2.4: 60s @@ -187,34 +162,6 @@ The same language changes dramatically in rank between runs: - 4.2.7: 79s - 4.2.8: 187s - 4.2.9: 153s -<<<<<<< Updated upstream - - **Range:** 8s → 1014s (12575.0% variance) - -**LUA:** - - 4.2.0: 32s - - 4.2.10: 51s - - 4.2.11: 56s - - 4.2.12: 53s - - 4.2.13: 203s - - 4.2.14: 127s - - 4.2.15: 166s - - 4.2.16: 242s - - 4.2.17: 40s - - 4.2.18: 73s - - 4.2.19: 472s - - 4.2.20: 133s - - 4.2.21: 183s - - 4.2.22: 975s - - 4.2.24: 9s - - 4.2.3: 63s - - 4.2.4: 71s - - 4.2.5: 52s - - 4.2.6: 36s - - 4.2.7: 152s - - 4.2.8: 84s - - 4.2.9: 56s - - **Range:** 9s → 975s (10733.3% variance) -======= - **Range:** 8s → 1058s (13125.0% variance) **V:** @@ -242,7 +189,6 @@ The same language changes dramatically in rank between runs: - 4.2.8: 109s - 4.2.9: 42s - **Range:** 21s → 1040s (4852.4% variance) ->>>>>>> Stashed changes --- @@ -265,10 +211,7 @@ The same language changes dramatically in rank between runs: 4.2.20: crystal, v, deno, r, csharp 4.2.21: erlang, r, ruby, awk, typescript 4.2.22: powershell, clojure, scheme, objc, v -<<<<<<< Updated upstream -======= 4.2.23: perl, r, d, groovy, powershell ->>>>>>> Stashed changes 4.2.24: zig, nim, kotlin, fortran, forth 4.2.3: v, d, kotlin, awk, raku 4.2.4: c, d, cobol, raku, v @@ -294,10 +237,7 @@ The same language changes dramatically in rank between runs: 4.2.20: java, zig, cobol, perl, haskell 4.2.21: nim, dart, java, cpp, rust 4.2.22: javascript, r, nim, zig, lua -<<<<<<< Updated upstream -======= 4.2.23: zig, v, commonlisp, deno, elixir ->>>>>>> Stashed changes 4.2.24: python, php, r, elixir, deno 4.2.3: rust, c, python, typescript, javascript 4.2.4: python, javascript, elixir, scheme, bash @@ -393,35 +333,20 @@ R: 9s → 1834s (+20277.8%) NIM: 8s → 1217s (+15112.5%) -<<<<<<< Updated upstream -ZIG: 8s → 1014s (+12575.0%) - -LUA: 9s → 975s (+10733.3%) -======= ZIG: 8s → 1058s (+13125.0%) V: 21s → 1040s (+4852.4%) ->>>>>>> Stashed changes LUA: 9s → 975s (+10733.3%) COMMONLISP: 27s → 956s (+3440.7%) -<<<<<<< Updated upstream -PYTHON: 19s → 494s (+2500.0%) - -PHP: 23s → 471s (+1947.8%) - -PERL: 26s → 468s (+1700.0%) - -======= DENO: 24s → 926s (+3758.3%) ELIXIR: 20s → 917s (+4485.0%) PHP: 23s → 824s (+3482.6%) ->>>>>>> Stashed changes These languages are most affected by resource contention. Likely reasons: - **Dynamic languages** (Python, Ruby, JavaScript): Startup time varies with GC/JIT @@ -473,28 +398,6 @@ Keep it as-is for stress testing, but in separate test environment. | Language | Min (s) | Max (s) | Avg (s) | Range (s) | Variance % | |----------|---------|---------|---------|-----------|------------| -<<<<<<< Updated upstream -| NIM | 8 | 1217 | 127.3 | 1209 | 15112.5% | -| ZIG | 8 | 1014 | 154.8 | 1006 | 12575.0% | -| LUA | 9 | 975 | 151.3 | 966 | 10733.3% | -| R | 24 | 1834 | 205.7 | 1810 | 7541.7% | -| JAVASCRIPT | 32 | 2173 | 218.7 | 2141 | 6690.6% | -| KOTLIN | 8 | 313 | 93.7 | 305 | 3812.5% | -| CLOJURE | 8 | 310 | 101.0 | 302 | 3775.0% | -| HASKELL | 21 | 754 | 123.0 | 733 | 3490.5% | -| DOTNET | 5 | 176 | 81.1 | 171 | 3420.0% | -| CSHARP | 8 | 266 | 89.0 | 258 | 3225.0% | -| PROLOG | 9 | 262 | 76.7 | 253 | 2811.1% | -| FORTH | 8 | 231 | 94.0 | 223 | 2787.5% | -| RUBY | 30 | 825 | 160.4 | 795 | 2650.0% | -| PYTHON | 19 | 494 | 108.1 | 475 | 2500.0% | -| RUST | 9 | 234 | 116.9 | 225 | 2500.0% | -| D | 8 | 203 | 82.8 | 195 | 2437.5% | -| SCHEME | 15 | 320 | 98.9 | 305 | 2033.3% | -| PHP | 23 | 471 | 123.0 | 448 | 1947.8% | -| ELIXIR | 20 | 404 | 129.9 | 384 | 1920.0% | -| FORTRAN | 8 | 161 | 71.9 | 153 | 1912.5% | -======= | R | 9 | 1834 | 197.2 | 1825 | 20277.8% | | NIM | 8 | 1217 | 134.2 | 1209 | 15112.5% | | ZIG | 8 | 1058 | 194.0 | 1050 | 13125.0% | @@ -515,7 +418,6 @@ Keep it as-is for stress testing, but in separate test environment. | PHP | 23 | 824 | 153.5 | 801 | 3482.6% | | TCL | 20 | 712 | 117.4 | 692 | 3460.0% | | COMMONLISP | 27 | 956 | 124.0 | 929 | 3440.7% | ->>>>>>> Stashed changes --- @@ -584,10 +486,7 @@ Individual Reports → Aggregation Script → Chart Generation (via UN) → Fina - `reports/4.2.20/perf.json` - 685 tests, generated 2026-01-23T20:41:23Z - `reports/4.2.21/perf.json` - 661 tests, generated 2026-01-23T21:16:07Z - `reports/4.2.22/perf.json` - 697 tests, generated 2026-01-24T17:57:56Z -<<<<<<< Updated upstream -======= - `reports/4.2.23/perf.json` - 713 tests, generated 2026-01-24T19:14:09Z ->>>>>>> Stashed changes - `reports/4.2.24/perf.json` - 665 tests, generated 2026-01-24T19:13:51Z - `reports/4.2.3/perf.json` - 642 tests, generated 2026-01-19T11:58:45Z - `reports/4.2.4/perf.json` - 682 tests, generated 2026-01-19T12:02:14Z @@ -792,9 +691,5 @@ For questions about this methodology or to report issues: --- **Generated by UN Inception Performance Analysis Pipeline** -<<<<<<< Updated upstream -**Analysis Date:** 2026-01-24T14:14:15.842875 -======= -**Analysis Date:** 2026-01-24T14:14:30.672853 ->>>>>>> Stashed changes +**Analysis Date:** 2026-01-24T15:45:42.561682 **Report Version:** 1.0.0 diff --git a/reports/aggregated-duration-trend.png b/reports/aggregated-duration-trend.png index b3f195d..0ce26fe 100644 Binary files a/reports/aggregated-duration-trend.png and b/reports/aggregated-duration-trend.png differ diff --git a/reports/aggregated-language-variance.png b/reports/aggregated-language-variance.png index caa62dc..c1dd740 100644 Binary files a/reports/aggregated-language-variance.png and b/reports/aggregated-language-variance.png differ diff --git a/reports/aggregated-ranking-changes.png b/reports/aggregated-ranking-changes.png index 91a46bb..2db7d08 100644 Binary files a/reports/aggregated-ranking-changes.png and b/reports/aggregated-ranking-changes.png differ