diff --git a/AGGREGATED-PERFORMANCE.md b/AGGREGATED-PERFORMANCE.md index f73e0ac..0f1cca7 100644 --- a/AGGREGATED-PERFORMANCE.md +++ b/AGGREGATED-PERFORMANCE.md @@ -1,13 +1,13 @@ # UN Inception: Aggregated Performance Analysis -**Analysis Date:** 1769160997.9100175 -**Reports Analyzed:** 4.2.0, 4.2.3, 4.2.4, 4.2.5, 4.2.6, 4.2.7 +**Analysis Date:** 1769162516.8008766 +**Reports Analyzed:** 4.2.0, 4.2.3, 4.2.4, 4.2.5, 4.2.6, 4.2.7, 4.2.8 --- ## Executive Summary -Analysis of 6 performance reports reveals **significant variance** in execution metrics across releases. Different languages rank as slowest/fastest in different runs, indicating **non-deterministic execution patterns** likely caused by: +Analysis of 7 performance reports reveals **significant variance** in execution metrics across releases. Different languages rank as slowest/fastest in different runs, indicating **non-deterministic execution patterns** likely caused by: 1. **Orchestrator placement on CPU-bound pool** (not an SRE best practice) 2. **Resource contention** between the orchestrator & test jobs @@ -28,6 +28,7 @@ Analysis of 6 performance reports reveals **significant variance** in execution | 4.2.5 | 67s | v (114s) | erlang (44s) | -3s (-4.3%) | | 4.2.6 | 54s | haskell (128s) | awk (23s) | -13s (-19.4%) | | 4.2.7 | 117s | typescript (319s) | dotnet (5s) | +63s (+116.7%) | +| 4.2.8 | 111s | kotlin (313s) | fortran (28s) | -6s (-5.1%) | **Observation:** Average duration increased **0.0%** from 0s to 0s. @@ -50,6 +51,7 @@ The same language changes dramatically in rank between runs: - 4.2.5: 55s - 4.2.6: 41s - 4.2.7: 313s + - 4.2.8: 127s - **Range:** 20s → 313s (1465.0% variance) **TYPESCRIPT:** @@ -59,6 +61,7 @@ The same language changes dramatically in rank between runs: - 4.2.5: 59s - 4.2.6: 53s - 4.2.7: 319s + - 4.2.8: 94s - **Range:** 29s → 319s (1000.0% variance) **R:** @@ -68,8 +71,19 @@ The same language changes dramatically in rank between runs: - 4.2.5: 52s - 4.2.6: 47s - 4.2.7: 313s + - 4.2.8: 126s - **Range:** 25s → 313s (1152.0% variance) +**KOTLIN:** + - 4.2.0: 27s + - 4.2.3: 45s + - 4.2.4: 54s + - 4.2.5: 61s + - 4.2.6: 69s + - 4.2.7: 49s + - 4.2.8: 313s + - **Range:** 27s → 313s (1059.3% variance) + **RUBY:** - 4.2.0: 35s - 4.2.3: 74s @@ -77,17 +91,9 @@ The same language changes dramatically in rank between runs: - 4.2.5: 60s - 4.2.6: 48s - 4.2.7: 318s + - 4.2.8: 93s - **Range:** 35s → 318s (808.6% variance) -**CRYSTAL:** - - 4.2.0: 24s - - 4.2.3: 61s - - 4.2.4: 60s - - 4.2.5: 88s - - 4.2.6: 28s - - 4.2.7: 260s - - **Range:** 24s → 260s (983.3% variance) - --- @@ -101,6 +107,7 @@ The same language changes dramatically in rank between runs: 4.2.5: erlang, awk, bash, deno, tcl 4.2.6: awk, powershell, crystal, raku, erlang 4.2.7: dotnet, deno, awk, fortran, commonlisp +4.2.8: fortran, groovy, crystal, java, powershell **Slowest Languages by Run:** @@ -110,6 +117,7 @@ The same language changes dramatically in rank between runs: 4.2.5: v, haskell, scheme, ocaml, powershell 4.2.6: haskell, go, cpp, rust, forth 4.2.7: typescript, ruby, r, elixir, crystal +4.2.8: kotlin, python, javascript, tcl, raku **Conclusion:** No consistent "fast" or "slow" languages across runs. This proves: - Execution order is random or system-dependent @@ -197,19 +205,19 @@ TYPESCRIPT: 29s → 319s (+1000.0%) R: 25s → 313s (+1152.0%) +KOTLIN: 27s → 313s (+1059.3%) + RUBY: 35s → 318s (+808.6%) CRYSTAL: 24s → 260s (+983.3%) +TCL: 20s → 247s (+1135.0%) + +PYTHON: 40s → 253s (+532.5%) + PHP: 23s → 235s (+921.7%) -BASH: 35s → 233s (+565.7%) - -JULIA: 32s → 226s (+606.2%) - -OBJC: 29s → 223s (+669.0%) - -PERL: 26s → 155s (+496.2%) +RAKU: 33s → 240s (+627.3%) These languages are most affected by resource contention. Likely reasons: @@ -262,26 +270,26 @@ Keep it as-is for stress testing, but in separate test environment. | Language | Min (s) | Max (s) | Avg (s) | Range (s) | Variance % | |----------|---------|---------|---------|-----------|------------| -| ELIXIR | 20 | 313 | 100.5 | 293 | 1465.0% | -| R | 25 | 313 | 95.8 | 288 | 1152.0% | -| TYPESCRIPT | 29 | 319 | 102.5 | 290 | 1000.0% | -| CRYSTAL | 24 | 260 | 86.8 | 236 | 983.3% | -| PHP | 23 | 235 | 84.2 | 212 | 921.7% | -| RUBY | 35 | 318 | 102.3 | 283 | 808.6% | -| OBJC | 29 | 223 | 85.8 | 194 | 669.0% | -| TCL | 20 | 148 | 69.7 | 128 | 640.0% | +| DOTNET | 5 | 102 | 53.5 | 97 | 1940.0% | +| ELIXIR | 20 | 313 | 104.3 | 293 | 1465.0% | +| R | 25 | 313 | 100.1 | 288 | 1152.0% | +| TCL | 20 | 247 | 95.0 | 227 | 1135.0% | +| KOTLIN | 27 | 313 | 88.3 | 286 | 1059.3% | +| TYPESCRIPT | 29 | 319 | 101.3 | 290 | 1000.0% | +| CRYSTAL | 24 | 260 | 80.7 | 236 | 983.3% | +| PHP | 23 | 235 | 85.1 | 212 | 921.7% | +| COBOL | 20 | 199 | 71.4 | 179 | 895.0% | +| RUBY | 35 | 318 | 101.0 | 283 | 808.6% | +| OCAML | 19 | 167 | 74.3 | 148 | 778.9% | +| HASKELL | 21 | 168 | 89.0 | 147 | 700.0% | +| OBJC | 29 | 223 | 86.3 | 194 | 669.0% | +| RAKU | 33 | 240 | 86.7 | 207 | 627.3% | | JULIA | 32 | 226 | 87.7 | 194 | 606.2% | -| BASH | 35 | 233 | 86.3 | 198 | 565.7% | -| HASKELL | 21 | 128 | 75.8 | 107 | 509.5% | -| SCHEME | 24 | 146 | 79.8 | 122 | 508.3% | -| PERL | 26 | 155 | 70.2 | 129 | 496.2% | -| POWERSHELL | 26 | 141 | 74.0 | 115 | 442.3% | -| V | 22 | 114 | 59.8 | 92 | 418.2% | -| ERLANG | 29 | 148 | 70.0 | 119 | 410.3% | -| OCAML | 19 | 94 | 58.8 | 75 | 394.7% | -| LUA | 32 | 152 | 67.7 | 120 | 375.0% | -| C | 23 | 107 | 61.2 | 84 | 365.2% | -| CLOJURE | 21 | 97 | 57.5 | 76 | 361.9% | +| BASH | 35 | 233 | 96.1 | 198 | 565.7% | +| FSHARP | 26 | 165 | 71.4 | 139 | 534.6% | +| PYTHON | 40 | 253 | 93.1 | 213 | 532.5% | +| SCHEME | 24 | 146 | 76.3 | 122 | 508.3% | +| PERL | 26 | 155 | 73.1 | 129 | 496.2% | --- @@ -342,6 +350,7 @@ Individual Reports → Aggregation Script → Chart Generation (via UN) → Fina - `reports/4.2.5/perf.json` - 658 tests, generated 2026-01-19T19:10:23Z - `reports/4.2.6/perf.json` - 642 tests, generated 2026-01-19T20:22:16Z - `reports/4.2.7/perf.json` - 631 tests, generated 2026-01-23T09:36:18Z +- `reports/4.2.8/perf.json` - 645 tests, generated 2026-01-23T10:01:33Z Each `perf.json` contains: @@ -538,5 +547,5 @@ For questions about this methodology or to report issues: --- **Generated by UN Inception Performance Analysis Pipeline** -**Analysis Date:** 2026-01-23T04:36:38.092232 +**Analysis Date:** 2026-01-23T05:01:56.883707 **Report Version:** 1.0.0