diff --git a/AGGREGATED-PERFORMANCE.md b/AGGREGATED-PERFORMANCE.md index d45ce69..6807170 100644 --- a/AGGREGATED-PERFORMANCE.md +++ b/AGGREGATED-PERFORMANCE.md @@ -1,13 +1,13 @@ # UN Inception: Aggregated Performance Analysis -**Analysis Date:** 1769287519.037188 -**Reports Analyzed:** 4.2.0, 4.2.10, 4.2.11, 4.2.12, 4.2.13, 4.2.14, 4.2.15, 4.2.16, 4.2.17, 4.2.18, 4.2.19, 4.2.20, 4.2.21, 4.2.22, 4.2.23, 4.2.24, 4.2.3, 4.2.4, 4.2.5, 4.2.6, 4.2.7, 4.2.8, 4.2.9 +**Analysis Date:** 1769288671.067574 +**Reports Analyzed:** 4.2.0, 4.2.10, 4.2.11, 4.2.12, 4.2.13, 4.2.14, 4.2.15, 4.2.16, 4.2.17, 4.2.18, 4.2.19, 4.2.20, 4.2.21, 4.2.22, 4.2.23, 4.2.24, 4.2.25, 4.2.3, 4.2.4, 4.2.5, 4.2.6, 4.2.7, 4.2.8, 4.2.9 --- ## Executive Summary -Analysis of 23 performance reports reveals **significant variance** in execution metrics across releases. Different languages rank as slowest/fastest in different runs, indicating **non-deterministic execution patterns** likely caused by: +Analysis of 24 performance reports reveals **significant variance** in execution metrics across releases. Different languages rank as slowest/fastest in different runs, indicating **non-deterministic execution patterns** likely caused by: 1. **Orchestrator placement on CPU-bound pool** (not an SRE best practice) 2. **Resource contention** between the orchestrator & test jobs @@ -38,7 +38,8 @@ Analysis of 23 performance reports reveals **significant variance** in execution | 4.2.22 | 300s | javascript (2173s) | clojure (14s) | +198s (+194.1%) | | 4.2.23 | 373s | zig (1058s) | perl (8s) | +73s (+24.3%) | | 4.2.24 | 129s | python (494s) | clojure (8s) | -244s (-65.4%) | -| 4.2.3 | 63s | rust (142s) | v (40s) | -66s (-51.2%) | +| 4.2.25 | 97s | fortran (151s) | bash (58s) | -32s (-24.8%) | +| 4.2.3 | 63s | rust (142s) | v (40s) | -34s (-35.1%) | | 4.2.4 | 70s | python (110s) | c (23s) | +7s (+11.1%) | | 4.2.5 | 67s | v (114s) | erlang (44s) | -3s (-4.3%) | | 4.2.6 | 54s | haskell (128s) | awk (23s) | -13s (-19.4%) | @@ -77,6 +78,7 @@ The same language changes dramatically in rank between runs: - 4.2.22: 2173s - 4.2.23: 425s - 4.2.24: 242s + - 4.2.25: 74s - 4.2.3: 75s - 4.2.4: 109s - 4.2.5: 60s @@ -103,6 +105,7 @@ The same language changes dramatically in rank between runs: - 4.2.22: 1834s - 4.2.23: 9s - 4.2.24: 434s + - 4.2.25: 65s - 4.2.3: 64s - 4.2.4: 74s - 4.2.5: 52s @@ -129,6 +132,7 @@ The same language changes dramatically in rank between runs: - 4.2.22: 1217s - 4.2.23: 286s - 4.2.24: 8s + - 4.2.25: 100s - 4.2.3: 52s - 4.2.4: 76s - 4.2.5: 58s @@ -155,6 +159,7 @@ The same language changes dramatically in rank between runs: - 4.2.22: 1014s - 4.2.23: 1058s - 4.2.24: 8s + - 4.2.25: 108s - 4.2.3: 61s - 4.2.4: 60s - 4.2.5: 59s @@ -181,6 +186,7 @@ The same language changes dramatically in rank between runs: - 4.2.22: 21s - 4.2.23: 1040s - 4.2.24: 101s + - 4.2.25: 97s - 4.2.3: 40s - 4.2.4: 49s - 4.2.5: 114s @@ -213,6 +219,7 @@ The same language changes dramatically in rank between runs: 4.2.22: powershell, clojure, scheme, objc, v 4.2.23: perl, r, d, groovy, powershell 4.2.24: zig, nim, kotlin, fortran, forth +4.2.25: bash, powershell, forth, r, prolog 4.2.3: v, d, kotlin, awk, raku 4.2.4: c, d, cobol, raku, v 4.2.5: erlang, awk, bash, deno, tcl @@ -239,6 +246,7 @@ The same language changes dramatically in rank between runs: 4.2.22: javascript, r, nim, zig, lua 4.2.23: zig, v, commonlisp, deno, elixir 4.2.24: python, php, r, elixir, deno +4.2.25: fortran, crystal, perl, awk, cpp 4.2.3: rust, c, python, typescript, javascript 4.2.4: python, javascript, elixir, scheme, bash 4.2.5: v, haskell, scheme, ocaml, powershell @@ -398,26 +406,26 @@ Keep it as-is for stress testing, but in separate test environment. | Language | Min (s) | Max (s) | Avg (s) | Range (s) | Variance % | |----------|---------|---------|---------|-----------|------------| -| R | 9 | 1834 | 197.2 | 1825 | 20277.8% | -| NIM | 8 | 1217 | 134.2 | 1209 | 15112.5% | -| ZIG | 8 | 1058 | 194.0 | 1050 | 13125.0% | -| LUA | 9 | 975 | 159.1 | 966 | 10733.3% | -| FORTH | 8 | 628 | 117.2 | 620 | 7750.0% | -| DOTNET | 5 | 360 | 96.6 | 355 | 7100.0% | -| JAVASCRIPT | 32 | 2173 | 227.7 | 2141 | 6690.6% | -| PERL | 8 | 468 | 109.0 | 460 | 5750.0% | -| V | 21 | 1040 | 118.7 | 1019 | 4852.4% | -| CSHARP | 8 | 385 | 101.9 | 377 | 4712.5% | -| ELIXIR | 20 | 917 | 164.1 | 897 | 4485.0% | -| OBJC | 17 | 693 | 111.2 | 676 | 3976.5% | -| KOTLIN | 8 | 313 | 95.3 | 305 | 3812.5% | -| CLOJURE | 8 | 310 | 109.1 | 302 | 3775.0% | -| DENO | 24 | 926 | 163.6 | 902 | 3758.3% | -| COBOL | 20 | 759 | 141.4 | 739 | 3695.0% | -| HASKELL | 21 | 754 | 132.3 | 733 | 3490.5% | -| PHP | 23 | 824 | 153.5 | 801 | 3482.6% | -| TCL | 20 | 712 | 117.4 | 692 | 3460.0% | -| COMMONLISP | 27 | 956 | 124.0 | 929 | 3440.7% | +| R | 9 | 1834 | 191.7 | 1825 | 20277.8% | +| NIM | 8 | 1217 | 132.8 | 1209 | 15112.5% | +| ZIG | 8 | 1058 | 190.5 | 1050 | 13125.0% | +| LUA | 9 | 975 | 155.5 | 966 | 10733.3% | +| FORTH | 8 | 628 | 115.0 | 620 | 7750.0% | +| DOTNET | 5 | 360 | 95.3 | 355 | 7100.0% | +| JAVASCRIPT | 32 | 2173 | 221.2 | 2141 | 6690.6% | +| PERL | 8 | 468 | 110.3 | 460 | 5750.0% | +| V | 21 | 1040 | 117.8 | 1019 | 4852.4% | +| CSHARP | 8 | 385 | 100.8 | 377 | 4712.5% | +| ELIXIR | 20 | 917 | 161.8 | 897 | 4485.0% | +| OBJC | 17 | 693 | 110.4 | 676 | 3976.5% | +| KOTLIN | 8 | 313 | 95.9 | 305 | 3812.5% | +| CLOJURE | 8 | 310 | 107.4 | 302 | 3775.0% | +| DENO | 24 | 926 | 161.5 | 902 | 3758.3% | +| COBOL | 20 | 759 | 140.2 | 739 | 3695.0% | +| HASKELL | 21 | 754 | 131.1 | 733 | 3490.5% | +| PHP | 23 | 824 | 150.1 | 801 | 3482.6% | +| TCL | 20 | 712 | 116.0 | 692 | 3460.0% | +| COMMONLISP | 27 | 956 | 122.1 | 929 | 3440.7% | --- @@ -488,6 +496,7 @@ Individual Reports → Aggregation Script → Chart Generation (via UN) → Fina - `reports/4.2.22/perf.json` - 697 tests, generated 2026-01-24T17:57:56Z - `reports/4.2.23/perf.json` - 713 tests, generated 2026-01-24T19:14:09Z - `reports/4.2.24/perf.json` - 665 tests, generated 2026-01-24T19:13:51Z +- `reports/4.2.25/perf.json` - 681 tests, generated 2026-01-24T21:04:06Z - `reports/4.2.3/perf.json` - 642 tests, generated 2026-01-19T11:58:45Z - `reports/4.2.4/perf.json` - 682 tests, generated 2026-01-19T12:02:14Z - `reports/4.2.5/perf.json` - 658 tests, generated 2026-01-19T19:10:23Z @@ -691,5 +700,5 @@ For questions about this methodology or to report issues: --- **Generated by UN Inception Performance Analysis Pipeline** -**Analysis Date:** 2026-01-24T15:45:42.561682 +**Analysis Date:** 2026-01-24T16:04:31.155396 **Report Version:** 1.0.0