engineering/un-inception
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

perf: Update aggregated performance analysis [ci skip]

Browse source
This commit is contained in:
GitLab CI 2026-02-14 10:57:36 -05:00
parent 9c91d8bbf7
commit 2fd5b5617f
1 changed files with 33 additions and 177 deletions
Download patch file Download diff file Expand all files Collapse all files

210
AGGREGATED-PERFORMANCE.md
View file

@ -1,31 +1,13 @@
# UN Inception: Aggregated Performance Analysis
<<<<<<< Updated upstream
<<<<<<< Updated upstream
**Analysis Date:** 1770551195.2485962
**Reports Analyzed:** 4.2.0, 4.2.10, 4.2.11, 4.2.12, 4.2.13, 4.2.14, 4.2.15, 4.2.16, 4.2.17, 4.2.18, 4.2.19, 4.2.20, 4.2.21, 4.2.22, 4.2.23, 4.2.24, 4.2.25, 4.2.26, 4.2.27, 4.2.28, 4.2.29, 4.2.3, 4.2.30, 4.2.31, 4.2.32, 4.2.36, 4.2.37, 4.2.38, 4.2.4, 4.2.46, 4.2.5, 4.2.50, 4.2.51, 4.2.52, 4.2.6, 4.2.7, 4.2.8, 4.2.9, 4.3.0, 4.3.1
=======
**Analysis Date:** 1770575731.1513839
**Reports Analyzed:** 4.2.0, 4.2.10, 4.2.11, 4.2.12, 4.2.13, 4.2.14, 4.2.15, 4.2.16, 4.2.17, 4.2.18, 4.2.19, 4.2.20, 4.2.21, 4.2.22, 4.2.23, 4.2.24, 4.2.25, 4.2.26, 4.2.27, 4.2.28, 4.2.29, 4.2.3, 4.2.30, 4.2.31, 4.2.32, 4.2.36, 4.2.37, 4.2.38, 4.2.4, 4.2.46, 4.2.5, 4.2.50, 4.2.51, 4.2.52, 4.2.6, 4.2.7, 4.2.8, 4.2.9, 4.3.0, 4.3.1, 4.3.2
>>>>>>> Stashed changes
=======
**Analysis Date:** 1770578949.9558537
**Reports Analyzed:** 4.2.0, 4.2.10, 4.2.11, 4.2.12, 4.2.13, 4.2.14, 4.2.15, 4.2.16, 4.2.17, 4.2.18, 4.2.19, 4.2.20, 4.2.21, 4.2.22, 4.2.23, 4.2.24, 4.2.25, 4.2.26, 4.2.27, 4.2.28, 4.2.29, 4.2.3, 4.2.30, 4.2.31, 4.2.32, 4.2.36, 4.2.37, 4.2.38, 4.2.4, 4.2.46, 4.2.5, 4.2.50, 4.2.51, 4.2.52, 4.2.6, 4.2.7, 4.2.8, 4.2.9, 4.3.0, 4.3.1, 4.3.2, 4.3.3
>>>>>>> Stashed changes
**Analysis Date:** 1771084652.659423
**Reports Analyzed:** 4.2.0, 4.2.10, 4.2.11, 4.2.12, 4.2.13, 4.2.14, 4.2.15, 4.2.16, 4.2.17, 4.2.18, 4.2.19, 4.2.20, 4.2.21, 4.2.22, 4.2.23, 4.2.24, 4.2.25, 4.2.26, 4.2.27, 4.2.28, 4.2.29, 4.2.3, 4.2.30, 4.2.31, 4.2.32, 4.2.36, 4.2.37, 4.2.38, 4.2.4, 4.2.46, 4.2.5, 4.2.50, 4.2.51, 4.2.52, 4.2.6, 4.2.7, 4.2.8, 4.2.9, 4.3.0, 4.3.1, 4.3.2, 4.3.3, 4.3.4
---
## Executive Summary
<<<<<<< Updated upstream
<<<<<<< Updated upstream
Analysis of 40 performance reports reveals **significant variance** in execution metrics across releases. Different languages rank as slowest/fastest in different runs, indicating **non-deterministic execution patterns** likely caused by:
=======
Analysis of 41 performance reports reveals **significant variance** in execution metrics across releases. Different languages rank as slowest/fastest in different runs, indicating **non-deterministic execution patterns** likely caused by:
>>>>>>> Stashed changes
=======
Analysis of 42 performance reports reveals **significant variance** in execution metrics across releases. Different languages rank as slowest/fastest in different runs, indicating **non-deterministic execution patterns** likely caused by:
>>>>>>> Stashed changes
Analysis of 43 performance reports reveals **significant variance** in execution metrics across releases. Different languages rank as slowest/fastest in different runs, indicating **non-deterministic execution patterns** likely caused by:
1. **Orchestrator placement on CPU-bound pool** (not an SRE best practice)
2. **Resource contention** between the orchestrator & test jobs
@ -80,15 +62,9 @@ Analysis of 42 performance reports reveals **significant variance** in execution
| 4.2.9 | 107s | ruby (279s) | d (19s) | -4s (-3.6%) |
| 4.3.0 | 376s | typescript (829s) | c (47s) | +269s (+251.4%) |
| 4.3.1 | 238s | go (414s) | prolog (58s) | -138s (-36.7%) |
<<<<<<< Updated upstream
<<<<<<< Updated upstream
=======
| 4.3.2 | 132s | crystal (314s) | c (66s) | -106s (-44.5%) |
>>>>>>> Stashed changes
=======
| 4.3.2 | 132s | crystal (314s) | c (66s) | -106s (-44.5%) |
| 4.3.3 | 175s | go (447s) | v (65s) | +43s (+32.6%) |
>>>>>>> Stashed changes
| 4.3.4 | 204s | perl (456s) | php (49s) | +29s (+16.6%) |
**Observation:** Average duration increased **0.0%** from 0s to 0s.
@ -145,15 +121,9 @@ The same language changes dramatically in rank between runs:
- 4.2.9: 166s
- 4.3.0: 196s
- 4.3.1: 164s
<<<<<<< Updated upstream
<<<<<<< Updated upstream
=======
- 4.3.2: 193s
>>>>>>> Stashed changes
=======
- 4.3.2: 193s
- 4.3.3: 202s
>>>>>>> Stashed changes
- 4.3.4: 198s
- **Range:** 32s → 2173s (6690.6% variance)
**R:**
@ -197,15 +167,9 @@ The same language changes dramatically in rank between runs:
- 4.2.9: 54s
- 4.3.0: 453s
- 4.3.1: 163s
<<<<<<< Updated upstream
<<<<<<< Updated upstream
=======
- 4.3.2: 179s
>>>>>>> Stashed changes
=======
- 4.3.2: 179s
- 4.3.3: 81s
>>>>>>> Stashed changes
- 4.3.4: 106s
- **Range:** 9s → 1834s (20277.8% variance)
**SCHEME:**
@ -249,15 +213,9 @@ The same language changes dramatically in rank between runs:
- 4.2.9: 153s
- 4.3.0: 171s
- 4.3.1: 276s
<<<<<<< Updated upstream
<<<<<<< Updated upstream
=======
- 4.3.2: 132s
>>>>>>> Stashed changes
=======
- 4.3.2: 132s
- 4.3.3: 196s
>>>>>>> Stashed changes
- 4.3.4: 148s
- **Range:** 15s → 1574s (10393.3% variance)
**PYTHON:**
@ -301,15 +259,9 @@ The same language changes dramatically in rank between runs:
- 4.2.9: 165s
- 4.3.0: 671s
- 4.3.1: 148s
<<<<<<< Updated upstream
<<<<<<< Updated upstream
=======
- 4.3.2: 85s
>>>>>>> Stashed changes
=======
- 4.3.2: 85s
- 4.3.3: 158s
>>>>>>> Stashed changes
- 4.3.4: 307s
- **Range:** 19s → 1574s (8184.2% variance)
**TCL:**
@ -353,15 +305,9 @@ The same language changes dramatically in rank between runs:
- 4.2.9: 54s
- 4.3.0: 476s
- 4.3.1: 132s
<<<<<<< Updated upstream
<<<<<<< Updated upstream
=======
- 4.3.2: 80s
>>>>>>> Stashed changes
=======
- 4.3.2: 80s
- 4.3.3: 186s
>>>>>>> Stashed changes
- 4.3.4: 247s
- **Range:** 20s → 1572s (7760.0% variance)
@ -411,15 +357,9 @@ The same language changes dramatically in rank between runs:
4.2.9: d, julia, csharp, v, objc
4.3.0: c, bash, php, fortran, ruby
4.3.1: prolog, awk, powershell, typescript, dart
<<<<<<< Updated upstream
<<<<<<< Updated upstream
=======
4.3.2: c, cpp, fsharp, perl, csharp
>>>>>>> Stashed changes
=======
4.3.2: c, cpp, fsharp, perl, csharp
4.3.3: v, r, dart, perl, rust
>>>>>>> Stashed changes
4.3.4: php, powershell, v, bash, fortran
**Slowest Languages by Run:**
@ -463,15 +403,9 @@ The same language changes dramatically in rank between runs:
4.2.9: ruby, deno, rust, crystal, java
4.3.0: typescript, go, python, java, objc
4.3.1: go, groovy, perl, deno, objc
<<<<<<< Updated upstream
<<<<<<< Updated upstream
=======
4.3.2: crystal, erlang, elixir, rust, groovy
>>>>>>> Stashed changes
=======
4.3.2: crystal, erlang, elixir, rust, groovy
4.3.3: go, crystal, raku, typescript, kotlin
>>>>>>> Stashed changes
4.3.4: perl, ruby, go, cobol, kotlin
**Conclusion:** No consistent "fast" or "slow" languages across runs. This proves:
- Execution order is random or system-dependent
@ -482,21 +416,9 @@ The same language changes dramatically in rank between runs:
### 4. API Health Trends
<<<<<<< Updated upstream
<<<<<<< Updated upstream
**Overall API Health:** 6.9/100 (avg across 9 releases)
**Overall API Health:** 5.2/100 (avg across 12 releases)
**Trend:** STABLE
**Total Retries (all releases):** 4293
=======
**Overall API Health:** 6.2/100 (avg across 10 releases)
**Trend:** STABLE
**Total Retries (all releases):** 4510
>>>>>>> Stashed changes
=======
**Overall API Health:** 5.6/100 (avg across 11 releases)
**Trend:** STABLE
**Total Retries (all releases):** 4840
>>>>>>> Stashed changes
**Total Retries (all releases):** 5332
| Release | Health Score | Total Retries | 429 (Rate Limit) | 5xx (Server) | Timeout | Connection |
|---------|--------------|---------------|------------------|--------------|---------|------------|
@ -509,15 +431,9 @@ The same language changes dramatically in rank between runs:
| 4.2.52 | 34/100 | 33 | 0 | 33 | 0 | 0 |
| 4.3.0 | 0/100 | 876 | 839 | 27 | 10 | 0 |
| 4.3.1 | 0/100 | 649 | 634 | 5 | 10 | 0 |
<<<<<<< Updated upstream
<<<<<<< Updated upstream
=======
| 4.3.2 | 0/100 | 217 | 207 | 0 | 10 | 0 |
>>>>>>> Stashed changes
=======
| 4.3.2 | 0/100 | 217 | 207 | 0 | 10 | 0 |
| 4.3.3 | 0/100 | 330 | 320 | 0 | 10 | 0 |
>>>>>>> Stashed changes
| 4.3.4 | 0/100 | 492 | 482 | 0 | 10 | 0 |
**Interpretation:**
- **Score 95-100:** API healthy, tests pass on first attempt
@ -673,72 +589,26 @@ Keep it as-is for stress testing, but in separate test environment.
| Language | Min (s) | Max (s) | Avg (s) | Range (s) | Variance % |
|----------|---------|---------|---------|-----------|------------|
<<<<<<< Updated upstream
<<<<<<< Updated upstream
| DOTNET | 5 | 1539 | 173.6 | 1534 | 30680.0% |
| R | 9 | 1834 | 227.0 | 1825 | 20277.8% |
| NIM | 8 | 1557 | 175.9 | 1549 | 19362.5% |
| CLOJURE | 8 | 1549 | 188.4 | 1541 | 19262.5% |
| FORTRAN | 8 | 1547 | 139.2 | 1539 | 19237.5% |
| PERL | 8 | 1546 | 182.1 | 1538 | 19225.0% |
| D | 8 | 1545 | 141.9 | 1537 | 19212.5% |
| ZIG | 8 | 1542 | 218.8 | 1534 | 19175.0% |
| FORTH | 8 | 1540 | 172.7 | 1532 | 19150.0% |
| KOTLIN | 8 | 1536 | 161.9 | 1528 | 19100.0% |
| CSHARP | 8 | 1534 | 162.4 | 1526 | 19075.0% |
| LUA | 9 | 1548 | 209.3 | 1539 | 17100.0% |
| PROLOG | 9 | 1540 | 124.2 | 1531 | 17011.1% |
| RUST | 9 | 1537 | 191.5 | 1528 | 16977.8% |
| SCHEME | 15 | 1574 | 168.0 | 1559 | 10393.3% |
| OBJC | 17 | 1559 | 168.4 | 1542 | 9070.6% |
| POWERSHELL | 14 | 1269 | 128.0 | 1255 | 8964.3% |
| PYTHON | 19 | 1574 | 174.8 | 1555 | 8184.2% |
| OCAML | 19 | 1537 | 168.4 | 1518 | 7989.5% |
| TCL | 20 | 1572 | 187.1 | 1552 | 7760.0% |
=======
| DOTNET | 5 | 1539 | 171.1 | 1534 | 30680.0% |
| R | 9 | 1834 | 225.8 | 1825 | 20277.8% |
| NIM | 8 | 1557 | 174.4 | 1549 | 19362.5% |
| CLOJURE | 8 | 1549 | 186.8 | 1541 | 19262.5% |
| DOTNET | 5 | 1539 | 171.3 | 1534 | 30680.0% |
| R | 9 | 1834 | 219.7 | 1825 | 20277.8% |
| NIM | 8 | 1557 | 173.6 | 1549 | 19362.5% |
| CLOJURE | 8 | 1549 | 190.6 | 1541 | 19262.5% |
| FORTRAN | 8 | 1547 | 138.9 | 1539 | 19237.5% |
| PERL | 8 | 1546 | 179.5 | 1538 | 19225.0% |
| D | 8 | 1545 | 142.1 | 1537 | 19212.5% |
| ZIG | 8 | 1542 | 216.1 | 1534 | 19175.0% |
| FORTH | 8 | 1540 | 172.7 | 1532 | 19150.0% |
| KOTLIN | 8 | 1536 | 161.9 | 1528 | 19100.0% |
| CSHARP | 8 | 1534 | 160.4 | 1526 | 19075.0% |
| LUA | 9 | 1548 | 206.3 | 1539 | 17100.0% |
| PROLOG | 9 | 1540 | 123.6 | 1531 | 17011.1% |
| RUST | 9 | 1537 | 192.1 | 1528 | 16977.8% |
| SCHEME | 15 | 1574 | 167.1 | 1559 | 10393.3% |
| OBJC | 17 | 1559 | 168.6 | 1542 | 9070.6% |
| POWERSHELL | 14 | 1269 | 127.1 | 1255 | 8964.3% |
| PYTHON | 19 | 1574 | 172.6 | 1555 | 8184.2% |
| OCAML | 19 | 1537 | 166.4 | 1518 | 7989.5% |
| TCL | 20 | 1572 | 184.5 | 1552 | 7760.0% |
>>>>>>> Stashed changes
=======
| DOTNET | 5 | 1539 | 169.9 | 1534 | 30680.0% |
| R | 9 | 1834 | 222.4 | 1825 | 20277.8% |
| NIM | 8 | 1557 | 172.4 | 1549 | 19362.5% |
| CLOJURE | 8 | 1549 | 188.9 | 1541 | 19262.5% |
| FORTRAN | 8 | 1547 | 140.0 | 1539 | 19237.5% |
| PERL | 8 | 1546 | 177.3 | 1538 | 19225.0% |
| D | 8 | 1545 | 141.9 | 1537 | 19212.5% |
| ZIG | 8 | 1542 | 213.6 | 1534 | 19175.0% |
| PERL | 8 | 1546 | 183.8 | 1538 | 19225.0% |
| D | 8 | 1545 | 142.7 | 1537 | 19212.5% |
| ZIG | 8 | 1542 | 215.1 | 1534 | 19175.0% |
| FORTH | 8 | 1540 | 172.4 | 1532 | 19150.0% |
| KOTLIN | 8 | 1536 | 164.7 | 1528 | 19100.0% |
| KOTLIN | 8 | 1536 | 168.5 | 1528 | 19100.0% |
| CSHARP | 8 | 1534 | 159.8 | 1526 | 19075.0% |
| LUA | 9 | 1548 | 206.1 | 1539 | 17100.0% |
| PROLOG | 9 | 1540 | 125.1 | 1531 | 17011.1% |
| RUST | 9 | 1537 | 189.6 | 1528 | 16977.8% |
| SCHEME | 15 | 1574 | 167.8 | 1559 | 10393.3% |
| OBJC | 17 | 1559 | 167.3 | 1542 | 9070.6% |
| POWERSHELL | 14 | 1269 | 126.6 | 1255 | 8964.3% |
| PYTHON | 19 | 1574 | 172.2 | 1555 | 8184.2% |
| OCAML | 19 | 1537 | 166.6 | 1518 | 7989.5% |
| TCL | 20 | 1572 | 184.5 | 1552 | 7760.0% |
>>>>>>> Stashed changes
| LUA | 9 | 1548 | 208.0 | 1539 | 17100.0% |
| PROLOG | 9 | 1540 | 127.8 | 1531 | 17011.1% |
| RUST | 9 | 1537 | 189.3 | 1528 | 16977.8% |
| SCHEME | 15 | 1574 | 167.3 | 1559 | 10393.3% |
| OBJC | 17 | 1559 | 165.9 | 1542 | 9070.6% |
| POWERSHELL | 14 | 1269 | 124.9 | 1255 | 8964.3% |
| PYTHON | 19 | 1574 | 175.4 | 1555 | 8184.2% |
| OCAML | 19 | 1537 | 168.0 | 1518 | 7989.5% |
| TCL | 20 | 1572 | 186.0 | 1552 | 7760.0% |
---
@ -833,15 +703,9 @@ Individual Reports → Aggregation Script → Chart Generation (via UN) → Fina
- `reports/4.2.9/perf.json` - 645 tests, generated 2026-01-23T10:05:34Z
- `reports/4.3.0/perf.json` - 820 tests, generated 2026-02-06T15:25:06Z
- `reports/4.3.1/perf.json` - 824 tests, generated 2026-02-08T11:44:58Z
<<<<<<< Updated upstream
<<<<<<< Updated upstream
=======
- `reports/4.3.2/perf.json` - 860 tests, generated 2026-02-08T18:34:37Z
>>>>>>> Stashed changes
=======
- `reports/4.3.2/perf.json` - 860 tests, generated 2026-02-08T18:34:37Z
- `reports/4.3.3/perf.json` - 848 tests, generated 2026-02-08T19:28:06Z
>>>>>>> Stashed changes
- `reports/4.3.4/perf.json` - 844 tests, generated 2026-02-14T15:56:41Z
Each `perf.json` contains:
@ -1038,13 +902,5 @@ For questions about this methodology or to report issues:
---
**Generated by UN Inception Performance Analysis Pipeline**
<<<<<<< Updated upstream
<<<<<<< Updated upstream
**Analysis Date:** 2026-02-08T06:46:35.375632
=======
**Analysis Date:** 2026-02-08T13:35:31.282306
>>>>>>> Stashed changes
=======
**Analysis Date:** 2026-02-08T14:29:10.036207
>>>>>>> Stashed changes
**Analysis Date:** 2026-02-14T10:57:32.791573
**Report Version:** 1.0.0