diff --git a/AGGREGATED-PERFORMANCE.md b/AGGREGATED-PERFORMANCE.md index 5d53bdd..21454f0 100644 --- a/AGGREGATED-PERFORMANCE.md +++ b/AGGREGATED-PERFORMANCE.md @@ -1,13 +1,13 @@ # UN Inception: Aggregated Performance Analysis -**Analysis Date:** 1769202990.8940048 -**Reports Analyzed:** 4.2.0, 4.2.10, 4.2.11, 4.2.12, 4.2.13, 4.2.14, 4.2.15, 4.2.16, 4.2.17, 4.2.18, 4.2.19, 4.2.20, 4.2.21, 4.2.3, 4.2.4, 4.2.5, 4.2.6, 4.2.7, 4.2.8, 4.2.9 +**Analysis Date:** 1769203039.9327624 +**Reports Analyzed:** 4.2.20, 4.2.21 --- ## Executive Summary -Analysis of 20 performance reports reveals **significant variance** in execution metrics across releases. Different languages rank as slowest/fastest in different runs, indicating **non-deterministic execution patterns** likely caused by: +Analysis of 2 performance reports reveals **significant variance** in execution metrics across releases. Different languages rank as slowest/fastest in different runs, indicating **non-deterministic execution patterns** likely caused by: 1. **Orchestrator placement on CPU-bound pool** (not an SRE best practice) 2. **Resource contention** between the orchestrator & test jobs @@ -22,26 +22,8 @@ Analysis of 20 performance reports reveals **significant variance** in execution | Release | Avg Duration | Slowest | Fastest | Change from Previous | |---------|--------------|---------|---------|----------------------| -| 4.2.0 | 33s | raku (93s) | ocaml (19s) | baseline | -| 4.2.10 | 153s | scheme (320s) | bash (29s) | +120s (+363.6%) | -| 4.2.11 | 103s | deno (289s) | cpp (43s) | -50s (-32.7%) | -| 4.2.12 | 142s | go (406s) | erlang (21s) | +39s (+37.9%) | -| 4.2.13 | 126s | deno (290s) | haskell (23s) | -16s (-11.3%) | -| 4.2.14 | 97s | javascript (172s) | go (38s) | -29s (-23.0%) | -| 4.2.15 | 103s | d (203s) | cobol (49s) | +6s (+6.2%) | -| 4.2.16 | 98s | javascript (361s) | c (40s) | -5s (-4.9%) | -| 4.2.17 | 104s | ruby (270s) | objc (17s) | +6s (+6.1%) | -| 4.2.18 | 100s | r (298s) | scheme (48s) | -4s (-3.8%) | -| 4.2.19 | 151s | lua (472s) | dart (40s) | +51s (+51.0%) | -| 4.2.20 | 114s | java (272s) | crystal (49s) | -37s (-24.5%) | +| 4.2.20 | 114s | java (272s) | crystal (49s) | baseline | | 4.2.21 | 102s | nim (215s) | erlang (22s) | -12s (-10.5%) | -| 4.2.3 | 63s | rust (142s) | v (40s) | -39s (-38.2%) | -| 4.2.4 | 70s | python (110s) | c (23s) | +7s (+11.1%) | -| 4.2.5 | 67s | v (114s) | erlang (44s) | -3s (-4.3%) | -| 4.2.6 | 54s | haskell (128s) | awk (23s) | -13s (-19.4%) | -| 4.2.7 | 117s | typescript (319s) | dotnet (5s) | +63s (+116.7%) | -| 4.2.8 | 111s | kotlin (313s) | fortran (28s) | -6s (-5.1%) | -| 4.2.9 | 107s | ruby (279s) | d (19s) | -4s (-3.6%) | **Observation:** Average duration increased **0.0%** from 0s to 0s. @@ -57,120 +39,30 @@ This **2-3x variance** is NOT normal for identical workloads. Indicates: The same language changes dramatically in rank between runs: -**PERL:** - - 4.2.0: 26s - - 4.2.10: 105s - - 4.2.11: 58s - - 4.2.12: 55s - - 4.2.13: 108s - - 4.2.14: 140s - - 4.2.15: 64s - - 4.2.16: 82s - - 4.2.17: 38s - - 4.2.18: 93s - - 4.2.19: 468s - - 4.2.20: 191s - - 4.2.21: 104s - - 4.2.3: 66s - - 4.2.4: 77s - - 4.2.5: 53s - - 4.2.6: 44s - - 4.2.7: 155s - - 4.2.8: 91s - - 4.2.9: 158s - - **Range:** 26s → 468s (1700.0% variance) +**COBOL:** + - 4.2.20: 198s + - 4.2.21: 62s + - **Range:** 62s → 198s (219.4% variance) -**LUA:** - - 4.2.0: 32s - - 4.2.10: 51s - - 4.2.11: 56s - - 4.2.12: 53s - - 4.2.13: 203s - - 4.2.14: 127s - - 4.2.15: 166s - - 4.2.16: 242s - - 4.2.17: 40s - - 4.2.18: 73s - - 4.2.19: 472s - - 4.2.20: 133s - - 4.2.21: 183s - - 4.2.3: 63s - - 4.2.4: 71s - - 4.2.5: 52s - - 4.2.6: 36s - - 4.2.7: 152s - - 4.2.8: 84s - - 4.2.9: 56s - - **Range:** 32s → 472s (1375.0% variance) +**AWK:** + - 4.2.20: 145s + - 4.2.21: 33s + - **Range:** 33s → 145s (339.4% variance) -**JAVA:** - - 4.2.0: 36s - - 4.2.10: 122s - - 4.2.11: 133s - - 4.2.12: 116s - - 4.2.13: 240s - - 4.2.14: 105s - - 4.2.15: 60s - - 4.2.16: 52s - - 4.2.17: 84s - - 4.2.18: 92s - - 4.2.19: 423s - - 4.2.20: 272s +**NIM:** + - 4.2.20: 103s + - 4.2.21: 215s + - **Range:** 103s → 215s (108.7% variance) + +**TYPESCRIPT:** + - 4.2.20: 138s + - 4.2.21: 39s + - **Range:** 39s → 138s (253.8% variance) + +**CPP:** + - 4.2.20: 107s - 4.2.21: 204s - - 4.2.3: 67s - - 4.2.4: 57s - - 4.2.5: 62s - - 4.2.6: 70s - - 4.2.7: 33s - - 4.2.8: 45s - - 4.2.9: 219s - - **Range:** 33s → 423s (1181.8% variance) - -**CRYSTAL:** - - 4.2.0: 24s - - 4.2.10: 89s - - 4.2.11: 90s - - 4.2.12: 397s - - 4.2.13: 158s - - 4.2.14: 87s - - 4.2.15: 74s - - 4.2.16: 273s - - 4.2.17: 55s - - 4.2.18: 101s - - 4.2.19: 100s - - 4.2.20: 49s - - 4.2.21: 76s - - 4.2.3: 61s - - 4.2.4: 60s - - 4.2.5: 88s - - 4.2.6: 28s - - 4.2.7: 260s - - 4.2.8: 44s - - 4.2.9: 221s - - **Range:** 24s → 397s (1554.2% variance) - -**GROOVY:** - - 4.2.0: 23s - - 4.2.10: 151s - - 4.2.11: 90s - - 4.2.12: 393s - - 4.2.13: 39s - - 4.2.14: 93s - - 4.2.15: 69s - - 4.2.16: 44s - - 4.2.17: 151s - - 4.2.18: 150s - - 4.2.19: 115s - - 4.2.20: 75s - - 4.2.21: 75s - - 4.2.3: 60s - - 4.2.4: 59s - - 4.2.5: 70s - - 4.2.6: 40s - - 4.2.7: 36s - - 4.2.8: 41s - - 4.2.9: 98s - - **Range:** 23s → 393s (1608.7% variance) + - **Range:** 107s → 204s (90.7% variance) --- @@ -179,49 +71,13 @@ The same language changes dramatically in rank between runs: **Fastest Languages by Run:** -4.2.0: ocaml, tcl, elixir, csharp, cobol -4.2.10: bash, powershell, erlang, ruby, typescript -4.2.11: cpp, forth, lua, typescript, ruby -4.2.12: erlang, php, python, javascript, haskell -4.2.13: haskell, v, groovy, nim, kotlin -4.2.14: go, cpp, powershell, erlang, typescript -4.2.15: cobol, csharp, ocaml, objc, kotlin -4.2.16: cpp, c, raku, awk, groovy -4.2.17: objc, python, erlang, csharp, perl -4.2.18: scheme, tcl, fortran, c, raku -4.2.19: dart, python, typescript, javascript, dotnet 4.2.20: crystal, v, deno, r, csharp 4.2.21: erlang, r, ruby, awk, typescript -4.2.3: v, d, kotlin, awk, raku -4.2.4: c, d, cobol, raku, v -4.2.5: erlang, awk, bash, deno, tcl -4.2.6: awk, powershell, crystal, raku, erlang -4.2.7: dotnet, deno, awk, fortran, commonlisp -4.2.8: fortran, groovy, crystal, java, powershell -4.2.9: d, julia, csharp, v, objc **Slowest Languages by Run:** -4.2.0: raku, javascript, cpp, rust, go -4.2.10: scheme, clojure, deno, c, julia -4.2.11: deno, awk, erlang, elixir, clojure -4.2.12: go, crystal, groovy, deno, awk -4.2.13: deno, raku, awk, cpp, java -4.2.14: javascript, python, php, bash, elixir -4.2.15: d, cpp, ruby, bash, lua -4.2.16: javascript, clojure, crystal, lua, fsharp -4.2.17: ruby, typescript, php, cobol, commonlisp -4.2.18: r, go, elixir, rust, forth -4.2.19: lua, perl, java, ruby, powershell 4.2.20: java, zig, cobol, perl, haskell 4.2.21: nim, dart, java, cpp, rust -4.2.3: rust, c, python, typescript, javascript -4.2.4: python, javascript, elixir, scheme, bash -4.2.5: v, haskell, scheme, ocaml, powershell -4.2.6: haskell, go, cpp, rust, forth -4.2.7: typescript, ruby, r, elixir, crystal -4.2.8: kotlin, python, javascript, tcl, raku -4.2.9: ruby, deno, rust, crystal, java **Conclusion:** No consistent "fast" or "slow" languages across runs. This proves: - Execution order is random or system-dependent @@ -303,25 +159,25 @@ If concurrency was fixed at N parallel jobs: ### Most Variable Languages -PERL: 26s → 468s (+1700.0%) +COBOL: 62s → 198s (+219.4%) -LUA: 32s → 472s (+1375.0%) +AWK: 33s → 145s (+339.4%) -JAVA: 33s → 423s (+1181.8%) +NIM: 103s → 215s (+108.7%) -CRYSTAL: 24s → 397s (+1554.2%) +TYPESCRIPT: 39s → 138s (+253.8%) -GROOVY: 23s → 393s (+1608.7%) +CPP: 107s → 204s (+90.7%) -GO: 38s → 406s (+968.4%) +PERL: 104s → 191s (+83.7%) -JAVASCRIPT: 32s → 361s (+1028.1%) +BASH: 104s → 186s (+78.8%) -SCHEME: 24s → 320s (+1233.3%) +DART: 127s → 208s (+63.8%) -ELIXIR: 20s → 313s (+1465.0%) +RAKU: 83s → 154s (+85.5%) -TYPESCRIPT: 29s → 319s (+1000.0%) +FORTH: 73s → 142s (+94.5%) These languages are most affected by resource contention. Likely reasons: @@ -374,26 +230,26 @@ Keep it as-is for stress testing, but in separate test environment. | Language | Min (s) | Max (s) | Avg (s) | Range (s) | Variance % | |----------|---------|---------|---------|-----------|------------| -| DOTNET | 5 | 125 | 79.5 | 120 | 2400.0% | -| PERL | 26 | 468 | 108.8 | 442 | 1700.0% | -| GROOVY | 23 | 393 | 93.6 | 370 | 1608.7% | -| CRYSTAL | 24 | 397 | 116.8 | 373 | 1554.2% | -| ELIXIR | 20 | 313 | 117.7 | 293 | 1465.0% | -| CLOJURE | 21 | 310 | 110.0 | 289 | 1376.2% | -| LUA | 32 | 472 | 117.2 | 440 | 1375.0% | -| COBOL | 20 | 290 | 99.8 | 270 | 1350.0% | -| SCHEME | 24 | 320 | 104.2 | 296 | 1233.3% | -| PYTHON | 19 | 253 | 88.8 | 234 | 1231.6% | -| AWK | 23 | 306 | 115.8 | 283 | 1230.4% | -| OBJC | 17 | 223 | 88.8 | 206 | 1211.8% | -| R | 24 | 313 | 112.9 | 289 | 1204.2% | -| DENO | 24 | 310 | 124.8 | 286 | 1191.7% | -| JAVA | 33 | 423 | 124.6 | 390 | 1181.8% | -| OCAML | 19 | 240 | 96.7 | 221 | 1163.2% | -| TCL | 20 | 247 | 92.2 | 227 | 1135.0% | -| C | 23 | 276 | 95.8 | 253 | 1100.0% | -| CSHARP | 20 | 235 | 84.2 | 215 | 1075.0% | -| KOTLIN | 27 | 313 | 100.0 | 286 | 1059.3% | +| AWK | 33 | 145 | 89.0 | 112 | 339.4% | +| ERLANG | 22 | 79 | 50.5 | 57 | 259.1% | +| TYPESCRIPT | 39 | 138 | 88.5 | 99 | 253.8% | +| COBOL | 62 | 198 | 130.0 | 136 | 219.4% | +| RUBY | 30 | 89 | 59.5 | 59 | 196.7% | +| R | 24 | 57 | 40.5 | 33 | 137.5% | +| V | 56 | 121 | 88.5 | 65 | 116.1% | +| CSHARP | 58 | 123 | 90.5 | 65 | 112.1% | +| NIM | 103 | 215 | 159.0 | 112 | 108.7% | +| JAVASCRIPT | 42 | 87 | 64.5 | 45 | 107.1% | +| PYTHON | 43 | 88 | 65.5 | 45 | 104.7% | +| FORTH | 73 | 142 | 107.5 | 69 | 94.5% | +| CPP | 107 | 204 | 155.5 | 97 | 90.7% | +| C | 76 | 144 | 110.0 | 68 | 89.5% | +| RAKU | 83 | 154 | 118.5 | 71 | 85.5% | +| PERL | 104 | 191 | 147.5 | 87 | 83.7% | +| BASH | 104 | 186 | 145.0 | 82 | 78.8% | +| FSHARP | 73 | 125 | 99.0 | 52 | 71.2% | +| DART | 127 | 208 | 167.5 | 81 | 63.8% | +| CRYSTAL | 49 | 76 | 62.5 | 27 | 55.1% | --- @@ -448,26 +304,8 @@ Individual Reports → Aggregation Script → Chart Generation (via UN) → Fina ### Data Sources **Input Files:** -- `reports/4.2.0/perf.json` - 642 tests, generated 2026-01-18T23:20:51Z -- `reports/4.2.10/perf.json` - 673 tests, generated 2026-01-23T11:46:18Z -- `reports/4.2.11/perf.json` - 669 tests, generated 2026-01-23T12:14:08Z -- `reports/4.2.12/perf.json` - 665 tests, generated 2026-01-23T13:30:32Z -- `reports/4.2.13/perf.json` - 673 tests, generated 2026-01-23T14:19:49Z -- `reports/4.2.14/perf.json` - 661 tests, generated 2026-01-23T14:48:26Z -- `reports/4.2.15/perf.json` - 657 tests, generated 2026-01-23T15:14:36Z -- `reports/4.2.16/perf.json` - 665 tests, generated 2026-01-23T15:25:53Z -- `reports/4.2.17/perf.json` - 665 tests, generated 2026-01-23T15:34:55Z -- `reports/4.2.18/perf.json` - 665 tests, generated 2026-01-23T16:05:03Z -- `reports/4.2.19/perf.json` - 701 tests, generated 2026-01-23T20:20:06Z - `reports/4.2.20/perf.json` - 685 tests, generated 2026-01-23T20:41:23Z - `reports/4.2.21/perf.json` - 661 tests, generated 2026-01-23T21:16:07Z -- `reports/4.2.3/perf.json` - 642 tests, generated 2026-01-19T11:58:45Z -- `reports/4.2.4/perf.json` - 682 tests, generated 2026-01-19T12:02:14Z -- `reports/4.2.5/perf.json` - 658 tests, generated 2026-01-19T19:10:23Z -- `reports/4.2.6/perf.json` - 642 tests, generated 2026-01-19T20:22:16Z -- `reports/4.2.7/perf.json` - 631 tests, generated 2026-01-23T09:36:18Z -- `reports/4.2.8/perf.json` - 645 tests, generated 2026-01-23T10:01:33Z -- `reports/4.2.9/perf.json` - 645 tests, generated 2026-01-23T10:05:34Z Each `perf.json` contains: @@ -664,5 +502,5 @@ For questions about this methodology or to report issues: --- **Generated by UN Inception Performance Analysis Pipeline** -**Analysis Date:** 2026-01-23T16:16:31.013605 +**Analysis Date:** 2026-01-23T16:21:28.533747 **Report Version:** 1.0.0 diff --git a/VERSION b/VERSION index d82f113..2869c55 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -4.2.21 +4.2.22 diff --git a/clients/c/src/un.c b/clients/c/src/un.c index 5b219fc..8c92f5c 100644 --- a/clients/c/src/un.c +++ b/clients/c/src/un.c @@ -6013,7 +6013,7 @@ void print_usage(const char *prog) { * ============================================================================ */ const char *unsandbox_version(void) { - return "4.2.21"; + return "4.2.22"; } const char *unsandbox_detect_language(const char *filename) { diff --git a/clients/dotnet/async/src/Un.cs b/clients/dotnet/async/src/Un.cs index 27b6611..014d885 100644 --- a/clients/dotnet/async/src/Un.cs +++ b/clients/dotnet/async/src/Un.cs @@ -12,7 +12,7 @@ using System.Text.Json.Serialization; const string API_BASE = "https://api.unsandbox.com"; const string PORTAL_BASE = "https://unsandbox.com"; -const string VERSION = "4.2.6"; +const string VERSION = "4.2.22"; // ANSI colors const string BLUE = "\x1B[34m"; diff --git a/clients/dotnet/sync/src/Un.cs b/clients/dotnet/sync/src/Un.cs index c25144c..0502074 100644 --- a/clients/dotnet/sync/src/Un.cs +++ b/clients/dotnet/sync/src/Un.cs @@ -12,7 +12,7 @@ using System.Text.Json.Serialization; const string API_BASE = "https://api.unsandbox.com"; const string PORTAL_BASE = "https://unsandbox.com"; -const string VERSION = "4.2.6"; +const string VERSION = "4.2.22"; // ANSI colors const string BLUE = "\x1B[34m"; diff --git a/clients/groovy/sync/src/un.groovy b/clients/groovy/sync/src/un.groovy index 27875de..8dc4719 100644 --- a/clients/groovy/sync/src/un.groovy +++ b/clients/groovy/sync/src/un.groovy @@ -72,7 +72,7 @@ * * * @author Permacomputer Project - * @version 4.2.21 + * @version 4.2.22 */ import javax.crypto.Mac diff --git a/clients/javascript/async/package.json b/clients/javascript/async/package.json index 5868f84..2e28c1a 100644 --- a/clients/javascript/async/package.json +++ b/clients/javascript/async/package.json @@ -1,6 +1,6 @@ { "name": "un-async", - "version": "4.2.21", + "version": "4.2.22", "description": "Unsandbox async JavaScript SDK - Execute code in 50+ languages", "main": "src/un_async.js", "type": "module", diff --git a/clients/javascript/sync/package.json b/clients/javascript/sync/package.json index 37b23fe..d185368 100644 --- a/clients/javascript/sync/package.json +++ b/clients/javascript/sync/package.json @@ -1,6 +1,6 @@ { "name": "un-sync", - "version": "4.2.21", + "version": "4.2.22", "description": "unsandbox.com JavaScript SDK (Isomorphic - Node.js + Browser)", "type": "module", "main": "src/un.js", diff --git a/clients/lua/sync/src/un.lua b/clients/lua/sync/src/un.lua index 82971e7..3971ddb 100644 --- a/clients/lua/sync/src/un.lua +++ b/clients/lua/sync/src/un.lua @@ -15,7 +15,7 @@ local ltn12 = require("ltn12") local Un = {} Un.API_BASE = "https://api.unsandbox.com" -Un.VERSION = "4.2.21" +Un.VERSION = "4.2.22" -- Credential loading function Un.load_accounts_csv(path) diff --git a/clients/perl/sync/src/un.pl b/clients/perl/sync/src/un.pl index 7d79001..e98d0bf 100644 --- a/clients/perl/sync/src/un.pl +++ b/clients/perl/sync/src/un.pl @@ -47,7 +47,7 @@ use Digest::HMAC_SHA256 qw(hmac_sha256_hex); use File::HomeDir; use Time::HiRes qw(time sleep); -our $VERSION = "4.2.21"; +our $VERSION = "4.2.22"; our $API_BASE = 'https://api.unsandbox.com'; # Credential system diff --git a/clients/python/async/setup.py b/clients/python/async/setup.py index ac11ce1..d8ce547 100644 --- a/clients/python/async/setup.py +++ b/clients/python/async/setup.py @@ -7,7 +7,7 @@ from setuptools import setup, find_packages setup( name="unsandbox-async", - version="4.2.21", + version="4.2.22", description="Asynchronous Python SDK for unsandbox.com code execution", long_description=open("README.md").read() if False else "Async Python SDK for unsandbox code execution", author="unsandbox.com", diff --git a/clients/python/sync/setup.py b/clients/python/sync/setup.py index 28545f2..8eff16f 100644 --- a/clients/python/sync/setup.py +++ b/clients/python/sync/setup.py @@ -7,7 +7,7 @@ with open("README.md", "r", encoding="utf-8") as fh: setup( name="unsandbox", - version="4.2.21", + version="4.2.22", author="Unsandbox", description="Synchronous Python SDK for unsandbox.com code execution", long_description=long_description, diff --git a/clients/python/sync/src/__init__.py b/clients/python/sync/src/__init__.py index 5315d7c..ed6ec05 100644 --- a/clients/python/sync/src/__init__.py +++ b/clients/python/sync/src/__init__.py @@ -27,7 +27,7 @@ from .un import ( CredentialsError, ) -__version__ = "4.2.21" +__version__ = "4.2.22" __all__ = [ "execute_code", "execute_async", diff --git a/clients/rust/async/Cargo.toml b/clients/rust/async/Cargo.toml index 3cda075..2cab239 100644 --- a/clients/rust/async/Cargo.toml +++ b/clients/rust/async/Cargo.toml @@ -7,7 +7,7 @@ [package] name = "un-async" -version = "4.2.21" +version = "4.2.22" edition = "2021" authors = ["unsandbox.com"] description = "Asynchronous Rust SDK for unsandbox.com secure code execution API" diff --git a/clients/rust/sync/Cargo.toml b/clients/rust/sync/Cargo.toml index 309a822..341af4d 100644 --- a/clients/rust/sync/Cargo.toml +++ b/clients/rust/sync/Cargo.toml @@ -7,7 +7,7 @@ [package] name = "un-sync" -version = "4.2.21" +version = "4.2.22" edition = "2021" authors = ["unsandbox.com"] description = "Synchronous Rust SDK for unsandbox.com secure code execution API"