When API returns 200 but output doesn't match expected pattern,
retry up to 3 times. This handles cases where service operations
succeed but the service isn't fully ready yet.
- generate-perf-report.sh now collects api-health.json from test artifacts
- Aggregates retry counts by type (429, 5xx, timeout, connection)
- Calculates API health score (100 = perfect, decreases with retries)
- Adds API Health section to perf.md with interpretation guide
- aggregate-performance-reports.py now shows API health trends over time
- Tracks scientific integrity improvements starting from 4.2.34
Track retry counts by error type (429, 5xx, timeout, connection) and
output API Health Report at end of each test run. Writes api-health.json
for aggregation across all 42 SDKs.
This lets us monitor API reliability over time and identify patterns.
SCIENTIFIC INTEGRITY: Tests now retry transient errors (429, 5xx,
timeouts) with exponential backoff instead of falsely passing.
Prior to this fix, ~35% of 'passing' tests were masked failures.
Now tests will PASS (verified), FAIL (couldn't verify), or SKIP.
- Fortran: use dlopen/dlsym at runtime instead of direct linking
(sandbox doesn't pass -lqrencode to gfortran)
- Both: try libqrencode.so.4, .so.3, .so in order
- Perl: use Text::QRCode (pure Perl, pre-installed) instead of Imager::QRCode
- Elixir: try Erlang :qrcode module, fallback to CLI
- OCaml: use Unix.open_process_in (no ctypes in compiled mode)
- Fortran: direct C FFI binding (requires -lqrencode at link)
With -s, the un CLI treats the positional argument as inline code text.
Without -s, it reads the file from disk and auto-detects the language
from the extension. The QR test was failing across 18 languages because
test/qr.py was being interpreted as code (test divided by qr.py).
Native QR generation for each language's ecosystem:
- 27 languages use package manager QR libraries (qrcode, rqrcode, ZXing, etc.)
- 10 languages use C FFI to libqrencode (dlopen, ISO_C_BINDING, @cImport, critcl)
- 2 shell languages use qrencode CLI (native shell paradigm)
- 1 language (Prolog) uses C FFI with CLI fallback
Section 9 added to test-sdk.sh with get_qr_file() mapping and
QR:unsandbox-qr-ok:ROWS pattern validation across the hydra matrix.
- New 'manual-matrix-run' job with play button in GitLab UI
- Click to run full 42-language matrix without creating a tag
- Useful for load testing, pool churn, and QR lib testing
- Added --force-all flag to generate-matrix.sh
Add CLI support for toggling the show_freeze_page service setting,
which controls whether frozen services display an HTML payment page
or return a JSON error response.
Updated 18 SDK implementations + CLI_SPEC.md documentation.
- API and orchestrator now on dedicated infrastructure (separated from compute)
- Two pool types: Xeon (32 vCPU, 300GB RAM) and i9 (32 vCPU, 32GB RAM)
- 4.2.27: 119s avg, 4.2.28: 116s avg (-2.5%)
