Add --golden-image and -f/--file CLI args to service command.
Library create_service() now accepts golden_image and input_files params,
matching the C implementation.
QR tests MUST use native language libraries, not qrencode CLI.
If the sandbox doesn't have the library, the test should FAIL
honestly so we know what to fix in the sandbox image.
Also adds CLAUDE.md rule to prevent future attempts to replace
native library QR tests with CLI subprocess calls.
- detect-changes: VERSION/Makefile changes trigger full test matrix
- test-sdk: fix service_destroy treating "not found" as success
- test-sdk: skip qr_generate on missing deps instead of failing
- test-sdk: increase backoff cap from 10s to 30s for overloaded pools
- test-sdk: increase no-match retry from 3 to 5 with longer delay
- test-sdk: detect nginx error messages (bad gateway, etc) as 5xx
- test-sdk: show HTTP status and error details on test failure
Use a single service name per test run instead of unique-per-attempt.
If a 5xx hides a successful create, the retry gets a 409 and we look
up the existing service ID instead of creating another orphan.
- Remove 'local' outside function (line 442)
- Reduce max retry from 10 to 5 for service_create
- Cap backoff at 10s instead of 60s (both run_test and service_create)
- Generate unique service name per attempt (fixes 409 conflicts)
- Handle HTTP 409 as retriable error
When API returns 200 but output doesn't match expected pattern,
retry up to 3 times. This handles cases where service operations
succeed but the service isn't fully ready yet.
- generate-perf-report.sh now collects api-health.json from test artifacts
- Aggregates retry counts by type (429, 5xx, timeout, connection)
- Calculates API health score (100 = perfect, decreases with retries)
- Adds API Health section to perf.md with interpretation guide
- aggregate-performance-reports.py now shows API health trends over time
- Tracks scientific integrity improvements starting from 4.2.34
Track retry counts by error type (429, 5xx, timeout, connection) and
output API Health Report at end of each test run. Writes api-health.json
for aggregation across all 42 SDKs.
This lets us monitor API reliability over time and identify patterns.
SCIENTIFIC INTEGRITY: Tests now retry transient errors (429, 5xx,
timeouts) with exponential backoff instead of falsely passing.
Prior to this fix, ~35% of 'passing' tests were masked failures.
Now tests will PASS (verified), FAIL (couldn't verify), or SKIP.
