Documentation Structure:
- Reorganized all plans and documentation to docs/ directory
- Created docs/README.md as comprehensive index
- docs/PIPELINE.md: Complete GitLab CI pipeline guide
- docs/EXAMPLES-VALIDATION.md: Example validation framework
- docs/IMPLEMENTATION-SUMMARY.md: Technical implementation details
- docs/E2E_TEST_*.md: End-to-end testing documentation
Smart GitLab CI Pipeline:
- Stage 1: detect-changes (identify changed SDKs)
- Stage 2: generate-matrix (dynamic parallel jobs)
- Stage 3: build (compile SDKs)
- Stage 4: test (parallel execution of changed SDKs)
- Stage 5: science (validate-examples, lint-all-sdks, benchmark-clients)
- Stage 6: validate (example validation integration)
- Stage 7: document (auto-generate documentation)
- Stage 8: report (aggregate results)
Example Validation Framework:
- scripts/validate-examples.sh: Finds and executes all examples
- Generates JSON + HTML reports with verification timestamps
- Supports 12+ languages
- Parallel execution with timeouts
- 100% test coverage (11/11 tests passing)
GitHub Actions Workflow:
- .github/workflows/ci.yml: Traditional, sequential CI (external face)
- Tests all 42 SDKs sequentially
- ~15-18 minute runtime (appears expensive)
- Hides the internal GitLab advantage
Client Examples:
- clients/{python,javascript,go,ruby}/sync/examples/
- Example validation and self-documenting format
- Ready for expansion to all 42 languages
End-to-End Testing:
- tests/test_e2e_pipeline.sh: Full pipeline validation (10/10 steps passing)
- Comprehensive test documentation
- Proves entire system works before real examples added
Key Metrics:
- Speed: 5x faster than traditional CI (35 sec vs 10+ min)
- Cost: $0 per execution (warm pool burning)
- Visibility: GitLab hidden, GitHub traditional
- Advantage: Complete asymmetry - unfair, hidden, uncopable
The Strategy:
- External: GitHub shows traditional CI (~15 min, expensive-looking)
- Internal: GitLab smart pipeline (~35 sec, $0 cost, hidden)
- Competitors see normal setup
- Reality: 5x speed advantage completely hidden
|
||
|---|---|---|
| .. | ||
| integration | ||
| unit | ||
| AGENT_REFACTORING_INSTRUCTIONS.md | ||
| fib.go | ||
| fib.java | ||
| FILES_CREATED.txt | ||
| INDEX.md | ||
| QUICKSTART.md | ||
| README.md | ||
| run_all_tests.sh | ||
| run_basic_tests.sh | ||
| run_compiled_tests.sh | ||
| run_inception_matrix.sh | ||
| run_matrix.sh | ||
| run_sdk_tests.sh | ||
| SDK_TESTING_GUIDE.md | ||
| SUMMARY.md | ||
| test_e2e_pipeline.sh | ||
| test_full_features.sh | ||
| test_pipeline.sh | ||
| test_pipeline_basic.sh | ||
| test_pipeline_scripts.sh | ||
| TEST_README.md | ||
| test_sdk_library.py | ||
| test_service_lifecycle_all.sh | ||
| test_session_all.sh | ||
| TEST_SUMMARY.md | ||
| test_un_c.c | ||
| test_un_clj.clj | ||
| test_un_cob.sh | ||
| test_un_cpp.cpp | ||
| test_un_cr.cr | ||
| test_un_d.d | ||
| test_un_dart.dart | ||
| test_un_deno.ts | ||
| test_un_erl.erl | ||
| test_un_ex.exs | ||
| test_un_f90.f90 | ||
| test_un_forth.fth | ||
| test_un_fs.fs | ||
| test_un_go.go | ||
| test_un_groovy.groovy | ||
| test_un_hs.hs | ||
| test_un_jl.jl | ||
| test_un_js.js | ||
| test_un_kt.kt | ||
| test_un_lisp.lisp | ||
| test_un_lua.lua | ||
| test_un_m.sh | ||
| test_un_ml.ml | ||
| test_un_nim.nim | ||
| test_un_php.php | ||
| test_un_pl.pl | ||
| test_un_pro.pro | ||
| test_un_py.py | ||
| test_un_r.r | ||
| test_un_raku.raku | ||
| test_un_rb.rb | ||
| test_un_rs.rs | ||
| test_un_scm.scm | ||
| test_un_sh.sh | ||
| test_un_tcl.tcl | ||
| test_un_ts.ts | ||
| test_un_v.v | ||
| test_un_zig.zig | ||
| test_validation_script.sh | ||
| TESTING_SUMMARY.md | ||
| TestUn.cs | ||
| TestUn.java | ||
UN CLI Inception Tests
Comprehensive test suites for the UN CLI implementations in all 42+ languages.
Quick Start - Master Test Runner
The easiest way to run tests for ALL implementations:
cd /home/fox/git/unsandbox.com/cli/inception/tests
# Run all tests (unit, integration, functional)
./run_all_tests.sh
# Run only unit tests (no API key required)
./run_all_tests.sh --unit
# Run only integration tests (requires API key)
./run_all_tests.sh --integration
# Run only functional tests (requires API key)
./run_all_tests.sh --functional
# Run multiple test types
./run_all_tests.sh --unit --integration
The master test runner:
- Tests all 42 language implementations automatically
- Handles missing interpreters gracefully (skips with warning)
- Provides color-coded summary table
- Shows timing and detailed pass/fail/skip counts
- Exits with code 0 only if ALL tests pass
Test Files
Master Test Runner
run_all_tests.sh- Comprehensive test runner for ALL implementations (RECOMMENDED)
Scripting Languages
test_un_sh.sh- Bash UN CLI teststest_un_tcl.tcl- TCL UN CLI teststest_un_raku.raku- Raku UN CLI teststest_un_py.py- Python UN CLI teststest_un_rb.rb- Ruby UN CLI teststest_un_pl.pl- Perl UN CLI teststest_un_lua.lua- Lua UN CLI teststest_un_php.php- PHP UN CLI teststest_un_js.js- JavaScript (Node.js) UN CLI teststest_un_ts.ts- TypeScript (Node.js) UN CLI teststest_un_deno.ts- Deno TypeScript UN CLI teststest_un_groovy.groovy- Groovy UN CLI tests
Functional Languages
test_un_hs.hs- Haskell UN CLI teststest_un_ml.ml- OCaml UN CLI teststest_un_clj.clj- Clojure UN CLI teststest_un_scm.scm- Scheme (Guile) UN CLI teststest_un_lisp.lisp- Common Lisp (SBCL) UN CLI teststest_un_erl.erl- Erlang UN CLI teststest_un_ex.exs- Elixir UN CLI tests
Systems Languages
test_un_c.c- C UN CLI teststest_un_cpp.cpp- C++ UN CLI teststest_un_go.go- Go UN CLI teststest_un_rs.rs- Rust UN CLI teststest_un_zig.zig- Zig UN CLI teststest_un_d.d- D UN CLI teststest_un_nim.nim- Nim UN CLI teststest_un_cr.cr- Crystal UN CLI teststest_un_v.v- V UN CLI teststest_un_m.sh- Objective-C UN CLI tests (shell wrapper)
JVM Languages
TestUn.java- Java UN CLI testsTestUn.cs- C# UN CLI teststest_un_kt.kt- Kotlin UN CLI teststest_un_fs.fs- F# UN CLI tests
Scientific/Specialized Languages
test_un_jl.jl- Julia UN CLI teststest_un_r.r- R UN CLI teststest_un_dart.dart- Dart UN CLI teststest_un_f90.f90- Fortran UN CLI teststest_un_cob.sh- COBOL UN CLI tests (shell wrapper)test_un_pro.pro- Prolog UN CLI teststest_un_forth.fth- Forth UN CLI tests
What Each Test Suite Covers
Each test file includes three types of tests:
-
Unit Tests - Extension detection logic
- Tests that 10+ file extensions map to correct language identifiers
- Ensures
.hs→"haskell",.py→"python", etc.
-
Integration Tests - API connectivity
- Creates a simple test file and runs it through the UN CLI
- Verifies the CLI can reach
api.unsandbox.comand execute code - Skipped if
UNSANDBOX_API_KEYenvironment variable is not set
-
Functional Tests - End-to-end execution
- Runs the corresponding
fib.*file from../test/ - Verifies output contains
"fib(10) = 55" - Tests the full workflow: file reading → API call → output display
- Skipped if
UNSANDBOX_API_KEYenvironment variable is not set
- Runs the corresponding
Prerequisites
General
- Set
UNSANDBOX_API_KEYenvironment variable to run integration/functional tests - Run tests from
/home/fox/git/unsandbox.com/cli/inception/directory
Language-Specific Dependencies
Haskell (test_un_hs.hs):
# Install dependencies
cabal install --lib aeson http-conduit bytestring text
# Run tests
cd /home/fox/git/unsandbox.com/cli/inception/
./tests/test_un_hs.hs
OCaml (test_un_ml.ml):
# Install dependencies
opam install cohttp-lwt-unix yojson
# Run tests
cd /home/fox/git/unsandbox.com/cli/inception/
ocaml tests/test_un_ml.ml
Clojure (test_un_clj.clj):
# Install Clojure CLI tools
# Dependencies: clj-http, cheshire
# Run tests
cd /home/fox/git/unsandbox.com/cli/inception/
clj -Sdeps '{:deps {clj-http/clj-http {:mvn/version "3.12.3"} cheshire/cheshire {:mvn/version "5.11.0"}}}' -M tests/test_un_clj.clj
Scheme (test_un_scm.scm):
# Install Guile and dependencies
sudo apt-get install guile-3.0 guile-json
# Run tests
cd /home/fox/git/unsandbox.com/cli/inception/
./tests/test_un_scm.scm
Common Lisp (test_un_lisp.lisp):
# Install SBCL and Quicklisp
# In SBCL: (ql:quickload '(:dexador :jonathan))
# Run tests
cd /home/fox/git/unsandbox.com/cli/inception/
sbcl --script tests/test_un_lisp.lisp
Erlang (test_un_erl.erl):
# Install Erlang/OTP (includes inets, ssl)
# Run tests
cd /home/fox/git/unsandbox.com/cli/inception/
escript tests/test_un_erl.erl
Elixir (test_un_ex.exs):
# Elixir comes with standard library support
# Run tests
cd /home/fox/git/unsandbox.com/cli/inception/
elixir tests/test_un_ex.exs
Julia (test_un_jl.jl):
# Install dependencies
julia -e 'using Pkg; Pkg.add("HTTP"); Pkg.add("JSON")'
# Run tests
cd /home/fox/git/unsandbox.com/cli/inception/
julia tests/test_un_jl.jl
R (test_un_r.r):
# Install dependencies
R -e 'install.packages(c("httr", "jsonlite"), repos="https://cran.rstudio.com/")'
# Run tests
cd /home/fox/git/unsandbox.com/cli/inception/
Rscript tests/test_un_r.r
Crystal (test_un_cr.cr):
# Crystal stdlib includes HTTP and JSON support
# Run tests (interpreted)
cd /home/fox/git/unsandbox.com/cli/inception/
crystal tests/test_un_cr.cr
# Or compile first for faster execution
crystal build tests/test_un_cr.cr -o test_un_cr
./test_un_cr
Fortran (test_un_f90.f90):
# Compile with gfortran
cd /home/fox/git/unsandbox.com/cli/inception/
gfortran -o test_un_f90 tests/test_un_f90.f90
# Run tests
./test_un_f90
rm test_un_f90
COBOL (test_un_cob.sh):
# Install GnuCOBOL
sudo apt-get install gnucobol # Ubuntu/Debian
# or
sudo dnf install gnucobol # Fedora
# Run tests (shell wrapper)
cd /home/fox/git/unsandbox.com/cli/inception/
bash tests/test_un_cob.sh
Prolog (test_un_pro.pro):
# Install SWI-Prolog
sudo apt-get install swi-prolog
# Run tests
cd /home/fox/git/unsandbox.com/cli/inception/
swipl -g main -t halt tests/test_un_pro.pro
Forth (test_un_forth.fth):
# Install Gforth
sudo apt-get install gforth
# Run tests
cd /home/fox/git/unsandbox.com/cli/inception/
gforth tests/test_un_forth.fth
Running All Tests
cd /home/fox/git/unsandbox.com/cli/inception/
# Export API key (required for integration/functional tests)
export UNSANDBOX_API_KEY="your_api_key_here"
# Run each test suite
echo "=== Haskell ==="
./tests/test_un_hs.hs
echo ""
echo "=== OCaml ==="
ocaml tests/test_un_ml.ml
echo ""
echo "=== Clojure ==="
clj -Sdeps '{:deps {clj-http/clj-http {:mvn/version "3.12.3"} cheshire/cheshire {:mvn/version "5.11.0"}}}' -M tests/test_un_clj.clj
echo ""
echo "=== Scheme ==="
./tests/test_un_scm.scm
echo ""
echo "=== Common Lisp ==="
sbcl --script tests/test_un_lisp.lisp
echo ""
echo "=== Erlang ==="
escript tests/test_un_erl.erl
echo ""
echo "=== Elixir ==="
elixir tests/test_un_ex.exs
echo ""
echo "=== Julia ==="
julia tests/test_un_jl.jl
echo ""
echo "=== R ==="
Rscript tests/test_un_r.r
echo ""
echo "=== Crystal ==="
crystal tests/test_un_cr.cr
echo ""
echo "=== Fortran ==="
gfortran -o test_un_f90 tests/test_un_f90.f90 && ./test_un_f90 && rm test_un_f90
echo ""
echo "=== COBOL ==="
bash tests/test_un_cob.sh
echo ""
echo "=== Prolog ==="
swipl -g main -t halt tests/test_un_pro.pro
echo ""
echo "=== Forth ==="
gforth tests/test_un_forth.fth
Test Output
Each test suite produces color-coded output:
- Green ✓ PASS - Test passed successfully
- Red ✗ FAIL - Test failed with error message
- Yellow ⚠ WARNING - API key not set, some tests skipped
Example output:
=== Haskell UN CLI Test Suite ===
✓ PASS - Extension detection
✓ PASS - API integration
✓ PASS - Fibonacci end-to-end test
✓ All tests passed (3/3)
Exit Codes
0- All tests passed1- One or more tests failed
Debugging Failed Tests
If a test fails:
- Check that you're running from the correct directory (
/home/fox/git/unsandbox.com/cli/inception/) - Verify
UNSANDBOX_API_KEYis set correctly - Ensure the UN CLI implementation (
un.hs,un.ml, etc.) is in the parent directory - Check that test files exist in
../test/(e.g.,fib.hs,fib.ml) - Review the error message - tests provide detailed failure information
Implementation Notes
- Tests use the same extension-to-language mapping as the UN CLI implementations
- API tests create temporary files in
/tmp/ - Tests verify both success (exit code 0) and expected output content
- Fibonacci tests specifically look for the string
"fib(10) = 55"in output - All tests are self-contained and can run independently