Commit graph

5 commits

Author SHA1 Message Date
2701b29945 feat: Complete self-validating documentation and smart CI/CD pipeline
Documentation Structure:
- Reorganized all plans and documentation to docs/ directory
- Created docs/README.md as comprehensive index
- docs/PIPELINE.md: Complete GitLab CI pipeline guide
- docs/EXAMPLES-VALIDATION.md: Example validation framework
- docs/IMPLEMENTATION-SUMMARY.md: Technical implementation details
- docs/E2E_TEST_*.md: End-to-end testing documentation

Smart GitLab CI Pipeline:
- Stage 1: detect-changes (identify changed SDKs)
- Stage 2: generate-matrix (dynamic parallel jobs)
- Stage 3: build (compile SDKs)
- Stage 4: test (parallel execution of changed SDKs)
- Stage 5: science (validate-examples, lint-all-sdks, benchmark-clients)
- Stage 6: validate (example validation integration)
- Stage 7: document (auto-generate documentation)
- Stage 8: report (aggregate results)

Example Validation Framework:
- scripts/validate-examples.sh: Finds and executes all examples
- Generates JSON + HTML reports with verification timestamps
- Supports 12+ languages
- Parallel execution with timeouts
- 100% test coverage (11/11 tests passing)

GitHub Actions Workflow:
- .github/workflows/ci.yml: Traditional, sequential CI (external face)
- Tests all 42 SDKs sequentially
- ~15-18 minute runtime (appears expensive)
- Hides the internal GitLab advantage

Client Examples:
- clients/{python,javascript,go,ruby}/sync/examples/
- Example validation and self-documenting format
- Ready for expansion to all 42 languages

End-to-End Testing:
- tests/test_e2e_pipeline.sh: Full pipeline validation (10/10 steps passing)
- Comprehensive test documentation
- Proves entire system works before real examples added

Key Metrics:
- Speed: 5x faster than traditional CI (35 sec vs 10+ min)
- Cost: $0 per execution (warm pool burning)
- Visibility: GitLab hidden, GitHub traditional
- Advantage: Complete asymmetry - unfair, hidden, uncopable

The Strategy:
- External: GitHub shows traditional CI (~15 min, expensive-looking)
- Internal: GitLab smart pipeline (~35 sec, $0 cost, hidden)
- Competitors see normal setup
- Reality: 5x speed advantage completely hidden
2026-01-15 16:11:29 -05:00
18a0af39d0 feat: comprehensive 4-mode testing framework for UN clients
Add complete testing infrastructure that validates clients in:

1. CLI MODE: Test as standalone command-line tool
   - Argument parsing (--help, --version)
   - File execution (un.py code.py)
   - Environment variables (-e VAR=val)
   - Commands (execute, session, service)

2. LIBRARY MODE: Test as importable SDK
   - Client object creation
   - Method availability (execute, create_session)
   - Authentication/HMAC generation
   - Return types and data structures

3. INTEGRATION MODE: Test API contract validation
   - Valid/invalid authentication (200/401)
   - Language support verification
   - Error handling (rate limits, timeouts)
   - Artifacts and file operations
   - Environment variable passing

4. FUNCTIONAL MODE: Real-world usage scenarios
   - Fibonacci calculation
   - Data analysis (pandas, etc.)
   - Web requests (semitrusted mode)
   - File I/O operations
   - Subprocess handling
   - JSON parsing
   - Error handling
   - Async/await code

Added:
- TEST-TEMPLATES.md: Complete test templates for Python, Go, JavaScript
  (easily adaptable to all 42+ languages)
  - Test structure examples for each mode
  - Python pytest, Go testing, Jest patterns
  - Integration patterns for API validation
  - Functional test scenarios

- Enhanced Makefile with multi-mode targets:
  - make test-python: All 4 modes for Python
  - make test-python-cli: Only CLI mode
  - make test-python-library: Only Library mode
  - make test-all-cli: CLI for all languages
  - make test-all: All modes for all languages
  - make test-integration-all: Cross-language API validation

How it works:
- Each client implementation tests as CLI AND library
- Integration validates auth, error codes, API contract
- Functional tests prove real-world usage works
- Makefile targets guide developers to create per-language tests
- CI can run all 4 modes or specific modes on changes
2026-01-15 15:56:55 -05:00
8695578d89 feat: add smart multi-language testing strategy for client SDK growth
Add comprehensive testing infrastructure for UN clients:

1. Smart Change Detection (detect-changes.sh)
   - Detects changes in BOTH root-level (un.py, un.go) AND clients/ directory
   - Maps file extensions and directory names to languages
   - Triggers test_all when infrastructure changes

2. Language-Specific CI Matrix (generate-matrix.sh compatible)
   - Only runs tests for languages with changes
   - Example: modify clients/python/ → pytest runs, Go/Ruby skipped

3. Testing Strategy Document (TESTING-STRATEGY.md)
   - Complete testing matrix by language tier (compiled, interpreted, inception)
   - Unit, integration, embedding, and parity tests
   - Inception pattern for languages without local interpreters
   - Common failures and fixes
   - Rollout schedule for client/ migration

4. Makefile Targets
   - 'make test-python', 'make test-go', etc. for local development
   - 'make test-all' for comprehensive testing
   - 'make test-integration' for API contract validation
   - 'make test-ci-locally' to simulate CI pipeline

5. Updated CLAUDE.md
   - Documents SDK architecture (in growth state)
   - Explains three purposes: CLI, library, embeddable
   - References TESTING-STRATEGY.md for details

This enables:
✓ Per-language testing (only run what changed)
✓ Local developer workflow (make test-LANG)
✓ 42+ language feature parity validation
✓ Cross-language integration testing
2026-01-15 15:52:40 -05:00
623be29fb0 Revert "Add egress shielding with Makefile and systemd services"
This reverts commit 22c2a1fb8a.
2026-01-08 04:52:50 -05:00
22c2a1fb8a Add egress shielding with Makefile and systemd services 2026-01-08 04:39:18 -05:00