un-inception/clients/python/ASYNC_vs_SYNC.md
russell@unturf.com 75f687f12f feat: Complete Python and C SDK implementations with examples and pipeline integration
Python Sync SDK (clients/python/sync/):
- 712 lines core implementation with 13 public APIs
- HMAC-SHA256 authentication with OpenSSL
- 4-tier credential system (args > env > ~/.unsandbox > ./accounts.csv)
- Language caching with 1-hour TTL
- 64+ comprehensive unit tests
- Full documentation (README, USAGE, IMPLEMENTATION)

Python Async SDK (clients/python/async/):
- 705 lines async implementation using aiohttp
- Full async/await pattern support
- Exponential backoff polling strategy
- 200+ test cases with ~95% coverage
- 7 working async examples
- 5 documentation guides

C SDK (clients/c/):
- 823 lines C implementation
- Header file with 15 public functions
- OpenSSL HMAC-SHA256 + libcurl HTTP client
- Language detection for 48 file extensions
- 22/22 tests passing
- Proper memory management

Examples:
- 14 Python examples (7 sync, 7 async) with docstrings
- 4 C examples (hello_world, fibonacci, error_handling, credentials)
- All examples ready for pipeline validation
- Expected outputs documented for validation

Pipeline Integration:
- Updated .gitlab-ci.yml with gcc/musl-dev for C compilation
- Enhanced validate-examples.sh with C compilation support
- Updated detect-changes.sh to recognize python/c changes
- Updated generate-matrix.sh with python/c in matrix
- E2E tests updated with mock Python/C examples
- All tests passing (30+ test cases)

Documentation:
- PYTHON_C_INTEGRATION_SUMMARY.md (452 lines)
- Complete API references for both SDKs
- Quick start guides
- Pattern documentation
- Error handling guides
2026-01-15 16:42:58 -05:00

421 lines
9.4 KiB
Markdown

# Async vs Sync Python SDK - Comparison Guide
Both sync and async SDKs provide the same functionality. Choose based on your use case.
## Quick Comparison
| Feature | Sync SDK | Async SDK |
|---------|----------|-----------|
| **HTTP Library** | `requests` | `aiohttp` |
| **I/O Model** | Blocking (threads) | Non-blocking (async/await) |
| **Concurrency** | Thread pools | Event loop |
| **Best For** | Scripts, simple apps | High-concurrency servers, async frameworks |
| **Import** | `from un import ...` | `from un_async import ...` |
| **Execution** | Direct calls | `await` calls in async context |
| **Location** | `clients/python/sync/` | `clients/python/async/` |
## Side-by-Side Examples
### Simple Execution
**Sync SDK:**
```python
from un import execute_code
result = execute_code("python", "print('hello')")
print(result["stdout"])
```
**Async SDK:**
```python
import asyncio
from un_async import execute_code
async def main():
result = await execute_code("python", "print('hello')")
print(result["stdout"])
asyncio.run(main())
```
### Concurrent Execution
**Sync SDK (using ThreadPoolExecutor):**
```python
from un import execute_code
from concurrent.futures import ThreadPoolExecutor
def execute_one(code):
return execute_code("python", code)
with ThreadPoolExecutor(max_workers=5) as executor:
results = list(executor.map(execute_one, [
"print(1)",
"print(2)",
"print(3)",
]))
for result in results:
print(result["stdout"])
```
**Async SDK (using asyncio):**
```python
import asyncio
from un_async import execute_code
async def main():
results = await asyncio.gather(
execute_code("python", "print(1)"),
execute_code("python", "print(2)"),
execute_code("python", "print(3)"),
)
for result in results:
print(result["stdout"])
asyncio.run(main())
```
### Fire-and-Forget Job
**Sync SDK:**
```python
from un import execute_async, wait_for_job
job_id = execute_async("python", "print('started')")
# Do other work...
result = wait_for_job(job_id)
print(result["stdout"])
```
**Async SDK:**
```python
import asyncio
from un_async import execute_async, wait_for_job
async def main():
job_id = await execute_async("python", "print('started')")
# Do other work...
result = await wait_for_job(job_id)
print(result["stdout"])
asyncio.run(main())
```
### Error Handling
**Sync SDK:**
```python
from un import execute_code, CredentialsError
import requests
try:
result = execute_code("python", "print('hello')")
except CredentialsError as e:
print(f"Auth error: {e}")
except requests.RequestException as e:
print(f"Network error: {e}")
```
**Async SDK:**
```python
import asyncio
from un_async import execute_code, CredentialsError
import aiohttp
async def main():
try:
result = await execute_code("python", "print('hello')")
except CredentialsError as e:
print(f"Auth error: {e}")
except aiohttp.ClientError as e:
print(f"Network error: {e}")
asyncio.run(main())
```
## When to Use Each
### Use Sync SDK When:
1. **Writing Simple Scripts**
```python
# Simple one-off scripts work great with sync
from un import execute_code
result = execute_code("python", "print('done')")
```
2. **Working in Jupyter Notebooks**
```python
from un import execute_code
result = execute_code("python", code_cell)
print(result["stdout"])
```
3. **Building Command-Line Tools**
```python
#!/usr/bin/env python3
from un import execute_code
# CLI logic using sync SDK
```
4. **Prototyping**
- Easier to understand and debug
- No async/await syntax required
### Use Async SDK When:
1. **Building High-Concurrency Servers**
```python
# FastAPI app with async SDK
from fastapi import FastAPI
from un_async import execute_code
@app.post("/execute")
async def run(request):
result = await execute_code("python", request.code)
return result
```
2. **Running Many Jobs Concurrently**
```python
# Execute 1000+ jobs efficiently
results = await asyncio.gather(*tasks)
```
3. **Long-Running Services**
- Better resource utilization
- No thread overhead
- Scales to thousands of concurrent operations
4. **Async Web Frameworks**
- FastAPI, Quart, aiohttp
- Django async views
- Any async/await codebase
## API Compatibility
Both SDKs have **identical APIs** - all functions exist in both versions:
```
✓ execute_code()
✓ execute_async()
✓ get_job()
✓ wait_for_job()
✓ cancel_job()
✓ list_jobs()
✓ get_languages()
✓ detect_language()
✓ session_snapshot()
✓ service_snapshot()
✓ list_snapshots()
✓ restore_snapshot()
✓ delete_snapshot()
```
The only difference is that async versions require `await`.
## Migration Guide
### From Sync to Async
1. **Add `async def` and `await` keywords:**
```python
# Before
result = execute_code("python", "print('hi')")
# After
result = await execute_code("python", "print('hi')")
```
2. **Wrap in async context:**
```python
# Before
result = execute_code("python", "print('hi')")
# After
import asyncio
async def main():
result = await execute_code("python", "print('hi')")
return result
result = asyncio.run(main())
```
3. **Replace concurrent.futures with asyncio:**
```python
# Before
from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor(max_workers=5) as executor:
results = executor.map(execute_code, codes)
# After
results = await asyncio.gather(
*[execute_code(lang, code) for lang, code in zip(langs, codes)]
)
```
### From Async to Sync
1. **Remove `async def` and `await` keywords:**
```python
# Before
async def main():
result = await execute_code("python", "print('hi')")
# After
result = execute_code("python", "print('hi')")
```
2. **Remove asyncio.run wrapper:**
```python
# Before
result = asyncio.run(main())
# After
result = execute_code("python", "print('hi')")
```
3. **Use ThreadPoolExecutor instead of asyncio:**
```python
# Before (async)
results = await asyncio.gather(*tasks)
# After (sync)
from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor() as executor:
results = list(executor.map(execute_code, codes))
```
## Performance Considerations
### Sync SDK
- **Throughput:** Good for 1-100 concurrent jobs
- **Resource Usage:** One thread per connection
- **Overhead:** Thread context switching
- **Best:** Small to medium workloads
### Async SDK
- **Throughput:** Good for 100-10,000+ concurrent jobs
- **Resource Usage:** Single-threaded event loop
- **Overhead:** Minimal (event loop overhead only)
- **Best:** Large workloads, high concurrency
### Benchmark Example
```python
import asyncio
import time
from concurrent.futures import ThreadPoolExecutor
# Sync version
def sync_benchmark():
from un import execute_code
start = time.time()
for i in range(100):
execute_code("python", "print(1)")
return time.time() - start
# Async version with concurrency
async def async_benchmark():
from un_async import execute_code
start = time.time()
tasks = [execute_code("python", "print(1)") for i in range(100)]
await asyncio.gather(*tasks)
return time.time() - start
# Async version sequential
async def async_sequential():
from un_async import execute_code
start = time.time()
for i in range(100):
await execute_code("python", "print(1)")
return time.time() - start
```
**Expected Results (100 jobs):**
- Sync sequential: ~50s (blocking)
- Async sequential: ~50s (same, just with await)
- Async concurrent: ~5-10s (10x faster due to concurrency)
## Shared Features
Both SDKs share:
1. **Credential System** (4-tier priority)
2. **HMAC-SHA256 Authentication**
3. **Language Detection**
4. **Language Caching**
5. **Error Classes** (CredentialsError, etc.)
6. **Response Format**
7. **Job Management**
8. **Snapshot Operations**
## Debugging
### Sync SDK
```python
import logging
logging.basicConfig(level=logging.DEBUG)
from un import execute_code
result = execute_code("python", "print('debug')")
```
### Async SDK
```python
import logging
import asyncio
logging.basicConfig(level=logging.DEBUG)
async def main():
from un_async import execute_code
result = await execute_code("python", "print('debug')")
asyncio.run(main())
```
## Choosing for Your Project
### Decision Tree
```
├─ Need to run many jobs concurrently?
│ ├─ Yes (100+) → Use ASYNC SDK
│ └─ No → Check next
├─ Building web service with async framework?
│ ├─ Yes (FastAPI, Quart, etc.) → Use ASYNC SDK
│ └─ No → Check next
├─ Building simple script or CLI?
│ ├─ Yes → Use SYNC SDK
│ └─ No → Check next
├─ Need to integrate into existing async codebase?
│ ├─ Yes → Use ASYNC SDK
│ └─ No → Use SYNC SDK
```
## Coexistence
You can use both SDKs in the same project:
```python
# In one module - sync operations
from un import execute_code
sync_result = execute_code("python", "print('sync')")
# In another module - async operations
from un_async import execute_code
async_result = await execute_code("python", "print('async')")
```
This allows gradual migration or hybrid approaches.
## Support
- **Sync SDK Docs:** `clients/python/sync/README.md`
- **Async SDK Docs:** `clients/python/async/README.md`
- **API Reference:** `clients/python/async/USAGE_GUIDE.md`
- **Examples:** See `examples/` in each folder