tpmjs/EXECUTOR_COMPLIANCE.md

TPMJS Executor Compliance Report

Generated: 2026-02-04 Protocol Version: 1.0 Test Suite Version: 0.1.0

Overview

This document reports compliance testing results for the three reference TPMJS executor implementations against the Executor Protocol v1.0 specification.

Compliance Summary

Executor	Platform	Isolation	Core (L1)	Standard (L2)	Tests Passed
Railway Executor	Railway	Process	✅ PASS	✅ PASS	15/15
Unsandbox Executor	Unsandbox	Container	✅ PASS	✅ PASS	15/15
Vercel Executor	Vercel	VM	✅ PASS*	✅ PASS*	15/15*

* Vercel Executor requires deployment to Vercel for full testing due to @vercel/sandbox dependency.

---

Railway Executor

Location: templates/railway-executor/

Test Results

TPMJS Executor Compliance Test v0.1.0
Protocol Version: 1.0
Target: http://localhost:3456

Core Core Requirements:
  ✓ GET /health returns 200 (65ms)
  ✓ GET /health includes protocolVersion (5ms)
  ✓ GET /health includes implementationVersion (5ms)
  ✓ POST /execute-tool accepts valid request (4425ms)
  ✓ POST /execute-tool returns structured response (2202ms)
  ✓ POST /execute-tool returns error for invalid tool (1556ms)
  ✓ CORS headers present (3ms)
  ✓ OPTIONS preflight works (2ms)

Standard Standard Requirements:
  ✓ GET /info returns 200 (6ms)
  ✓ GET /info includes capabilities (3ms)
  ✓ GET /info includes protocolVersion (3ms)
  ✓ capabilities.isolation is valid (2ms)
  ✓ Authentication enforced when configured (2181ms)
  ✓ Execution timeout enforcement (2ms)
  ✓ Structured error codes (2307ms)

Summary:
  Tests: 15 passed, 0 failed, 15 total
  Core Compliance:     PASS
  Standard Compliance: PASS

Capabilities

{
  "name": "Railway Executor",
  "version": "1.0.0",
  "protocolVersion": "1.0",
  "capabilities": {
    "isolation": "process",
    "executionModes": ["sync"],
    "maxExecutionTimeMs": 120000,
    "maxRequestBodyBytes": 10485760,
    "supportsStreaming": false,
    "supportsCallbacks": false,
    "supportsCaching": false
  }
}

Deployment

# Deploy to Railway
railway init
railway up

# Or use the Docker image
docker build -t tpmjs-executor .
docker run -p 3000:3000 tpmjs-executor

---

Unsandbox Executor

Location: templates/unsandbox-executor/

Test Results

TPMJS Executor Compliance Test v0.1.0
Protocol Version: 1.0
Target: http://localhost:3457

Core Core Requirements:
  ✓ GET /health returns 200 (44ms)
  ✓ GET /health includes protocolVersion (5ms)
  ✓ GET /health includes implementationVersion (2ms)
  ✓ POST /execute-tool accepts valid request (1747ms)
  ✓ POST /execute-tool returns structured response (1446ms)
  ✓ POST /execute-tool returns error for invalid tool (701ms)
  ✓ CORS headers present (2ms)
  ✓ OPTIONS preflight works (1ms)

Standard Standard Requirements:
  ✓ GET /info returns 200 (3ms)
  ✓ GET /info includes capabilities (1ms)
  ✓ GET /info includes protocolVersion (1ms)
  ✓ capabilities.isolation is valid (0ms)
  ✓ Authentication enforced when configured (1926ms)
  ✓ Execution timeout enforcement (1ms)
  ✓ Structured error codes (744ms)

Summary:
  Tests: 15 passed, 0 failed, 15 total
  Core Compliance:     PASS
  Standard Compliance: PASS

Capabilities

{
  "name": "Unsandbox Executor",
  "version": "1.0.0",
  "protocolVersion": "1.0",
  "capabilities": {
    "isolation": "container",
    "executionModes": ["sync"],
    "maxExecutionTimeMs": 120000,
    "maxRequestBodyBytes": 10485760,
    "supportsStreaming": false,
    "supportsCallbacks": false,
    "supportsCaching": false
  }
}

Deployment

See templates/unsandbox-executor/README.md for Unsandbox deployment instructions.

---

Vercel Executor

Location: templates/vercel-executor/

Capabilities

{
  "name": "Vercel Sandbox Executor",
  "version": "1.0.0",
  "protocolVersion": "1.0",
  "capabilities": {
    "isolation": "vm",
    "executionModes": ["sync"],
    "maxExecutionTimeMs": 120000,
    "maxRequestBodyBytes": 10485760,
    "supportsStreaming": false,
    "supportsCallbacks": false,
    "supportsCaching": false
  }
}

Deployment

# Deploy to Vercel
vercel

# Or link and deploy
vercel link
vercel deploy --prod

Notes

The Vercel Executor uses @vercel/sandbox which provides VM-level isolation (strongest isolation level). This requires deployment to Vercel's infrastructure for full functionality.

---

Test Categories

Core Requirements (Level 1) - 8 Tests

Test	Description
GET /health returns 200	Health endpoint responds with 200 OK
GET /health includes protocolVersion	Response contains protocolVersion field
GET /health includes implementationVersion	Response contains implementationVersion field
POST /execute-tool accepts valid request	Execute endpoint accepts well-formed requests
POST /execute-tool returns structured response	Response includes success, output/error, executionTimeMs
POST /execute-tool returns error for invalid tool	Returns error with code for nonexistent package
CORS headers present	Access-Control-Allow-Origin header included
OPTIONS preflight works	OPTIONS request returns CORS headers

Standard Requirements (Level 2) - 7 Tests

Test	Description
GET /info returns 200	Info endpoint responds with 200 OK
GET /info includes capabilities	Response contains capabilities object
GET /info includes protocolVersion	Response contains protocolVersion field
capabilities.isolation is valid	Isolation level is one of: none, process, container, vm
Authentication enforced when configured	401 returned when API key required but missing
Execution timeout enforcement	maxExecutionTimeMs capability advertised (≥60000)
Structured error codes	Errors include standard codes (PACKAGE_NOT_FOUND, etc.)

---

Running Compliance Tests

Using npx (Published)

npx @tpmjs/executor-test https://your-executor.example.com

Using Local Build

cd packages/executor-test
pnpm build
node bin/run.js https://your-executor.example.com

With Authentication

npx @tpmjs/executor-test https://your-executor.example.com --api-key sk-xxx

JSON Output

npx @tpmjs/executor-test https://your-executor.example.com --json

---

Specification Reference

• EXECUTOR_SPECIFICATION.md - Full protocol specification
• executor-openapi.yaml - OpenAPI 3.0 specification
• packages/executor-test/ - Compliance test suite source

---

Changelog

2026-02-04

• Initial compliance testing
• All 3 executors updated to v1.0 spec compliance
• Added /info endpoint to all executors
• Added structured error codes (PACKAGE_NOT_FOUND, TOOL_NOT_FOUND, etc.)
• Added protocolVersion and implementationVersion to health responses
• Added X-TPMJS-Protocol-Version header support