tpmjs/IMPLEMENTATION_STATUS.md

Dynamic Tool Loading - Implementation Status

✅ Completed

1. Search Tool Package (@tpmjs/search-registry)

• ✅ Created package with AI SDK v6 JSON Schema format
• ✅ Connects to search API endpoint
• ✅ Returns tool metadata (packageName, exportName, version, importUrl)
• ✅ Fixed schema format (was using Zod, now uses jsonSchema)
• ✅ Location: packages/tools/search-registry/

2. Search API Endpoint (/api/tools/search)

• ✅ Implemented simple text-based search (BM25 had dependency issues)
• ✅ Searches by keywords in description, package name, keywords
• ✅ Returns tools with import URLs for esm.sh
• ✅ Location: apps/web/src/app/api/tools/search/route.ts

3. Pre-flight Tool Loading in Playground

• ✅ Automatic search on every user message
• ✅ Extracts user query from last message
• ✅ Calls searchTpmjsTools automatically
• ✅ Attempts to load discovered tools dynamically
• ✅ Location: apps/playground/src/app/api/chat/route.ts

4. Dynamic Tool Loader (Railway Service Approach)

• ✅ Updated to call Railway service instead of local imports
• ✅ Calls /load-and-describe endpoint to get tool schema
• ✅ Wraps tool with remote execution via /execute-tool endpoint
• ✅ Caches tool wrappers locally
• ✅ Location: apps/playground/src/lib/dynamic-tool-loader.ts

5. Documentation

• ✅ DYNAMIC_IMPORT_ISSUE.md - Comprehensive problem analysis
• ✅ RAILWAY_DYNAMIC_TOOL_LOADER.md - Railway implementation guide
• ✅ This file - Implementation status

🚧 Pending (Railway Service Implementation)

Railway Service Endpoints Needed

You need to add these two endpoints to your existing Railway service:

1. POST /load-and-describe

Purpose: Load a tool from esm.sh and return its schema

Request:

{
  "packageName": "firecrawl-aisdk",
  "exportName": "webSearchTool",
  "version": "0.7.2",
  "importUrl": "https://esm.sh/firecrawl-aisdk@0.7.2"
}

Response:

{
  "success": true,
  "tool": {
    "exportName": "webSearchTool",
    "description": "Search the web using Firecrawl",
    "inputSchema": {
      "type": "object",
      "properties": {
        "query": { "type": "string" }
      }
    }
  }
}

Implementation Reference: See RAILWAY_DYNAMIC_TOOL_LOADER.md for full code

2. POST /execute-tool

Purpose: Execute a dynamically loaded tool with parameters

Request:

{
  "packageName": "firecrawl-aisdk",
  "exportName": "webSearchTool",
  "version": "0.7.2",
  "importUrl": "https://esm.sh/firecrawl-aisdk@0.7.2",
  "params": {
    "query": "latest AI news"
  }
}

Response:

{
  "success": true,
  "output": { "results": [...] },
  "executionTimeMs": 1234
}

Implementation Reference: See RAILWAY_DYNAMIC_TOOL_LOADER.md for full code

Deployment Requirements

1. Railway Service:

   • Must run with --experimental-network-imports flag
   • Add to start command: node --experimental-network-imports server.js

2. Environment Variables (Vercel):

   RAILWAY_SERVICE_URL=https://your-railway-service.up.railway.app
   # or reuse existing:
   SANDBOX_EXECUTOR_URL=https://your-railway-service.up.railway.app
   

3. Local Testing (Railway service on port 3001):

   RAILWAY_SERVICE_URL=http://localhost:3001
   

🎯 Testing Checklist

Once Railway endpoints are deployed:

• Test /load-and-describe endpoint directly with curl
• Test /execute-tool endpoint directly with curl
• Test full flow in playground:
  • Ask: "search the web for latest AI news"
  • Verify pre-flight search finds tools
  • Verify tools load via Railway
  • Verify tool execution works
  • Check console logs for debugging info

📊 Current Flow

User: "search the web for latest AI news"
                ↓
    Chat API extracts query
                ↓
    Automatically calls searchTpmjsTools
                ↓
    Search API returns matching tools
     (packageName, exportName, version)
                ↓
    loadToolsBatch() called for each tool
                ↓
    For each tool:
      1. Check local cache
      2. If not cached:
         → POST to Railway: /load-and-describe
         ← Get back: description + inputSchema
      3. Create wrapper tool with:
         - description from Railway
         - inputSchema from Railway
         - execute() → calls Railway /execute-tool
      4. Cache wrapper locally
                ↓
    All tools available to agent
                ↓
    Agent calls tool (wrapper)
                ↓
    Wrapper → POST to Railway: /execute-tool
                ↓
    Railway imports from esm.sh and executes
                ↓
    Result returned to agent
                ↓
    Agent uses result to answer user

🔍 Debugging

Check console logs for:

• 📦 Loading from Railway - Tool loading initiated
• ✅ Tool loaded from Railway - Tool schema received
• 🚀 Executing ... remotely - Tool execution initiated
• ✅ Tool executed successfully - Tool execution complete
• ❌ Railway service error - Connection failed
• ❌ Failed to load tool - Import failed

📁 Files Modified

1. packages/tools/search-registry/src/index.ts - Search tool
2. packages/tools/search-registry/package.json - AI SDK version
3. apps/web/src/app/api/tools/search/route.ts - Search endpoint
4. apps/playground/src/app/api/chat/route.ts - Pre-flight search
5. apps/playground/src/lib/dynamic-tool-loader.ts - Railway integration
6. apps/playground/next.config.ts - Added urlImports (unused)
7. apps/playground/src/lib/tool-loader.ts - Removed firecrawl

🚀 Next Steps

1. Deploy Railway endpoints using code from RAILWAY_DYNAMIC_TOOL_LOADER.md
2. Set environment variables in Vercel
3. Test locally with Railway service running on localhost:3001
4. Deploy to production and test with real tools
5. Monitor logs for any issues

💡 Key Insights

• Next.js Limitation: Cannot do dynamic HTTP imports due to bundler
• Railway Solution: Plain Node.js with --experimental-network-imports
• Caching Strategy: Two-level cache (local wrapper + Railway module)
• Execution Model: Remote execution in Railway, not Next.js
• Security: Tools execute in Railway sandbox, not Vercel
• Performance: First load ~1-2s (import), cached loads <10ms

📚 Related Documentation

• DYNAMIC_IMPORT_ISSUE.md - Problem analysis and ChatGPT response
• RAILWAY_DYNAMIC_TOOL_LOADER.md - Full Railway implementation guide
• Plan file: ~/.claude/plans/jiggly-inventing-dragon.md

---

Status: Ready for Railway deployment Blocker: Railway /load-and-describe and /execute-tool endpoints need implementation ETA: 30-60 minutes to implement Railway endpoints + test