fix(executor): pass execution context with abortSignal to tool execute()

Some AI SDK tools (like @parallel-web/ai-sdk-tools) expect execute(params, context)
where context contains { abortSignal, messages, toolCallId }. Previously we only
passed params which caused 'Cannot destructure abortSignal' errors.

Also:
- Improved playground system prompt for better tool execution
- Playground /api/tools now proxies to web app with response transformation
- Added broken-tools.md documenting tool failure categories

Co-Authored-By: Claude <noreply@anthropic.com>

apps/playground/src/app/api/chat/route.ts

@@ -194,12 +194,35 @@ export async function POST(request: NextRequest) {
       })
       .join('\n');
 
-    const system = `You are a helpful AI assistant that can use TPMJS tools to help users.
+    const system = `You are an AI assistant with access to a dynamic tool registry containing thousands of tools. Your job is to EXECUTE tools to help users accomplish tasks.
 
-Available tools:
+## Tool Execution Rules
+
+1. **When a user asks you to "call", "use", "run", or "execute" a tool** - you MUST invoke that tool immediately. Do not just describe it or search for it.
+
+2. **When a user asks a question that could be answered by a tool** - invoke the appropriate tool to get real data, don't make up answers.
+
+3. **searchTpmjsTools is for DISCOVERY only** - use it when you need to find tools you don't have loaded yet. Once a tool is loaded (listed below), call it directly.
+
+4. **Tool names are sanitized** - if user says "extractTool from @parallel-web/ai-sdk-tools", look for a loaded tool like "parallel-web_ai-sdk-tools-extractTool".
+
+5. **Always execute, then explain** - after calling a tool, summarize the results for the user.
+
+## Currently Loaded Tools
 ${toolsList}
 
-When you use a tool, you MUST always follow up with a natural language answer to the user summarizing the result.`;
+## Examples
+
+User: "call extractTool on https://example.com"
+→ Invoke the extractTool with url parameter, then explain results
+
+User: "search for web scraping tools"
+→ Use searchTpmjsTools to find tools, then tell user what's available
+
+User: "what's the weather in Tokyo"
+→ Search for a weather tool, load it, then invoke it
+
+Remember: Your value is in EXECUTING tools to get real results, not just describing what tools could do.`;
 
     // 6. Stream response with all available tools
     const result = streamText({

apps/playground/src/app/api/tools/route.ts

@@ -1,4 +1,3 @@
-import { searchTpmjsToolsTool } from '@tpmjs/search-registry';
 import { NextResponse } from 'next/server';
 
 export const runtime = 'nodejs';
@@ -6,23 +5,38 @@ export const dynamic = 'force-dynamic';
 
 export async function GET() {
   try {
-    // Search for all tools (empty query returns all)
-    // biome-ignore lint/style/noNonNullAssertion: Tool created with tool() always has execute
-    const result = await searchTpmjsToolsTool.execute!(
-      {
-        query: '',
-        limit: 100,
-      },
-      {} as any
-    );
+    const baseUrl = process.env.TPMJS_API_URL || 'https://tpmjs.com';
+    const response = await fetch(`${baseUrl}/api/tools`);
 
-    // Type assertion: searchTpmjsToolsTool returns direct result, not AsyncIterable
-    const searchResult = result as { query: string; matchCount: number; tools: any[] };
+    if (!response.ok) {
+      throw new Error(`Failed to fetch tools: ${response.statusText}`);
+    }
 
+    const data = await response.json();
+
+    // Transform web app response format to playground format
+    // Web app returns { success, data: Tool[] }
+    // Playground expects { success, tools: Tool[], total }
+    const tools = data.data || [];
     return NextResponse.json({
-      success: true,
-      tools: searchResult.tools,
-      total: searchResult.matchCount,
+      success: data.success,
+      tools: tools.map((tool: any) => ({
+        toolId: tool.id,
+        packageName: tool.package?.npmPackageName,
+        exportName: tool.exportName,
+        description: tool.description,
+        category: tool.package?.category,
+        version: tool.package?.npmVersion,
+        qualityScore: tool.qualityScore,
+        frameworks: tool.package?.frameworks,
+        env: tool.package?.env,
+        importUrl: `https://esm.sh/${tool.package?.npmPackageName}@${tool.package?.npmVersion}`,
+        importHealth: tool.importHealth,
+        executionHealth: tool.executionHealth,
+        healthCheckError: tool.healthCheckError,
+        lastHealthCheck: tool.lastHealthCheck,
+      })),
+      total: tools.length,
     });
   } catch (error) {
     console.error('Failed to fetch tools:', error);

apps/railway-executor/server.ts

@@ -530,9 +530,18 @@ async function executeTool(req: Request): Promise<Response> {
       console.log('⚠️  No env object in request body');
     }
 
-    // Execute the tool
+    // Execute the tool with AI SDK execution context
+    // Some tools expect a second argument with { abortSignal, ... }
+    const abortController = new AbortController();
+    const executionContext = {
+      abortSignal: abortController.signal,
+      // Add other context properties that AI SDK tools might expect
+      messages: [],
+      toolCallId: `exec_${Date.now()}`,
+    };
+
     console.log(`🚀 Executing ${cacheKey} with params:`, params);
-    const result = await toolModule.execute(params || {});
+    const result = await toolModule.execute(params || {}, executionContext);
 
     const executionTimeMs = Date.now() - startTime;
     console.log(`✅ Execution complete in ${executionTimeMs}ms`);

broken-tools.md

@@ -0,0 +1,61 @@
+# Broken Tools Classification
+
+This document tracks the different types of tool failures encountered in the TPMJS executor and the strategies for handling them.
+
+## Error Categories
+
+| Error Type | Example | Root Cause | Strategy |
+|------------|---------|------------|----------|
+| **Invalid structure** | `fish-joke-generator`, `@tpmjs/text-transformer` | Not an AI SDK tool (missing `description` or `execute`) | Mark as BROKEN, filter from search results |
+| **Module not found** | `@thomasdavis/cows@0.0.1` | Package doesn't exist on npm/esm.sh | Mark as BROKEN, consider removing from registry |
+| **Factory function** | `@tavily/ai-sdk/tavilySearch` | Tool is a factory that needs config to initialize | Need to detect and call with appropriate config |
+| **Missing env var** | `@exalabs/ai-sdk/webSearch` | Requires API key (e.g., `EXA_API_KEY`) not provided | Import: HEALTHY, Execution: BROKEN with clear error message |
+| **Missing execution context** | `@parallel-web/ai-sdk-tools/extractTool` | Tool expects `{ abortSignal }` as 2nd arg to `execute()` | Fix executor to pass execution context |
+
+## Detailed Examples
+
+### Invalid Structure
+```
+❌ Invalid AI SDK tool structure: {
+  hasDescription: false,
+  hasExecute: false,
+  hasInputSchema: false,
+  keys: ["FishJokeSchema", "fishJoker", "createFishJoker", ...]
+}
+```
+These packages export utility functions or schemas, not AI SDK tools.
+
+### Module Not Found
+```
+❌ Failed to load tool: TypeError: Module not found "https://esm.sh/@thomasdavis/cows@0.0.1"
+```
+Package was registered but doesn't exist on npm or was unpublished.
+
+### Factory Function
+```
+❌ Tool "tavilySearch" is a factory function but couldn't be initialized.
+   Tried: no-args, config object, and single-arg patterns.
+   Hint: This tool may require specific configuration. Check package documentation.
+```
+Tool exports a factory like `tavilySearch({ apiKey })` instead of a ready-to-use tool object.
+
+### Missing Env Var
+```
+❌ EXA_API_KEY is required. Set it in environment variables or pass it in config.
+```
+Tool loaded successfully but execution fails without required credentials.
+
+### Missing Execution Context
+```
+❌ Tool execution failed: TypeError: Cannot destructure property 'abortSignal' of 'undefined' as it is undefined.
+    at Object.execute (https://esm.sh/@parallel-web/ai-sdk-tools@0.1.6/...)
+```
+AI SDK tools expect `execute(params, { abortSignal, ... })` but executor only passes params.
+
+## Resolution Status
+
+- [ ] Invalid structure - Health check marks as BROKEN ✓
+- [ ] Module not found - Health check marks as BROKEN ✓
+- [ ] Factory function - Partial support (tries common patterns)
+- [ ] Missing env var - Shows clear error message ✓
+- [x] Missing execution context - **Fixed in executor**