From d597a71eb4553bacdf5fb5aa08372b08fce5d662 Mon Sep 17 00:00:00 2001 From: Ajax Davis Date: Thu, 4 Dec 2025 08:25:53 +1000 Subject: [PATCH] feat: add dynamic tool loading system with Railway executor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Implements a complete dynamic tool loading system that allows the playground to discover and load tools from the TPMJS registry at runtime. **Architecture:** - Search tool package (@tpmjs/search-registry) - Searches registry for tools - Search API endpoint (/api/tools/search) - Text-based search with scoring - Pre-flight tool loading - Automatically searches and loads tools on every message - Railway executor service (Deno) - Loads tools from esm.sh via HTTP imports - Dynamic tool loader - Calls Railway to load and execute tools remotely **Key Components:** 1. Railway Executor (apps/railway-executor/) - Deno-based service that natively supports HTTP imports - Endpoints: /load-and-describe, /execute-tool, /cache/stats, /cache/clear - Deploys to Railway with deno run --allow-net --allow-env server.ts 2. Search Tool Package (packages/tools/search-registry/) - AI SDK v6 tool for searching TPMJS registry - Uses jsonSchema + inputSchema pattern - Searches /api/tools/search endpoint 3. Search API (apps/web/src/app/api/tools/search/) - Text-based search with composite scoring - Scores: text relevance + quality boost + download boost - Returns tool metadata with importUrl for dynamic loading 4. Dynamic Tool Loader (apps/playground/src/lib/dynamic-tool-loader.ts) - Calls Railway service to load tools from esm.sh - Creates tool wrappers that execute remotely - Process-level module cache + per-conversation tracking 5. Pre-flight Loading (apps/playground/src/app/api/chat/route.ts) - Automatically searches for tools on every user message - Loads top 5 matching tools before agent processes request - Merges with static tools for seamless experience **Technical Decisions:** - Deno over Node.js: Native HTTP import support without flags - Remote execution: Tools run in Railway sandbox, not Vercel - Pre-flight loading: Better UX than two-turn search pattern - Text search: BM25 had dependency issues, simple scoring works well **Environment Variables:** - RAILWAY_SERVICE_URL: https://endearing-commitment-production.up.railway.app 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- DYNAMIC_IMPORT_ISSUE.md | 864 + DYNAMIC_TOOL_LOADING_PRD.md | 988 + IMPLEMENTATION_STATUS.md | 220 + RAILWAY_DEPLOYMENT_NOTE.md | 98 + RAILWAY_DYNAMIC_TOOL_LOADER.md | 376 + ai-sdk-v6.md | 31867 ++++++++++++++++ apps/playground/next-env.d.ts | 2 +- apps/playground/next.config.ts | 3 + apps/playground/package.json | 2 + apps/playground/src/app/api/chat/route.ts | 125 +- apps/playground/src/hooks/useChat.ts | 29 +- .../playground/src/lib/dynamic-tool-loader.ts | 182 + apps/playground/src/lib/tool-loader.ts | 8 +- apps/railway-executor/.gitignore | 3 + apps/railway-executor/README.md | 160 + apps/railway-executor/package-lock.json | 939 + apps/railway-executor/package.json | 15 + apps/railway-executor/railway.json | 11 + apps/railway-executor/server.js | 229 + apps/railway-executor/server.ts | 279 + apps/web/package.json | 1 + apps/web/src/app/api/tools/search/route.ts | 124 + manual-tools.ts | 42 + packages/tools/search-registry/.npmignore | 5 + packages/tools/search-registry/package.json | 63 + packages/tools/search-registry/src/index.ts | 101 + packages/tools/search-registry/tsconfig.json | 12 + pnpm-lock.yaml | 322 +- 28 files changed, 37043 insertions(+), 27 deletions(-) create mode 100644 DYNAMIC_IMPORT_ISSUE.md create mode 100644 DYNAMIC_TOOL_LOADING_PRD.md create mode 100644 IMPLEMENTATION_STATUS.md create mode 100644 RAILWAY_DEPLOYMENT_NOTE.md create mode 100644 RAILWAY_DYNAMIC_TOOL_LOADER.md create mode 100644 ai-sdk-v6.md create mode 100644 apps/playground/src/lib/dynamic-tool-loader.ts create mode 100644 apps/railway-executor/.gitignore create mode 100644 apps/railway-executor/README.md create mode 100644 apps/railway-executor/package-lock.json create mode 100644 apps/railway-executor/package.json create mode 100644 apps/railway-executor/railway.json create mode 100644 apps/railway-executor/server.js create mode 100644 apps/railway-executor/server.ts create mode 100644 apps/web/src/app/api/tools/search/route.ts create mode 100644 packages/tools/search-registry/.npmignore create mode 100644 packages/tools/search-registry/package.json create mode 100644 packages/tools/search-registry/src/index.ts create mode 100644 packages/tools/search-registry/tsconfig.json diff --git a/DYNAMIC_IMPORT_ISSUE.md b/DYNAMIC_IMPORT_ISSUE.md new file mode 100644 index 0000000..9523485 --- /dev/null +++ b/DYNAMIC_IMPORT_ISSUE.md @@ -0,0 +1,864 @@ +# Dynamic Import Issue: Cannot Import ESM Modules from CDN in Next.js Server-Side API Route + +## Executive Summary + +We're building a dynamic tool loading system where AI agents can discover and load tools at runtime from npm packages via esm.sh CDN. The system successfully searches and finds relevant tools, but fails when trying to dynamically import them using `import()` in a Next.js App Router API route. + +**Error**: `Error: Cannot find module 'unknown'` with code `MODULE_NOT_FOUND` + +**Critical Question**: How can we dynamically import ESM modules from external URLs (like esm.sh) in Next.js 16 App Router API routes running in Node.js runtime? + +--- + +## System Architecture + +### High-Level Flow + +``` +1. User sends message → "use firecrawl to search for ajax davis" +2. Chat API extracts query → "use firecrawl to search for ajax davis" +3. Pre-flight search → Calls searchTpmjsToolsTool.execute({ query, limit: 5 }) +4. Search API returns → Top 5 matching tools from database (BM25-like scoring) +5. Dynamic loading → Tries to import tools from esm.sh URLs ❌ FAILS HERE +6. Agent uses tools → Would pass loaded tools to AI model +``` + +### Tech Stack + +- **Framework**: Next.js 16.0.4 +- **Build Tool**: Turbopack (default in Next.js 15+) +- **Runtime**: Node.js (not edge) +- **Package Manager**: pnpm (monorepo with workspaces) +- **Deployment Target**: Vercel (eventually, currently local dev) +- **AI SDK**: Vercel AI SDK v6.0.0-beta.124 +- **Model**: OpenAI GPT-4o-mini via `streamText()` + +### Monorepo Structure + +``` +tpmjs/ +├── apps/ +│ ├── playground/ # Next.js app with chat interface +│ │ └── src/ +│ │ ├── app/api/chat/route.ts # Where dynamic import fails +│ │ └── lib/dynamic-tool-loader.ts +│ └── web/ # Tool registry website +│ └── src/app/api/tools/search/route.ts +└── packages/ + └── tools/ + ├── hello/ # Static tool (works fine) + └── search-registry/ # Meta-tool for searching registry +``` + +--- + +## Detailed Code Implementation + +### File 1: `apps/playground/src/lib/dynamic-tool-loader.ts` + +**Purpose**: Load tools dynamically from esm.sh CDN + +```typescript +// Cache for imported tool modules (process-level) +const moduleCache = new Map(); + +// Cache for per-conversation active tools +const conversationTools = new Map>(); + +/** + * Generate cache key for a tool + */ +function getCacheKey(packageName: string, exportName: string): string { + return `${packageName}::${exportName}`; +} + +/** + * Validate that an import is a valid AI SDK tool + */ +function isValidTool(value: any): boolean { + return ( + value && + typeof value === 'object' && + typeof value.description === 'string' && + typeof value.execute === 'function' + ); +} + +/** + * Dynamically import a tool from ESM CDN + * + * THIS IS WHERE IT FAILS ❌ + */ +export async function loadToolDynamically( + packageName: string, + exportName: string, + version: string, + importUrl?: string +): Promise { + const cacheKey = getCacheKey(packageName, exportName); + + // Check cache first + if (moduleCache.has(cacheKey)) { + console.log(`✅ Cache hit: ${cacheKey}`); + return moduleCache.get(cacheKey); + } + + // Build import URL + const url = importUrl || `https://esm.sh/${packageName}@${version}`; + + try { + console.log(`đŸ“Ļ Importing: ${url}`); + // Example: https://esm.sh/firecrawl-aisdk@0.7.2 + + // Dynamic import with @vite-ignore to bypass bundler + const module = await import(/* @vite-ignore */ url); + + console.log(`🔍 Module imported successfully`); + console.log(`🔍 Module type: ${typeof module}`); + console.log(`🔍 Module keys: ${Object.keys(module).join(', ')}`); + console.log(`🔍 Looking for export: "${exportName}"`); + console.log(`🔍 Export exists: ${exportName in module}`); + console.log(`🔍 Export type: ${typeof module[exportName]}`); + + // Get the specific export + const tool = module[exportName]; + + if (!tool) { + console.error(`❌ Export "${exportName}" not found in module. Available exports:`, Object.keys(module)); + return null; + } + + console.log(`🔍 Tool structure:`, { + hasDescription: 'description' in tool, + hasExecute: 'execute' in tool, + hasInputSchema: 'inputSchema' in tool, + keys: Object.keys(tool), + }); + + if (!isValidTool(tool)) { + console.error(`❌ Invalid tool structure: ${exportName} from ${packageName}`); + console.error(` Tool:`, tool); + return null; + } + + // Cache successful import + moduleCache.set(cacheKey, tool); + console.log(`✅ Loaded: ${cacheKey}`); + + return tool; + } catch (error) { + console.error(`❌ Failed to load ${packageName}#${exportName}:`, error); + console.error(` URL: ${url}`); + console.error(` Stack:`, error instanceof Error ? error.stack : 'No stack trace'); + return null; + } +} + +/** + * Load multiple tools in parallel + */ +export async function loadToolsBatch( + toolMetadata: Array<{ + packageName: string; + exportName: string; + version: string; + importUrl?: string; + }> +): Promise> { + const promises = toolMetadata.map((meta) => + loadToolDynamically( + meta.packageName, + meta.exportName, + meta.version, + meta.importUrl + ).then((tool) => ({ + key: getCacheKey(meta.packageName, meta.exportName), + tool, + })) + ); + + const results = await Promise.all(promises); + + const tools: Record = {}; + for (const { key, tool } of results) { + if (tool) { + tools[key] = tool; + } + } + + return tools; +} +``` + +### File 2: `apps/playground/src/app/api/chat/route.ts` + +**Purpose**: Main chat API that orchestrates tool discovery and loading + +```typescript +import { createOpenAI } from '@ai-sdk/openai'; +import { type UIMessage, convertToModelMessages, stepCountIs, streamText } from 'ai'; +import type { NextRequest } from 'next/server'; +import { NextResponse } from 'next/server'; +import { env } from '~/env'; +import { loadAllTools, sanitizeToolName } from '~/lib/tool-loader'; +import { searchTpmjsToolsTool } from '@tpmjs/search-registry'; +import { + loadToolsBatch, + addConversationTools, +} from '~/lib/dynamic-tool-loader'; + +export const runtime = 'nodejs'; // ⚠ī¸ Important: We're using Node.js runtime, not edge +export const dynamic = 'force-dynamic'; +export const maxDuration = 60; + +// Initialize OpenAI provider +const openai = createOpenAI({ + apiKey: env.OPENAI_API_KEY, +}); + +// Add conversation state tracking (in-memory for MVP) +const conversationStates = new Map }>(); + +/** + * POST /api/chat + * Chat with AI agent that can execute TPMJS tools + */ +export async function POST(request: NextRequest) { + try { + const body = await request.json(); + console.log('đŸ“Ĩ Request body:', JSON.stringify(body, null, 2)); + + const messages: UIMessage[] = body.messages || []; + const conversationId: string = body.conversationId || 'default'; + + console.log(`🔑 Conversation ID: ${conversationId}`); + + // Get or create conversation state + if (!conversationStates.has(conversationId)) { + console.log('✨ Creating new conversation state'); + conversationStates.set(conversationId, { loadedTools: {} }); + } + const state = conversationStates.get(conversationId)!; + console.log(`📊 Current loaded tools in conversation: ${Object.keys(state.loadedTools).length}`); + + // 1. Load static tools + search tool + const staticTools = await loadAllTools(); + console.log(`🔧 Loaded ${Object.keys(staticTools).length} static tools`); + + staticTools.searchTpmjsTools = searchTpmjsToolsTool; + console.log('✅ Added searchTpmjsTools to static tools'); + + // 2. Extract user query from last message for tool search + const lastMessage = messages[messages.length - 1]; + let userQuery = ''; + if (lastMessage?.role === 'user') { + const parts = (lastMessage as any).parts || []; + for (const part of parts) { + if (part.type === 'text') { + userQuery = part.text; + break; + } + } + } + + console.log(`đŸ’Ŧ User query: "${userQuery}"`); + + // 3. Automatically search for relevant tools based on the user's message + if (userQuery && userQuery.trim().length > 0) { + console.log('🔎 Searching for relevant tools...'); + + try { + const searchResult = await searchTpmjsToolsTool.execute({ + query: userQuery, + limit: 5, // Get top 5 relevant tools + }, {} as any); + + console.log(`đŸ“Ļ Found ${searchResult.matchCount} matching tools`); + + if (searchResult.tools && searchResult.tools.length > 0) { + console.log(`🔧 Tools found:`, searchResult.tools.map((t: any) => `${t.packageName}/${t.exportName}`)); + + // Dynamically load tools from esm.sh + console.log(`đŸ“Ĩ Loading ${searchResult.tools.length} tools dynamically...`); + + const toolsToLoad = searchResult.tools.map((meta: any) => ({ + packageName: meta.packageName, + exportName: meta.exportName, + version: meta.version, + importUrl: meta.importUrl, + })); + + try { + // ❌ THIS IS WHERE IT FAILS + const loadedTools = await loadToolsBatch(toolsToLoad); + console.log(`✅ Successfully loaded ${Object.keys(loadedTools).length} tools`); + + // Add sanitized tools to conversation state + for (const [key, tool] of Object.entries(loadedTools)) { + const [pkg, exp] = key.split('::'); + const sanitizedKey = sanitizeToolName(`${pkg}-${exp}`); + state.loadedTools[sanitizedKey] = tool; + console.log(`✅ Added to conversation: ${sanitizedKey}`); + } + + // Track for this conversation + addConversationTools(conversationId, Object.keys(state.loadedTools)); + } catch (error) { + console.error('❌ Error loading tools:', error); + } + } else { + console.log('ℹī¸ No matching tools found for this query'); + } + } catch (error) { + console.error('❌ Error searching for tools:', error); + } + } + + // 4. Merge with conversation's dynamically loaded tools + const allTools: Record = { ...staticTools, ...state.loadedTools }; + + // 5. Build system prompt with available tools + const toolsList = Object.keys(allTools) + .map((name) => { + const tool = allTools[name] as { description?: string } | undefined; + return `- ${name}: ${tool?.description || 'No description'}`; + }) + .join('\n'); + + const system = `You are a helpful AI assistant that can use TPMJS tools to help users. + +Available tools: +${toolsList} + +When you use a tool, you MUST always follow up with a natural language answer to the user summarizing the result.`; + + // 6. Stream response with all available tools + const result = streamText({ + model: openai('gpt-4o-mini'), + system, + messages: convertToModelMessages(messages), + tools: allTools, + stopWhen: stepCountIs(5), + }); + + return result.toUIMessageStreamResponse(); + } catch (error) { + console.error('Chat API error:', error); + + return new Response( + JSON.stringify({ + success: false, + error: error instanceof Error ? error.message : 'Unknown error', + }), + { + status: 500, + headers: { 'Content-Type': 'application/json' }, + } + ); + } +} +``` + +### File 3: Example Tool Metadata (from search API) + +When we search for "firecrawl", the search API returns: + +```json +{ + "success": true, + "query": "firecrawl ajax davis", + "results": { + "total": 29, + "returned": 5, + "tools": [ + { + "id": "cm4abc123", + "exportName": "searchTool", + "description": "Search the web using Firecrawl's search API", + "qualityScore": 0.85, + "package": { + "npmPackageName": "firecrawl-aisdk", + "npmVersion": "0.7.2", + "category": "web-scraping", + "frameworks": ["vercel-ai"], + "env": "server" + }, + "importUrl": "https://esm.sh/firecrawl-aisdk@0.7.2", + "cdnUrl": "https://cdn.jsdelivr.net/npm/firecrawl-aisdk@0.7.2/+esm" + } + ] + } +} +``` + +So we're trying to: +```typescript +const module = await import('https://esm.sh/firecrawl-aisdk@0.7.2'); +const tool = module.searchTool; // Get the exported tool +``` + +--- + +## The Error + +### Console Output + +``` +đŸ“Ĩ Loading 5 tools dynamically... +đŸ“Ļ Importing: https://esm.sh/firecrawl-aisdk@0.7.2 +❌ Failed to load firecrawl-aisdk#searchTool: Error: Cannot find module 'unknown' + at (.next/dev/server/chunks/[root-of-the-server]__746deca2._.js:357:23) + at loadToolDynamically (.next/dev/server/chunks/[root-of-the-server]__746deca2._.js:360:11) + at (src/lib/dynamic-tool-loader.ts:108:5) + at Array.map () + at loadToolsBatch (src/lib/dynamic-tool-loader.ts:107:33) + at POST (src/app/api/chat/route.ts:104:53) + { + code: 'MODULE_NOT_FOUND' + } + URL: https://esm.sh/firecrawl-aisdk@0.7.2 + Stack: Error: Cannot find module 'unknown' + at /Users/ajaxdavis/repos/tpmjs/tpmjs/apps/playground/.next/dev/server/chunks/[root-of-the-server]__746deca2._.js:357:23 + at loadToolDynamically (/Users/ajaxdavis/repos/tpmjs/tpmjs/apps/playground/.next/dev/server/chunks/[root-of-the-server]__746deca2._.js:360:11) +``` + +### Key Observations + +1. **Error happens immediately** - Never gets to our debug logs after `await import()` +2. **Error is MODULE_NOT_FOUND** - Treating URL as a module path +3. **Error says "unknown"** - Not even using the actual module name +4. **Code is in .next/dev/server/chunks/** - Next.js/Turbopack transformed our code +5. **Same error for all packages** - firecrawl-aisdk, @exalabs/ai-sdk, etc. + +--- + +## Verification: The URL Works + +### Manual Test 1: Browser + +``` +Visit: https://esm.sh/firecrawl-aisdk@0.7.2 +``` + +Returns valid ESM module: +```javascript +/* esm.sh - firecrawl-aisdk@0.7.2 */ +import * as __1$ from "/v135/@ai-sdk/provider-utils@2.0.8/..."; +// ... rest of module code +export { searchTool, scrapeTool, crawlTool }; +``` + +### Manual Test 2: Plain Node.js Script + +Create `test-import.mjs`: +```javascript +const module = await import('https://esm.sh/firecrawl-aisdk@0.7.2'); +console.log('Module:', module); +console.log('Exports:', Object.keys(module)); +``` + +Run: `node test-import.mjs` + +**Expected**: Would work in plain Node.js with `--experimental-network-imports` flag +**In Next.js**: Can't even get this far + +--- + +## What We've Tried + +### Attempt 1: `/* @vite-ignore */` Comment +```typescript +const module = await import(/* @vite-ignore */ url); +``` +**Result**: Still fails with MODULE_NOT_FOUND + +### Attempt 2: `/* webpackIgnore: true */` Comment +```typescript +const module = await import(/* webpackIgnore: true */ url); +``` +**Result**: Still fails with MODULE_NOT_FOUND + +### Attempt 3: Force Dynamic Runtime +```typescript +export const runtime = 'nodejs'; +export const dynamic = 'force-dynamic'; +``` +**Result**: Still fails (we're already using this) + +### Attempt 4: Verify esm.sh Works +- Tested URLs in browser: ✅ Works +- All packages return valid ESM: ✅ Valid +- esm.sh is accessible: ✅ Reachable + +### Attempt 5: Check Static Imports +```typescript +import { helloWorldTool } from '@tpmjs/hello'; +``` +**Result**: Works perfectly (but bundled at build time) + +--- + +## Configuration Files + +### `apps/playground/next.config.ts` + +```typescript +import type { NextConfig } from 'next'; + +const config: NextConfig = { + reactStrictMode: true, + transpilePackages: ['@tpmjs/ui'], + experimental: { + turbo: { + // Using Turbopack (Next.js 15+ default) + }, + }, +}; + +export default config; +``` + +### `apps/playground/package.json` (relevant parts) + +```json +{ + "name": "@tpmjs/playground", + "version": "0.0.0", + "private": true, + "scripts": { + "dev": "next dev --port 3001", + "build": "next build", + "start": "next start" + }, + "dependencies": { + "@ai-sdk/openai": "^1.0.15", + "@tpmjs/hello": "workspace:*", + "@tpmjs/search-registry": "workspace:*", + "ai": "6.0.0-beta.124", + "next": "16.0.4", + "react": "19.0.0" + } +} +``` + +### `turbo.json` (monorepo config) + +```json +{ + "$schema": "https://turbo.build/schema.json", + "tasks": { + "dev": { + "cache": false, + "persistent": true + }, + "build": { + "dependsOn": ["^build"], + "outputs": [".next/**", "dist/**"] + } + } +} +``` + +--- + +## Why This Matters + +### The Bigger Picture + +We're building a **self-referential tool discovery system**: + +1. **Tool Registry** (tpmjs.com) - Indexes all TPMJS-compatible tools from npm +2. **Search Tool** - AI SDK tool that searches the registry +3. **Dynamic Loader** - Loads found tools at runtime +4. **AI Agent** - Uses dynamically loaded tools + +This creates infinite extensibility: +- No need to bundle all possible tools +- Tools can be published to npm independently +- System discovers and loads tools as needed +- Bundle size stays small + +### Use Case Example + +``` +User: "Search Wikipedia for quantum computing" + ↓ +System searches registry: Finds "wikipedia-aisdk" tool + ↓ +System loads tool: import('https://esm.sh/wikipedia-aisdk@1.0.0') + ↓ +AI uses tool: wikipediaSearchTool.execute({ query: "quantum computing" }) + ↓ +User gets answer with Wikipedia citations +``` + +--- + +## Possible Root Causes + +### Hypothesis 1: Turbopack Doesn't Support Dynamic Import URLs +- Turbopack intercepts all `import()` calls +- Transforms them to module resolution +- Doesn't handle external URLs + +### Hypothesis 2: Next.js Security Restriction +- Next.js blocks dynamic imports from external URLs for security +- Prevents arbitrary code execution +- No way to whitelist esm.sh + +### Hypothesis 3: Dev Mode Only Issue +- Turbopack dev mode has more restrictions +- Production webpack build might work +- But we need dev mode to work too + +### Hypothesis 4: Node.js Runtime Limitation in Next.js +- Next.js Node.js runtime is sandboxed +- Dynamic imports are intercepted before reaching Node.js +- Plain Node.js would work with --experimental-network-imports + +--- + +## Alternative Approaches We're Considering + +### Option A: Fetch + VM Module +```typescript +import { SourceTextModule } from 'vm'; + +const response = await fetch(url); +const code = await response.text(); +const module = new SourceTextModule(code); +await module.link(() => {}); +await module.evaluate(); +const exports = module.namespace; +``` + +**Pros**: Bypasses import() entirely +**Cons**: Complex, security concerns, might not work in Next.js + +### Option B: Separate Microservice +```typescript +// New service: tool-loader-service (Express or Fastify) +POST /load-tool +Body: { packageName, exportName, version } +Response: { tool: } +``` + +**Pros**: Full control, definitely works +**Cons**: Extra infrastructure, latency, complexity + +### Option C: Switch to Edge Runtime +```typescript +export const runtime = 'edge'; // Instead of 'nodejs' +``` + +**Pros**: Edge might have different import behavior +**Cons**: Edge has limitations (no Node.js APIs), might still not work + +### Option D: Pre-bundle Common Tools +```typescript +// Generate static imports for top 100 tools +import { tool1 } from 'package1'; +import { tool2 } from 'package2'; +// ... etc +``` + +**Pros**: Definitely works +**Cons**: Defeats the purpose, huge bundle size + +### Option E: Use unpkg or jsdelivr with Different Strategy +```typescript +// Fetch raw code, eval in isolated context +const response = await fetch(`https://unpkg.com/${pkg}@${ver}/dist/index.mjs`); +const code = await response.text(); +const exports = evalInContext(code); +``` + +**Pros**: More control +**Cons**: Same security/execution issues + +--- + +## Specific Questions for ChatGPT + +### Question 1: Is This Possible? +**Can Next.js 16 App Router API routes (Node.js runtime) dynamically import ESM modules from external URLs using `import()`?** + +If yes: +- What configuration is needed? +- Are there security allowlists? +- Does it work in both dev and production? + +If no: +- Why not? +- What's the recommended alternative? +- Is this a Turbopack limitation or Next.js design? + +### Question 2: Turbopack Behavior +**Does Turbopack intercept all `import()` calls, even with magic comments?** + +We've tried: +- `/* @vite-ignore */` +- `/* webpackIgnore: true */` + +None work. Is there a Turbopack-specific comment or config? + +### Question 3: Edge vs Node Runtime +**Would switching to edge runtime change import behavior?** + +```typescript +export const runtime = 'edge'; // vs 'nodejs' +``` + +Does edge runtime allow dynamic imports from URLs? + +### Question 4: Best Practice +**What's the recommended way to implement dynamic tool loading in Next.js?** + +Given constraints: +- Need to load arbitrary npm packages at runtime +- Packages are ESM modules from CDN +- Can't pre-bundle all possibilities +- Need to work in production on Vercel + +### Question 5: Security Model +**Is Next.js intentionally blocking this for security?** + +- Is there a whitelist for allowed CDNs? +- Can we configure allowed import sources? +- Is this related to CSP or other security headers? + +--- + +## Environment Details + +### Versions +```json +{ + "next": "16.0.4", + "react": "19.0.0", + "turbo": "2.6.1", + "pnpm": "9.15.0", + "node": "v20.11.0", + "ai": "6.0.0-beta.124" +} +``` + +### Operating System +- **OS**: macOS (Darwin 23.5.0) +- **Architecture**: arm64 (Apple Silicon) + +### Development Commands +```bash +# Start dev server +pnpm dev --filter=@tpmjs/playground + +# Output +▲ Next.js 16.0.4 (Turbopack) +- Local: http://localhost:3001 +- Network: http://192.168.0.25:3001 +✓ Ready in 2.5s +``` + +### Build Output Structure +``` +apps/playground/.next/ +├── dev/ +│ └── server/ +│ └── chunks/ +│ └── [root-of-the-server]__746deca2._.js # ← Error originates here +``` + +--- + +## Success Criteria + +### What We Need Working + +```typescript +// In Next.js API route (Node.js runtime) +const url = 'https://esm.sh/firecrawl-aisdk@0.7.2'; +const module = await import(url); +const tool = module.searchTool; + +console.log(tool.description); // "Search the web using Firecrawl's search API" +console.log(typeof tool.execute); // "function" + +// Tool is ready to use with AI SDK +const result = await tool.execute({ query: "test" }, context); +``` + +### Acceptable Outcomes + +1. ✅ **Best**: Dynamic `import()` works with configuration change +2. ✅ **Good**: Alternative approach that doesn't require microservice +3. ✅ **Acceptable**: Workaround that works in production even if dev is tricky +4. ❌ **Unacceptable**: "You can't do this in Next.js" without alternative + +--- + +## Additional Context + +### Why Not Just Bundle Everything? + +Currently have ~30 tools in registry, growing to 100s or 1000s: +- Bundle size would be massive (10+ MB) +- Most tools won't be used in most conversations +- Tools are published independently by community +- Want instant availability of new tools without redeploying + +### Why esm.sh Specifically? + +- ✅ Converts any npm package to ESM +- ✅ Handles dependencies automatically +- ✅ Fast CDN with caching +- ✅ No build step required +- ✅ Version pinning built-in + +But we're flexible - if jsdelivr, unpkg, or another approach works better, we'll use it. + +### Static Imports Work Fine + +This works perfectly (but defeats the purpose): +```typescript +import { searchTool } from 'firecrawl-aisdk'; +``` + +The tools themselves are fine. We just can't load them dynamically. + +--- + +## What We're Hoping For + +### Ideal Answer Format + +1. **Root cause**: Why it's failing +2. **Solution**: How to fix it (with code example) +3. **Configuration**: Any Next.js config needed +4. **Limitations**: What won't work / tradeoffs +5. **Alternatives**: If dynamic import truly impossible + +### We're Happy to Try + +- Different CDN (unpkg, jsdelivr, etc.) +- Different import strategy (fetch + eval, vm module, etc.) +- Different runtime (edge if it works) +- Different Next.js version (if specific version supports this) +- Webpack instead of Turbopack (if webpack handles this better) + +We just need a path forward that enables runtime tool loading in a production Next.js app on Vercel. + +--- + +## Files to Reference + +All code is in this monorepo: +- `apps/playground/src/lib/dynamic-tool-loader.ts` - Import logic +- `apps/playground/src/app/api/chat/route.ts` - API route +- `apps/playground/next.config.ts` - Next.js config +- `DYNAMIC_IMPORT_ISSUE.md` - This document + +--- + +## Thank You + +This is a critical blocker for our dynamic tool loading system. Any insights, workarounds, or alternative approaches would be immensely helpful! diff --git a/DYNAMIC_TOOL_LOADING_PRD.md b/DYNAMIC_TOOL_LOADING_PRD.md new file mode 100644 index 0000000..f777951 --- /dev/null +++ b/DYNAMIC_TOOL_LOADING_PRD.md @@ -0,0 +1,988 @@ +# Dynamic Tool Loading System - Product Requirements Document + +## Executive Summary + +Build a self-referential tool discovery system where AI agents can search the TPMJS registry, find relevant tools, and dynamically import them during conversation. This creates a "meta-tool" that makes the entire TPMJS ecosystem available to any agent at runtime. + +**Core Innovation:** An AI agent can discover and load tools on-demand by searching the registry, rather than having all tools pre-loaded. This enables infinite tool extensibility without bundle size concerns. + +--- + +## Problem Statement + +### Current Limitations + +1. **Static Tool Loading**: Playground requires all tools to be hardcoded in `tool-loader.ts` +2. **Bundle Size**: Loading many tools increases bundle size and initialization time +3. **Discovery Gap**: Agents can't discover new tools that match their current task +4. **Manual Updates**: Adding tools requires code changes and redeployment + +### User Pain Points + +- Users want agents to access the full TPMJS registry without manual configuration +- Developers want to publish tools that are immediately available to all agents +- Agents need context-aware tool selection based on the conversation + +--- + +## Solution Overview + +### The Meta-Tool: `searchTpmjsTools` + +A TPMJS tool that searches the TPMJS registry and returns tool metadata needed for dynamic import. + +**Flow:** +``` +User: "Search Wikipedia for quantum computing" + ↓ +Agent: Calls searchTpmjsTools("wikipedia search") + ↓ +API: Returns tools matching "wikipedia" (BM25 search) + ↓ +Playground: Dynamically imports matching tools + ↓ +Agent: Now has Wikipedia tools available, uses them +``` + +### Key Components + +1. **`@tpmjs/search-registry`** - NPM package exporting `searchTpmjsToolsTool` +2. **`/api/tools/search`** - New API endpoint with BM25 full-text search +3. **Playground Dynamic Loader** - Runtime tool import system +4. **Tool Import Strategy** - ESM CDN imports or bundled approach + +--- + +## Technical Architecture + +### Component 1: Search Tool Package + +**Package:** `packages/tools/search-registry/` + +```typescript +// packages/tools/search-registry/src/index.ts +import { tool } from 'ai'; +import { z } from 'zod'; + +export const searchTpmjsToolsTool = tool({ + description: 'Search the TPMJS tool registry to find AI SDK tools. Use this when you need a tool that isn\'t currently available. Returns tool metadata including package names and descriptions.', + parameters: z.object({ + query: z.string().describe('Search query (e.g., "weather", "database", "wikipedia")'), + category: z.enum([ + 'text-analysis', + 'code-generation', + 'data-processing', + 'image-generation', + 'audio-processing', + 'search', + 'integration', + 'other' + ]).optional().describe('Filter by tool category'), + limit: z.number().min(1).max(20).default(10).describe('Max number of tools to return'), + }), + execute: async ({ query, category, limit }) => { + // Call TPMJS search API + const params = new URLSearchParams({ + q: query, + limit: String(limit), + ...(category && { category }), + }); + + const response = await fetch( + `https://tpmjs.com/api/tools/search?${params}` + ); + + if (!response.ok) { + throw new Error(`Search failed: ${response.statusText}`); + } + + const data = await response.json(); + + // Return structured tool metadata + return { + query, + matchCount: data.tools.length, + tools: data.tools.map((tool: any) => ({ + packageName: tool.package.npmPackageName, + exportName: tool.exportName, + description: tool.description, + category: tool.package.category, + qualityScore: tool.qualityScore, + frameworks: tool.package.frameworks, + env: tool.package.env, + })), + }; + }, +}); +``` + +**Package Metadata:** + +```json +{ + "name": "@tpmjs/search-registry", + "version": "0.1.0", + "description": "AI SDK tool for searching the TPMJS tool registry", + "keywords": ["tpmjs-tool", "ai", "search"], + "tpmjs": { + "category": "search", + "frameworks": ["vercel-ai"], + "tools": [ + { + "exportName": "searchTpmjsToolsTool", + "description": "Search the TPMJS tool registry to find AI SDK tools by keyword, category, or description. Returns tool metadata for dynamic loading.", + "parameters": [ + { + "name": "query", + "type": "string", + "description": "Search query (keywords, tool names, descriptions)", + "required": true + }, + { + "name": "category", + "type": "string", + "description": "Filter by category (text-analysis, search, etc.)", + "required": false + }, + { + "name": "limit", + "type": "number", + "description": "Maximum number of results (1-20, default 10)", + "required": false + } + ], + "returns": { + "type": "object", + "description": "Search results with tool metadata for dynamic import" + }, + "aiAgent": { + "useCase": "Use this tool when you need a tool that isn't currently available. For example, if asked to search Wikipedia but you don't have a Wikipedia tool, search for 'wikipedia' to find and load it.", + "examples": [ + "Search for 'weather' tools when asked about weather", + "Search for 'database' tools when working with data", + "Search for 'code' tools when generating code" + ], + "limitations": "Returns metadata only - the playground handles actual tool loading" + } + } + ] + } +} +``` + +--- + +### Component 2: BM25 Search API Endpoint + +**File:** `apps/web/src/app/api/tools/search/route.ts` + +**Requirements:** + +1. **Full-Text Search with BM25** + - Search across: tool description, package name, npm description, npm keywords + - BM25 scoring for relevance ranking + - Category filtering + - Quality score boosting (rich tier tools rank higher) + +2. **Search Implementation Options** + + **Option A: PostgreSQL Full-Text Search** + ```sql + -- Add tsvector column to tools table + ALTER TABLE tools ADD COLUMN search_vector tsvector; + + -- Create GIN index for fast full-text search + CREATE INDEX tools_search_idx ON tools USING GIN(search_vector); + + -- Update search vector on insert/update + CREATE TRIGGER tools_search_update + BEFORE INSERT OR UPDATE ON tools + FOR EACH ROW EXECUTE FUNCTION + tsvector_update_trigger(search_vector, 'pg_catalog.english', + description); + ``` + + **Option B: JavaScript BM25 Library** + ```typescript + import { BM25 } from 'bm25'; + + // Load all tools into memory (cached) + const tools = await prisma.tool.findMany({ + include: { package: true }, + }); + + // Build BM25 index + const documents = tools.map(tool => ({ + id: tool.id, + text: `${tool.description} ${tool.package.npmPackageName} ${tool.package.npmDescription} ${tool.package.npmKeywords.join(' ')}`, + })); + + const bm25 = new BM25(documents); + const results = bm25.search(query); + ``` + + **Option C: Hybrid Approach** + - Use PostgreSQL `LIKE` for exact matches (fastest) + - Fall back to BM25 for fuzzy/semantic search + - Cache search results in Redis + +3. **API Response Format** + +```typescript +// GET /api/tools/search?q=weather&category=integration&limit=10 + +{ + "success": true, + "query": "weather", + "filters": { + "category": "integration" + }, + "results": { + "total": 23, + "returned": 10, + "tools": [ + { + "id": "clx...", + "exportName": "getWeatherTool", + "description": "Get current weather data for any location using OpenWeatherMap API", + "qualityScore": 0.85, + "package": { + "npmPackageName": "@tpmjs/weather", + "npmVersion": "1.2.0", + "category": "integration", + "frameworks": ["vercel-ai"], + "env": [ + { + "name": "OPENWEATHER_API_KEY", + "description": "OpenWeatherMap API key", + "required": true + } + ], + "npmRepository": { + "type": "git", + "url": "https://github.com/user/weather-tool" + }, + "isOfficial": false + }, + // Include everything needed for dynamic import + "importUrl": "https://esm.sh/@tpmjs/weather@1.2.0", + "cdnUrl": "https://cdn.jsdelivr.net/npm/@tpmjs/weather@1.2.0/+esm" + } + // ... more tools + ] + } +} +``` + +--- + +### Component 3: Dynamic Tool Loader (Playground) + +**File:** `apps/playground/src/lib/dynamic-tool-loader.ts` + +**Requirements:** + +1. **Runtime ESM Import** + ```typescript + async function loadToolDynamically( + packageName: string, + exportName: string, + version: string + ) { + // Option 1: ESM CDN (esm.sh, unpkg, jsdelivr) + const cdnUrl = `https://esm.sh/${packageName}@${version}`; + + try { + const module = await import(/* @vite-ignore */ cdnUrl); + const tool = module[exportName]; + + if (!isValidTool(tool)) { + throw new Error(`Invalid tool: ${exportName}`); + } + + return tool; + } catch (error) { + console.error(`Failed to load ${packageName}:`, error); + return null; + } + } + ``` + +2. **Tool Caching Strategy** + ```typescript + // Cache loaded tools to avoid redundant imports + const toolCache = new Map(); + + function getCacheKey(packageName: string, exportName: string): string { + return `${packageName}::${exportName}`; + } + + async function loadToolWithCache( + packageName: string, + exportName: string, + version: string + ) { + const key = getCacheKey(packageName, exportName); + + if (toolCache.has(key)) { + return toolCache.get(key); + } + + const tool = await loadToolDynamically(packageName, exportName, version); + + if (tool) { + toolCache.set(key, tool); + } + + return tool; + } + ``` + +3. **Tool Registry Integration** + ```typescript + // Merge static tools + dynamically loaded tools + async function getAllAvailableTools( + staticTools: Record, + searchResults: SearchResult[] + ): Promise> { + const allTools = { ...staticTools }; + + // Load tools from search results + for (const result of searchResults) { + const tool = await loadToolWithCache( + result.package.npmPackageName, + result.exportName, + result.package.npmVersion + ); + + if (tool) { + const key = sanitizeToolName( + `${result.package.npmPackageName}-${result.exportName}` + ); + allTools[key] = tool; + } + } + + return allTools; + } + ``` + +--- + +### Component 4: Playground Chat Integration + +**File:** `apps/playground/src/app/api/chat/route.ts` + +**Flow:** + +1. **Initial Tool Set** + - Load static tools (hardcoded in tool-loader) + - Always include `searchTpmjsToolsTool` in initial set + +2. **Agent Invokes Search** + - Agent calls `searchTpmjsToolsTool` with query + - Search API returns matching tool metadata + - Response includes tool metadata + +3. **Dynamic Loading Trigger** + - Detect when agent successfully calls `searchTpmjsToolsTool` + - Extract tool metadata from response + - Load tools dynamically before next agent turn + +4. **Tool Availability Update** + - Merge dynamically loaded tools into available tool set + - Agent can now use newly loaded tools in subsequent turns + +**Implementation:** + +```typescript +// apps/playground/src/app/api/chat/route.ts +export async function POST(req: Request) { + const { messages } = await req.json(); + + // 1. Load static tools + search tool + let availableTools = await loadAllTools(); // static + availableTools['searchTpmjsTools'] = searchTpmjsToolsTool; // meta-tool + + // 2. Create streamText with current tools + const result = streamText({ + model: openai('gpt-4'), + messages, + tools: availableTools, + maxSteps: 10, // Allow multiple tool call rounds + + onStepFinish: async (step) => { + // 3. Check if agent called searchTpmjsToolsTool + for (const toolCall of step.toolCalls) { + if (toolCall.toolName === 'searchTpmjsTools') { + const searchResults = toolCall.result?.tools || []; + + // 4. Dynamically load tools from search results + console.log(`Loading ${searchResults.length} tools dynamically...`); + + for (const toolMeta of searchResults) { + const tool = await loadToolWithCache( + toolMeta.packageName, + toolMeta.exportName, + 'latest' // or toolMeta.version + ); + + if (tool) { + const key = sanitizeToolName( + `${toolMeta.packageName}-${toolMeta.exportName}` + ); + availableTools[key] = tool; + console.log(`✅ Loaded: ${key}`); + } + } + + // 5. Update tool registry for subsequent steps + // Note: This requires AI SDK to support dynamic tool updates + // May need to restart the streamText with updated tools + } + } + }, + }); + + return result.toDataStreamResponse(); +} +``` + +--- + +## Technical Challenges & Solutions + +### Challenge 1: AI SDK Doesn't Support Dynamic Tool Updates Mid-Stream + +**Problem:** Vercel AI SDK's `streamText` sets tools at initialization. Can't add tools after streaming starts. + +**Solutions:** + +**Option A: Multi-Turn Pattern** +```typescript +// Turn 1: Agent searches for tools +// Turn 2: Agent uses loaded tools + +// Detect search tool call, return early +if (hasSearchToolCall) { + return new Response(JSON.stringify({ + type: 'tools_loaded', + tools: searchResults, + message: 'Tools loaded. Please continue your request.', + })); +} +``` + +**Option B: Pre-Flight Search (Recommended)** +```typescript +// Before calling streamText, analyze user message +const needsTools = await analyzeMessageForToolNeeds(userMessage); + +if (needsTools.length > 0) { + // Pre-load tools based on intent + const searchResults = await searchTools(needsTools); + const dynamicTools = await loadToolsFromResults(searchResults); + availableTools = { ...staticTools, ...dynamicTools }; +} + +// Now call streamText with full tool set +const result = streamText({ + model, + messages, + tools: availableTools, +}); +``` + +**Option C: Agent-Driven Two-Phase** +```typescript +// Phase 1: Planning +const planResult = await generateText({ + model, + messages: [ + { role: 'system', content: 'Analyze this request and determine what tools are needed. Call searchTpmjsTools if needed.' }, + ...messages, + ], + tools: { searchTpmjsTools }, +}); + +// Phase 2: Execution with loaded tools +const executionResult = await streamText({ + model, + messages, + tools: { ...staticTools, ...loadedTools }, +}); +``` + +--- + +### Challenge 2: ESM Dynamic Import in Browser vs Node.js + +**Problem:** Dynamic `import()` works differently in browser vs server environments. + +**Solutions:** + +**Server-Side (Recommended):** +```typescript +// Use Node.js dynamic import +// Works with esm.sh CDN +const tool = await import(`https://esm.sh/${pkg}@${version}`); +``` + +**Client-Side (Avoid):** +```typescript +// Browser import() has CORS and CSP restrictions +// Would require: +// 1. CDN supports CORS +// 2. CSP allows script-src from CDN +// 3. Tools are browser-compatible (no Node.js APIs) +``` + +**Hybrid Approach:** +```typescript +// Load tools server-side, serialize to client +// Client displays available tools +// Server executes tool calls +``` + +--- + +### Challenge 3: Tool Dependencies & Environment Variables + +**Problem:** Dynamically loaded tools may require: +- Environment variables (API keys) +- npm dependencies not in bundle +- Node.js-specific APIs + +**Solutions:** + +**Option A: Require Pre-Configuration** +```typescript +// Before loading, check if tool requirements are met +async function canLoadTool(toolMeta: ToolMetadata): Promise { + // Check required env vars + for (const env of toolMeta.package.env || []) { + if (env.required && !process.env[env.name]) { + console.warn(`Missing required env: ${env.name}`); + return false; + } + } + + return true; +} +``` + +**Option B: Graceful Degradation** +```typescript +// Load tool, catch errors, inform agent +try { + const tool = await loadTool(packageName, exportName); + return tool; +} catch (error) { + return createStubTool(packageName, exportName, error); +} + +function createStubTool(pkg: string, exp: string, error: Error) { + return tool({ + description: `[UNAVAILABLE] ${exp} from ${pkg}: ${error.message}`, + parameters: z.object({}), + execute: async () => { + throw new Error(`Cannot execute ${exp}: ${error.message}`); + }, + }); +} +``` + +**Option C: Proxy Through Server** +```typescript +// All tools execute server-side where env vars exist +// Client just displays tool calls, server handles execution +``` + +--- + +### Challenge 4: Security & Sandboxing + +**Problem:** Dynamically importing arbitrary npm packages is a security risk. + +**Solutions:** + +**Option A: Allowlist Only** +```typescript +// Only load tools from TPMJS registry (already vetted) +const allowedPackages = await prisma.package.findMany({ + select: { npmPackageName: true } +}); + +if (!allowedPackages.includes(packageName)) { + throw new Error('Package not in TPMJS registry'); +} +``` + +**Option B: Version Pinning** +```typescript +// Only load specific versions from registry +// Don't use 'latest' to avoid supply chain attacks +const version = toolMeta.package.npmVersion; // e.g., "1.2.0" +const url = `https://esm.sh/${pkg}@${version}`; +``` + +**Option C: VM Sandbox (Advanced)** +```typescript +// Execute tools in isolated VM context +import { VM } from 'vm2'; + +const vm = new VM({ + timeout: 5000, + sandbox: { + fetch: safeFetch, // Wrapped fetch with rate limits + console: safeConsole, + }, +}); + +const tool = vm.run(toolCode); +``` + +--- + +### Challenge 5: Performance & Bundle Size + +**Problem:** Loading many tools dynamically could be slow. + +**Solutions:** + +**Option A: Lazy Loading** +```typescript +// Only load tools when agent decides to use them +// Not when they're discovered +``` + +**Option B: Parallel Loading** +```typescript +// Load multiple tools concurrently +const toolPromises = searchResults.map(result => + loadToolWithCache(result.package.npmPackageName, result.exportName, result.package.npmVersion) +); + +const tools = await Promise.all(toolPromises); +``` + +**Option C: CDN Caching** +```typescript +// Use CDN with aggressive caching +// esm.sh has built-in caching +const url = `https://esm.sh/${pkg}@${version}?target=es2022&bundle`; +``` + +--- + +## Implementation Plan + +### Phase 1: MVP (Week 1-2) + +**Goal:** Prove dynamic loading works with simple prototype + +1. **Create `@tpmjs/search-registry` package** + - Implement `searchTpmjsToolsTool` + - Publish to npm + - Add to manual-tools registry + +2. **Build `/api/tools/search` endpoint** + - Start with simple PostgreSQL `LIKE` search + - Return tool metadata with package info + - Test with curl + +3. **Implement basic dynamic loader** + - Use esm.sh CDN for imports + - Load tools server-side only + - Cache in memory + +4. **Playground integration - Two-Turn Pattern** + - User asks question + - Agent calls `searchTpmjsToolsTool` + - Backend loads tools + - Agent uses tools in next turn + +**Success Criteria:** +- Agent can search registry +- Agent can use dynamically loaded tools +- End-to-end flow works for 1-2 example tools + +--- + +### Phase 2: BM25 Search (Week 3) + +**Goal:** Improve search relevance with BM25 + +1. **Research BM25 implementation options** + - Test PostgreSQL full-text search + - Test JavaScript BM25 libraries + - Benchmark performance + +2. **Implement chosen approach** + - Add search vector column if using PostgreSQL + - Create search index + - Update search endpoint + +3. **Test search quality** + - Create test queries + - Measure precision/recall + - Compare to baseline `LIKE` search + +**Success Criteria:** +- BM25 search returns more relevant results than LIKE +- Search latency < 100ms for 95th percentile +- Agent can find tools for diverse queries + +--- + +### Phase 3: Production Hardening (Week 4) + +**Goal:** Make system production-ready + +1. **Error Handling** + - Handle import failures gracefully + - Validate tool schemas + - Return helpful error messages to agent + +2. **Security** + - Implement package allowlist + - Pin versions from registry + - Add rate limiting to search API + +3. **Performance** + - Implement Redis caching for search results + - Add CDN caching headers + - Optimize tool loading parallelism + +4. **Monitoring** + - Log all dynamic tool loads + - Track search queries and results + - Monitor import success/failure rates + +**Success Criteria:** +- System handles errors without crashing +- Security review passes +- Latency and reliability SLOs met + +--- + +### Phase 4: Advanced Features (Week 5+) + +**Goal:** Enhance UX and capabilities + +1. **Pre-flight Search** + - Analyze user message for intent + - Proactively load tools before agent call + - Reduce total turns needed + +2. **Tool Recommendations** + - "You might also need..." suggestions + - Based on tool co-occurrence data + - Help agent discover related tools + +3. **Client-Side Tool Display** + - Show which tools are available + - Indicate dynamically loaded tools + - Allow user to manually load tools + +4. **Tool Versioning** + - Support multiple versions of same tool + - Let agent choose version + - Handle breaking changes gracefully + +--- + +## Success Metrics + +### Technical Metrics + +1. **Search Quality** + - Precision@10 > 0.8 (80% of top 10 results are relevant) + - Mean Reciprocal Rank (MRR) > 0.7 + - Search latency p95 < 100ms + +2. **Tool Loading** + - Import success rate > 95% + - Tool load time p95 < 2 seconds + - Cache hit rate > 70% after warmup + +3. **End-to-End Performance** + - Total conversation latency < 5 seconds (including tool search + load + execution) + - Agent uses correct tools > 90% of time + +### User Metrics + +1. **Adoption** + - % of playground sessions using dynamic tools > 30% + - Number of unique tools loaded dynamically per week > 50 + +2. **Tool Coverage** + - % of user queries satisfied with available tools > 80% + - Tool search leading to successful task completion > 70% + +--- + +## Open Questions + +### 1. CDN Choice for ESM Imports + +**Options:** +- **esm.sh** - Purpose-built for ESM imports, fast, reliable +- **unpkg** - Popular, simple, but slower +- **jsdelivr** - Fast CDN, good for production +- **Custom bundler** - Pre-bundle tools, serve from our CDN + +**Recommendation:** Start with esm.sh for MVP, evaluate custom bundler for production. + +--- + +### 2. When to Load Tools? + +**Options:** +- **On-demand**: Load when agent calls search tool (current plan) +- **Pre-flight**: Analyze user message, load proactively +- **Lazy**: Load when agent tries to use tool (not when discovered) +- **Eager**: Load all tools from search results immediately + +**Recommendation:** Start with on-demand (Phase 1), add pre-flight in Phase 4. + +--- + +### 3. How to Handle Environment Variables? + +**Problem:** Dynamically loaded tools may need API keys (e.g., OpenWeather API). + +**Options:** +- **User provides**: UI for users to enter API keys (like playground settings) +- **Server-managed**: Admin pre-configures keys in .env +- **Graceful fail**: Load tool, but execution fails if env missing +- **Hybrid**: Some tools work without keys (free tier), others require keys + +**Recommendation:** Start with graceful fail (Phase 1), add user-provided keys (Phase 4). + +--- + +### 4. Should Tools Load Client-Side or Server-Side? + +**Client-Side Pros:** +- Reduces server load +- Faster for subsequent uses +- Better for browser-compatible tools + +**Client-Side Cons:** +- Requires CORS-enabled CDN +- CSP restrictions +- Many tools need Node.js APIs +- Exposing API keys in browser is insecure + +**Server-Side Pros:** +- Access to Node.js APIs +- Secure environment variable access +- No CORS issues +- Easier to implement + +**Server-Side Cons:** +- Requires server memory for caching +- Increases server load +- Cold starts for new tools + +**Recommendation:** Server-side for MVP (Phase 1), evaluate client-side for browser-compatible tools (Phase 4+). + +--- + +### 5. How to Handle Tool Dependencies? + +**Problem:** Some tools depend on other npm packages (e.g., `axios`, `cheerio`). + +**Options:** +- **Bundled**: CDN bundles dependencies (esm.sh does this) +- **Peer deps**: Require dependencies in playground package.json +- **Dynamic install**: npm install on-the-fly (slow, risky) +- **Pre-vetted**: Only allow tools with no/minimal dependencies + +**Recommendation:** Use esm.sh bundling (Phase 1), bundle size limits if issues arise. + +--- + +## Risk Assessment + +### High Risk + +1. **Security Vulnerability** + - **Risk**: Malicious package in registry executes code + - **Mitigation**: Allowlist registry packages, version pinning, VM sandboxing + - **Owner**: Security team + +2. **Performance Degradation** + - **Risk**: Loading many tools causes timeout/slow response + - **Mitigation**: Parallel loading, caching, lazy loading, timeouts + - **Owner**: Backend team + +### Medium Risk + +3. **Import Failures** + - **Risk**: CDN down, package incompatible, missing dependencies + - **Mitigation**: Fallback CDNs, error handling, stub tools + - **Owner**: Frontend team + +4. **AI SDK Limitations** + - **Risk**: Can't dynamically update tools mid-stream + - **Mitigation**: Two-turn pattern, pre-flight search + - **Owner**: AI team + +### Low Risk + +5. **Search Quality** + - **Risk**: BM25 doesn't return relevant tools + - **Mitigation**: A/B test search algorithms, collect feedback + - **Owner**: Search team + +--- + +## Future Enhancements + +### 1. Tool Composition +- Agent can combine multiple tools +- Example: `searchTool` + `summarizeTool` = search and summarize + +### 2. Tool Learning +- Track which tools are used together +- Recommend tool combinations +- "Users who used X also used Y" + +### 3. Custom Tool Registry +- Users can add private tools +- Organization-specific tool registry +- Access control and permissions + +### 4. Tool Marketplace +- Developers promote their tools +- Usage analytics and ratings +- Paid/premium tools + +### 5. Agent Templates +- Pre-configured agents with tool sets +- "Research Agent" has search + summarize tools +- "Code Agent" has code generation tools + +--- + +## Conclusion + +This dynamic tool loading system represents a paradigm shift in how AI agents discover and use tools. By making the TPMJS registry itself searchable, we enable infinite extensibility without the limitations of static bundling. + +**Key Innovation:** Self-referential tool discovery - a tool that searches for tools. + +**Next Steps:** +1. Review this PRD with team +2. Validate technical feasibility with ChatGPT/Claude +3. Spike on BM25 search implementation +4. Spike on dynamic ESM import +5. Begin Phase 1 implementation + +**Success Looks Like:** +- User: "Search Wikipedia for quantum computing" +- Agent: *searches registry, finds Wikipedia tool, loads it, uses it* +- User: Gets Wikipedia results without any manual tool configuration + +This is a novel approach that could define how AI agents discover and use tools. Let's build it. 🚀 diff --git a/IMPLEMENTATION_STATUS.md b/IMPLEMENTATION_STATUS.md new file mode 100644 index 0000000..50bf6c1 --- /dev/null +++ b/IMPLEMENTATION_STATUS.md @@ -0,0 +1,220 @@ +# Dynamic Tool Loading - Implementation Status + +## ✅ Completed + +### 1. Search Tool Package (`@tpmjs/search-registry`) +- ✅ Created package with AI SDK v6 JSON Schema format +- ✅ Connects to search API endpoint +- ✅ Returns tool metadata (packageName, exportName, version, importUrl) +- ✅ Fixed schema format (was using Zod, now uses jsonSchema) +- ✅ Location: `packages/tools/search-registry/` + +### 2. Search API Endpoint (`/api/tools/search`) +- ✅ Implemented simple text-based search (BM25 had dependency issues) +- ✅ Searches by keywords in description, package name, keywords +- ✅ Returns tools with import URLs for esm.sh +- ✅ Location: `apps/web/src/app/api/tools/search/route.ts` + +### 3. Pre-flight Tool Loading in Playground +- ✅ Automatic search on every user message +- ✅ Extracts user query from last message +- ✅ Calls searchTpmjsTools automatically +- ✅ Attempts to load discovered tools dynamically +- ✅ Location: `apps/playground/src/app/api/chat/route.ts` + +### 4. Dynamic Tool Loader (Railway Service Approach) +- ✅ Updated to call Railway service instead of local imports +- ✅ Calls `/load-and-describe` endpoint to get tool schema +- ✅ Wraps tool with remote execution via `/execute-tool` endpoint +- ✅ Caches tool wrappers locally +- ✅ Location: `apps/playground/src/lib/dynamic-tool-loader.ts` + +### 5. Documentation +- ✅ DYNAMIC_IMPORT_ISSUE.md - Comprehensive problem analysis +- ✅ RAILWAY_DYNAMIC_TOOL_LOADER.md - Railway implementation guide +- ✅ This file - Implementation status + +## 🚧 Pending (Railway Service Implementation) + +### Railway Service Endpoints Needed + +You need to add these two endpoints to your existing Railway service: + +#### 1. `POST /load-and-describe` + +**Purpose**: Load a tool from esm.sh and return its schema + +**Request**: +```json +{ + "packageName": "firecrawl-aisdk", + "exportName": "webSearchTool", + "version": "0.7.2", + "importUrl": "https://esm.sh/firecrawl-aisdk@0.7.2" +} +``` + +**Response**: +```json +{ + "success": true, + "tool": { + "exportName": "webSearchTool", + "description": "Search the web using Firecrawl", + "inputSchema": { + "type": "object", + "properties": { + "query": { "type": "string" } + } + } + } +} +``` + +**Implementation Reference**: See `RAILWAY_DYNAMIC_TOOL_LOADER.md` for full code + +#### 2. `POST /execute-tool` + +**Purpose**: Execute a dynamically loaded tool with parameters + +**Request**: +```json +{ + "packageName": "firecrawl-aisdk", + "exportName": "webSearchTool", + "version": "0.7.2", + "importUrl": "https://esm.sh/firecrawl-aisdk@0.7.2", + "params": { + "query": "latest AI news" + } +} +``` + +**Response**: +```json +{ + "success": true, + "output": { "results": [...] }, + "executionTimeMs": 1234 +} +``` + +**Implementation Reference**: See `RAILWAY_DYNAMIC_TOOL_LOADER.md` for full code + +### Deployment Requirements + +1. **Railway Service**: + - Must run with `--experimental-network-imports` flag + - Add to start command: `node --experimental-network-imports server.js` + +2. **Environment Variables** (Vercel): + ```bash + RAILWAY_SERVICE_URL=https://your-railway-service.up.railway.app + # or reuse existing: + SANDBOX_EXECUTOR_URL=https://your-railway-service.up.railway.app + ``` + +3. **Local Testing** (Railway service on port 3001): + ```bash + RAILWAY_SERVICE_URL=http://localhost:3001 + ``` + +## đŸŽ¯ Testing Checklist + +Once Railway endpoints are deployed: + +- [ ] Test `/load-and-describe` endpoint directly with curl +- [ ] Test `/execute-tool` endpoint directly with curl +- [ ] Test full flow in playground: + - [ ] Ask: "search the web for latest AI news" + - [ ] Verify pre-flight search finds tools + - [ ] Verify tools load via Railway + - [ ] Verify tool execution works + - [ ] Check console logs for debugging info + +## 📊 Current Flow + +``` +User: "search the web for latest AI news" + ↓ + Chat API extracts query + ↓ + Automatically calls searchTpmjsTools + ↓ + Search API returns matching tools + (packageName, exportName, version) + ↓ + loadToolsBatch() called for each tool + ↓ + For each tool: + 1. Check local cache + 2. If not cached: + → POST to Railway: /load-and-describe + ← Get back: description + inputSchema + 3. Create wrapper tool with: + - description from Railway + - inputSchema from Railway + - execute() → calls Railway /execute-tool + 4. Cache wrapper locally + ↓ + All tools available to agent + ↓ + Agent calls tool (wrapper) + ↓ + Wrapper → POST to Railway: /execute-tool + ↓ + Railway imports from esm.sh and executes + ↓ + Result returned to agent + ↓ + Agent uses result to answer user +``` + +## 🔍 Debugging + +Check console logs for: +- `đŸ“Ļ Loading from Railway` - Tool loading initiated +- `✅ Tool loaded from Railway` - Tool schema received +- `🚀 Executing ... remotely` - Tool execution initiated +- `✅ Tool executed successfully` - Tool execution complete +- `❌ Railway service error` - Connection failed +- `❌ Failed to load tool` - Import failed + +## 📁 Files Modified + +1. `packages/tools/search-registry/src/index.ts` - Search tool +2. `packages/tools/search-registry/package.json` - AI SDK version +3. `apps/web/src/app/api/tools/search/route.ts` - Search endpoint +4. `apps/playground/src/app/api/chat/route.ts` - Pre-flight search +5. `apps/playground/src/lib/dynamic-tool-loader.ts` - Railway integration +6. `apps/playground/next.config.ts` - Added urlImports (unused) +7. `apps/playground/src/lib/tool-loader.ts` - Removed firecrawl + +## 🚀 Next Steps + +1. **Deploy Railway endpoints** using code from `RAILWAY_DYNAMIC_TOOL_LOADER.md` +2. **Set environment variables** in Vercel +3. **Test locally** with Railway service running on localhost:3001 +4. **Deploy to production** and test with real tools +5. **Monitor logs** for any issues + +## 💡 Key Insights + +- **Next.js Limitation**: Cannot do dynamic HTTP imports due to bundler +- **Railway Solution**: Plain Node.js with `--experimental-network-imports` +- **Caching Strategy**: Two-level cache (local wrapper + Railway module) +- **Execution Model**: Remote execution in Railway, not Next.js +- **Security**: Tools execute in Railway sandbox, not Vercel +- **Performance**: First load ~1-2s (import), cached loads <10ms + +## 📚 Related Documentation + +- `DYNAMIC_IMPORT_ISSUE.md` - Problem analysis and ChatGPT response +- `RAILWAY_DYNAMIC_TOOL_LOADER.md` - Full Railway implementation guide +- Plan file: `~/.claude/plans/jiggly-inventing-dragon.md` + +--- + +**Status**: Ready for Railway deployment +**Blocker**: Railway `/load-and-describe` and `/execute-tool` endpoints need implementation +**ETA**: 30-60 minutes to implement Railway endpoints + test diff --git a/RAILWAY_DEPLOYMENT_NOTE.md b/RAILWAY_DEPLOYMENT_NOTE.md new file mode 100644 index 0000000..2f192ce --- /dev/null +++ b/RAILWAY_DEPLOYMENT_NOTE.md @@ -0,0 +1,98 @@ +# Railway Executor - Deployment Status + +## Issue Discovered + +Node.js does not support HTTP(S) imports by default, even with `--experimental-network-imports` flag (that flag doesn't exist in current Node versions). + +## Solutions Considered + +1. **Custom ESM Loader** - Complex, requires Node.js 18.19+ with `--loader` flag +2. **fetch + eval** - Security concerns, doesn't handle ES modules properly +3. **Bundler approach** - Would defeat the purpose of dynamic imports +4. **Deno** - Supports HTTP imports natively, but different ecosystem + +## Recommended Solution + +Since the core issue is that we need truly dynamic runtime imports from HTTP URLs, and Node.js doesn't support this, we have **two viable paths**: + +### Option A: Use Deno on Railway (RECOMMENDED) + +Deno supports HTTP imports natively: + +```typescript +// server.ts (Deno) +import { serve } from "https://deno.land/std@0.208.0/http/server.ts"; + +const moduleCache = new Map(); + +async function loadTool(url: string, exportName: string) { + if (moduleCache.has(url)) { + return moduleCache.get(url); + } + + // Deno supports this natively! + const module = await import(url); + const tool = module[exportName]; + moduleCache.set(url, tool); + return tool; +} + +serve(async (req) => { + // ... handle requests +}, { port: 3002 }); +``` + +**Deploy to Railway:** +```bash +# In Railway dashboard: +# - Set Start Command: deno run --allow-net --allow-env server.ts +# - Or use railway.json with deno runtime +``` + +### Option B: Pre-build Bundle Approach + +Instead of truly dynamic imports, pre-fetch and cache tools: + +1. Playground searches for tools +2. Backend fetches tool code once and caches it +3. Use `vm2` or similar to execute in sandbox +4. Not truly "dynamic" but works with Node.js + +## Current Status + +The Railway executor service is **created** but **not deployed** because Node.js doesn't support the required HTTP imports. + +**Files created:** +- `apps/railway-executor/package.json` +- `apps/railway-executor/server.js` (incomplete - needs Deno or vm2 approach) +- `apps/railway-executor/README.md` + +## Next Steps + +**If using Deno (recommended):** +1. Rewrite server.js as server.ts for Deno +2. Deploy to Railway with Deno runtime +3. Test HTTP imports work +4. Update playground to use Railway URL + +**If sticking with Node.js:** +1. Install `vm2` package for sandboxed execution +2. Implement fetch + vm2 approach +3. Deploy to Railway +4. Accept limitations (less dynamic, more complex) + +## Alternative: Skip Railway, Use Different Architecture + +Since the original issue is Next.js bundler limitations, consider: + +**Web Workers in Browser** - Load tools client-side using native `import()` +- Pros: No server needed, truly dynamic +- Cons: Exposes API keys, security concerns + +**Serverless Functions with Pre-installed Tools** - Deploy each tool as separate function +- Pros: Works with Vercel/Next.js +- Cons: Not truly dynamic, requires redeployment for new tools + +--- + +**Recommendation**: Use Deno on Railway. It's designed for exactly this use case. diff --git a/RAILWAY_DYNAMIC_TOOL_LOADER.md b/RAILWAY_DYNAMIC_TOOL_LOADER.md new file mode 100644 index 0000000..e77b66a --- /dev/null +++ b/RAILWAY_DYNAMIC_TOOL_LOADER.md @@ -0,0 +1,376 @@ +# Railway Service - Dynamic Tool Loader Implementation + +## Overview + +This document describes the Railway service implementation needed to support dynamic tool loading from esm.sh in the TPMJS playground. + +## Why Railway Service? + +Next.js/Turbopack intercepts all `import()` calls and tries to resolve them through its module graph. HTTP URLs like `https://esm.sh/...` are not supported. + +**Solution**: Use a plain Node.js service on Railway that: +- Runs with `--experimental-network-imports` flag +- Can dynamically import from HTTP URLs (esm.sh) +- Executes tool functions and returns results +- Is already set up for existing ToolPlayground + +## New Endpoint Required + +### `POST /load-and-describe` + +**Purpose**: Dynamically import a tool package and return its AI SDK tool definition (description, schema) without executing it. + +**Request**: +```json +{ + "packageName": "firecrawl-aisdk", + "exportName": "webSearchTool", + "version": "0.7.2", + "importUrl": "https://esm.sh/firecrawl-aisdk@0.7.2" +} +``` + +**Response**: +```json +{ + "success": true, + "tool": { + "exportName": "webSearchTool", + "description": "Search the web using Firecrawl", + "inputSchema": { + "type": "object", + "properties": { + "query": { "type": "string", "description": "Search query" } + }, + "required": ["query"] + } + } +} +``` + +**Implementation** (pseudo-code for Railway service): + +```javascript +// server.js (Railway service) +import express from 'express'; + +const app = express(); +app.use(express.json()); + +// Cache for imported modules +const moduleCache = new Map(); + +app.post('/load-and-describe', async (req, res) => { + const { packageName, exportName, version, importUrl } = req.body; + + const cacheKey = `${packageName}::${exportName}`; + + try { + let toolModule; + + // Check cache first + if (moduleCache.has(cacheKey)) { + console.log(`✅ Cache hit: ${cacheKey}`); + toolModule = moduleCache.get(cacheKey); + } else { + // Dynamic import from esm.sh + const url = importUrl || `https://esm.sh/${packageName}@${version}`; + console.log(`đŸ“Ļ Importing: ${url}`); + + const module = await import(url); + toolModule = module[exportName]; + + if (!toolModule) { + return res.status(404).json({ + success: false, + error: `Export "${exportName}" not found in module` + }); + } + + // Validate it's an AI SDK tool + if (!toolModule.description || !toolModule.execute) { + return res.status(400).json({ + success: false, + error: `Invalid AI SDK tool structure` + }); + } + + // Cache it + moduleCache.set(cacheKey, toolModule); + } + + // Extract tool definition (description + schema) + // AI SDK v6 tools have: description, inputSchema, execute + res.json({ + success: true, + tool: { + exportName, + description: toolModule.description, + inputSchema: toolModule.inputSchema || toolModule.parameters?.shape || {}, + } + }); + } catch (error) { + console.error('Failed to load tool:', error); + res.status(500).json({ + success: false, + error: error.message + }); + } +}); + +// Start server +const PORT = process.env.PORT || 3000; +app.listen(PORT, () => { + console.log(`Railway tool loader running on port ${PORT}`); +}); +``` + +**Railway Deployment**: +```bash +# Start command in Railway settings: +node --experimental-network-imports server.js + +# Or in package.json: +{ + "scripts": { + "start": "node --experimental-network-imports server.js" + } +} +``` + +## Modified Endpoint: `POST /execute-tool` + +**Purpose**: Execute a dynamically loaded tool with parameters. + +**Request**: +```json +{ + "packageName": "firecrawl-aisdk", + "exportName": "webSearchTool", + "version": "0.7.2", + "importUrl": "https://esm.sh/firecrawl-aisdk@0.7.2", + "params": { + "query": "latest AI news" + } +} +``` + +**Response**: +```json +{ + "success": true, + "output": { + "results": [...] + }, + "executionTimeMs": 1234 +} +``` + +**Implementation** (pseudo-code): + +```javascript +app.post('/execute-tool', async (req, res) => { + const { packageName, exportName, version, importUrl, params } = req.body; + + const cacheKey = `${packageName}::${exportName}`; + const startTime = Date.now(); + + try { + let toolModule; + + // Check cache or import + if (moduleCache.has(cacheKey)) { + toolModule = moduleCache.get(cacheKey); + } else { + const url = importUrl || `https://esm.sh/${packageName}@${version}`; + const module = await import(url); + toolModule = module[exportName]; + + if (!toolModule || !toolModule.execute) { + return res.status(404).json({ + success: false, + error: 'Tool not found or invalid' + }); + } + + moduleCache.set(cacheKey, toolModule); + } + + // Execute the tool + const result = await toolModule.execute(params); + + res.json({ + success: true, + output: result, + executionTimeMs: Date.now() - startTime + }); + } catch (error) { + res.status(500).json({ + success: false, + error: error.message, + executionTimeMs: Date.now() - startTime + }); + } +}); +``` + +## Integration with Playground + +### 1. Update `dynamic-tool-loader.ts` + +Replace local dynamic imports with Railway service calls: + +```typescript +// apps/playground/src/lib/dynamic-tool-loader.ts + +const RAILWAY_SERVICE_URL = process.env.RAILWAY_SERVICE_URL || 'http://localhost:3001'; + +export async function loadToolDynamically( + packageName: string, + exportName: string, + version: string, + importUrl?: string +): Promise { + const cacheKey = getCacheKey(packageName, exportName); + + // Check local cache first + if (moduleCache.has(cacheKey)) { + console.log(`✅ Cache hit: ${cacheKey}`); + return moduleCache.get(cacheKey); + } + + try { + console.log(`đŸ“Ļ Loading from Railway: ${packageName}/${exportName}`); + + // Call Railway service to load and describe tool + const response = await fetch(`${RAILWAY_SERVICE_URL}/load-and-describe`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + packageName, + exportName, + version, + importUrl, + }), + }); + + if (!response.ok) { + console.error(`❌ Railway service error: ${response.status}`); + return null; + } + + const data = await response.json(); + + if (!data.success) { + console.error(`❌ Failed to load tool: ${data.error}`); + return null; + } + + // Create a tool wrapper that executes remotely + const tool = { + description: data.tool.description, + inputSchema: data.tool.inputSchema, + execute: async (params: any) => { + console.log(`🚀 Executing ${packageName}/${exportName} remotely`); + + const execResponse = await fetch(`${RAILWAY_SERVICE_URL}/execute-tool`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + packageName, + exportName, + version, + importUrl, + params, + }), + }); + + const result = await execResponse.json(); + + if (!result.success) { + throw new Error(result.error || 'Tool execution failed'); + } + + return result.output; + }, + }; + + // Cache the wrapper + moduleCache.set(cacheKey, tool); + console.log(`✅ Loaded and cached: ${cacheKey}`); + + return tool; + } catch (error) { + console.error(`❌ Failed to load ${packageName}#${exportName}:`, error); + return null; + } +} +``` + +### 2. Environment Variables + +Add to `.env.local`: +```bash +RAILWAY_SERVICE_URL=https://your-railway-service.up.railway.app +``` + +Or for local testing with Railway running locally: +```bash +RAILWAY_SERVICE_URL=http://localhost:3001 +``` + +## Testing Locally + +### Terminal 1: Run Railway service locally +```bash +cd railway-service +node --experimental-network-imports server.js +``` + +### Terminal 2: Run playground +```bash +cd tpmjs +pnpm dev --filter=@tpmjs/playground +``` + +### Test the flow: +```bash +# Test Railway service directly +curl -X POST http://localhost:3001/load-and-describe \ + -H "Content-Type: application/json" \ + -d '{ + "packageName": "firecrawl-aisdk", + "exportName": "webSearchTool", + "version": "0.7.2" + }' + +# Then test via playground UI +# Navigate to http://localhost:3000/playground +# Ask: "search the web for latest AI news" +``` + +## Deployment Checklist + +- [ ] Create Railway service with Node.js +- [ ] Add `--experimental-network-imports` flag to start command +- [ ] Deploy `/load-and-describe` endpoint +- [ ] Deploy `/execute-tool` endpoint (or modify existing `/execute`) +- [ ] Set `RAILWAY_SERVICE_URL` in Vercel environment variables +- [ ] Test with real tools from TPMJS registry +- [ ] Monitor Railway logs for import errors + +## Benefits + +1. ✅ **Works around Next.js limitations** - Imports happen in plain Node +2. ✅ **Reuses existing Railway infrastructure** - No new service needed +3. ✅ **Caching on both sides** - Local cache + Railway cache +4. ✅ **Security** - Tools execute in Railway sandbox, not Next.js +5. ✅ **Scalability** - Railway handles the heavy lifting + +## Next Steps + +1. Implement Railway service endpoints +2. Update `dynamic-tool-loader.ts` to use Railway +3. Test locally +4. Deploy to Railway + Vercel +5. Celebrate dynamic tool loading! 🎉 diff --git a/ai-sdk-v6.md b/ai-sdk-v6.md new file mode 100644 index 0000000..37b1572 --- /dev/null +++ b/ai-sdk-v6.md @@ -0,0 +1,31867 @@ +--- +title: RAG Agent +description: Learn how to build a RAG Agent with the AI SDK and Next.js +tags: + [ + 'rag', + 'chatbot', + 'next', + 'embeddings', + 'database', + 'retrieval', + 'memory', + 'agent', + ] +--- + +# RAG Agent Guide + +In this guide, you will learn how to build a retrieval-augmented generation (RAG) agent. + +