Commit graph

6 commits

Author SHA1 Message Date
Ajax Davis
ada3ffb341 refactor: rename exportName to name across entire codebase
- Database: Migrate column export_name to name in tools table
- Prisma schema: Update Tool model to use name field
- Sync routes: Update keyword and changes sync to use name
- Railway executor: Update API endpoints to use name parameter
- API routes: Update all tool routes to use name field
- Web app: Update all pages and components
- Playground: Update tool loader and sidebar
- create-basic-tools: Update types and generators
- Scripts: Update sync and test scripts

Database migration was done via direct SQL:
  ALTER TABLE tools RENAME COLUMN export_name TO name;

The unique constraint remains on (package_id, name).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-17 17:19:21 +10:00
Ajax Davis
10792991ea refactor: rename exportName to name in ManualTool interface
- Update ManualTool interface to use `name` instead of `exportName`
- Update all tool definitions in manual-tools.ts
- Update sync-manual-tools.ts to use manualTool.name
- Update sync-vercel-registry.ts OpenAI prompt and output generation
- Database column `exportName` remains unchanged (stores the value)
2025-12-17 16:43:15 +10:00
github-actions[bot]
cad30ab931 chore: sync 1 new tools from Vercel AI registry
Added 1 tools from Vercel AI SDK registry:
- Total tools in registry: 11
- Already synced: 10
- Newly added: 1
- Errors: 0

🤖 Automated by GitHub Actions
Run: https://github.com/tpmjs/tpmjs/actions/runs/20293730295
2025-12-17 06:23:14 +00:00
Ajax Davis
d3b928c97e fix: use Dockerfile instead of Nixpacks for Railway deployment
Railway is deprecating Nixpacks, so switching to a standard Dockerfile
with the official Deno image. This provides a cleaner and more maintainable
deployment configuration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 08:35:50 +10:00
Ajax Davis
2158ee6dfd feat: add dynamic tool loading system with Railway executor
Implements a complete dynamic tool loading system that allows the playground to discover and load tools from the TPMJS registry at runtime.

**Architecture:**
- Search tool package (@tpmjs/search-registry) - Searches registry for tools
- Search API endpoint (/api/tools/search) - Text-based search with scoring
- Pre-flight tool loading - Automatically searches and loads tools on every message
- Railway executor service (Deno) - Loads tools from esm.sh via HTTP imports
- Dynamic tool loader - Calls Railway to load and execute tools remotely

**Key Components:**

1. Railway Executor (apps/railway-executor/)
   - Deno-based service that natively supports HTTP imports
   - Endpoints: /load-and-describe, /execute-tool, /cache/stats, /cache/clear
   - Deploys to Railway with deno run --allow-net --allow-env server.ts

2. Search Tool Package (packages/tools/search-registry/)
   - AI SDK v6 tool for searching TPMJS registry
   - Uses jsonSchema + inputSchema pattern
   - Searches /api/tools/search endpoint

3. Search API (apps/web/src/app/api/tools/search/)
   - Text-based search with composite scoring
   - Scores: text relevance + quality boost + download boost
   - Returns tool metadata with importUrl for dynamic loading

4. Dynamic Tool Loader (apps/playground/src/lib/dynamic-tool-loader.ts)
   - Calls Railway service to load tools from esm.sh
   - Creates tool wrappers that execute remotely
   - Process-level module cache + per-conversation tracking

5. Pre-flight Loading (apps/playground/src/app/api/chat/route.ts)
   - Automatically searches for tools on every user message
   - Loads top 5 matching tools before agent processes request
   - Merges with static tools for seamless experience

**Technical Decisions:**
- Deno over Node.js: Native HTTP import support without flags
- Remote execution: Tools run in Railway sandbox, not Vercel
- Pre-flight loading: Better UX than two-turn search pattern
- Text search: BM25 had dependency issues, simple scoring works well

**Environment Variables:**
- RAILWAY_SERVICE_URL: https://endearing-commitment-production.up.railway.app

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 08:25:53 +10:00
Ajax Davis
0612eac5e2 feat: implement multi-tool package architecture with manual tool registry
BREAKING CHANGE: Complete refactoring from single-tool to multi-tool package support

Database Schema:
- Split Tool model into Package (1) and Tool (many) with one-to-many relationship
- Package stores npm metadata and package-level tpmjs fields (category, env, frameworks, tier)
- Tool stores individual tool exports with tool-level metadata (exportName, description, parameters, returns, aiAgent)
- Unique constraint on (packageId, exportName) to prevent duplicate tools
- Cascade deletes when packages are removed

Type System:
- Updated tpmjs field schema to support tools array
- Each tool has exportName, description, parameters, returns, aiAgent
- Package-level fields: category, env, frameworks shared across all tools
- Backward compatible with legacy single-tool format (auto-migrates to exportName: "default")

API Updates:
- Updated all /api/tools routes to query Tool model with Package relations
- Updated /api/tools/[slug] to accept package/export path segments
- Updated tool-executor-agent to use actual exportName instead of hardcoded "default"
- Updated metrics sync to calculate quality scores per Tool

Frontend Updates:
- Updated tool search page to display exportName as primary heading
- Updated tool detail pages to show package name as secondary info
- Removed tag-based filtering (tags moved to package level)

Manual Tool Registry:
- Added manual-tools.ts with 23 curated tools from major providers
- Created sync-manual-tools.ts script to sync manual tools to database
- Added MANUAL_TOOLS.md documentation for manual tool system
- Added GitHub workflow for automated daily sync
- Includes tools from: Vercel, Exa, Firecrawl, AWS Bedrock, Perplexity, Tavily, Superagent, Valyu

Playground Updates:
- Updated tool loader to load multiple tools per package
- Added sanitizeToolName for OpenAI API compatibility

Sync System Updates:
- Updated changes feed sync to handle multi-tool packages
- Updated keyword sync to upsert multiple tools per package
- Added orphaned tool deletion when tools removed from package.json

Migration Strategy:
- Database uses same Neon instance for dev and prod
- Schema updated via prisma db push (no migration files yet)
- All data repopulates from npm via sync system

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 06:39:58 +10:00