feat: implement multi-tool package architecture with manual tool registry

BREAKING CHANGE: Complete refactoring from single-tool to multi-tool package support

Database Schema:
- Split Tool model into Package (1) and Tool (many) with one-to-many relationship
- Package stores npm metadata and package-level tpmjs fields (category, env, frameworks, tier)
- Tool stores individual tool exports with tool-level metadata (exportName, description, parameters, returns, aiAgent)
- Unique constraint on (packageId, exportName) to prevent duplicate tools
- Cascade deletes when packages are removed

Type System:
- Updated tpmjs field schema to support tools array
- Each tool has exportName, description, parameters, returns, aiAgent
- Package-level fields: category, env, frameworks shared across all tools
- Backward compatible with legacy single-tool format (auto-migrates to exportName: "default")

API Updates:
- Updated all /api/tools routes to query Tool model with Package relations
- Updated /api/tools/[slug] to accept package/export path segments
- Updated tool-executor-agent to use actual exportName instead of hardcoded "default"
- Updated metrics sync to calculate quality scores per Tool

Frontend Updates:
- Updated tool search page to display exportName as primary heading
- Updated tool detail pages to show package name as secondary info
- Removed tag-based filtering (tags moved to package level)

Manual Tool Registry:
- Added manual-tools.ts with 23 curated tools from major providers
- Created sync-manual-tools.ts script to sync manual tools to database
- Added MANUAL_TOOLS.md documentation for manual tool system
- Added GitHub workflow for automated daily sync
- Includes tools from: Vercel, Exa, Firecrawl, AWS Bedrock, Perplexity, Tavily, Superagent, Valyu

Playground Updates:
- Updated tool loader to load multiple tools per package
- Added sanitizeToolName for OpenAI API compatibility

Sync System Updates:
- Updated changes feed sync to handle multi-tool packages
- Updated keyword sync to upsert multiple tools per package
- Added orphaned tool deletion when tools removed from package.json

Migration Strategy:
- Database uses same Neon instance for dev and prod
- Schema updated via prisma db push (no migration files yet)
- All data repopulates from npm via sync system

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Ajax Davis 2025-12-04 06:39:58 +10:00
parent fdd1b2c304
commit 0612eac5e2
31 changed files with 3783 additions and 786 deletions

88
test-schema.ts Normal file
View file

@ -0,0 +1,88 @@
import { PrismaClient } from '@prisma/client';
const prisma = new PrismaClient();
async function testSchema() {
console.log('Testing new Package + Tool schema...\n');
// Test 1: Create a package
console.log('1. Creating test package...');
const pkg = await prisma.package.create({
data: {
npmPackageName: '@test/hello',
npmVersion: '1.0.0',
npmPublishedAt: new Date(),
category: 'text-analysis',
tier: 'rich',
discoveryMethod: 'test',
isOfficial: false,
frameworks: ['vercel-ai'],
npmDownloadsLastMonth: 100,
},
});
console.log(`✅ Package created: ${pkg.npmPackageName} (${pkg.id})\n`);
// Test 2: Create multiple tools for the package
console.log('2. Creating tools for package...');
const tool1 = await prisma.tool.create({
data: {
packageId: pkg.id,
exportName: 'helloWorldTool',
description: 'Returns a simple Hello World greeting',
},
});
console.log(`✅ Tool 1 created: ${tool1.exportName}`);
const tool2 = await prisma.tool.create({
data: {
packageId: pkg.id,
exportName: 'helloNameTool',
description: 'Returns a personalized greeting with name',
},
});
console.log(`✅ Tool 2 created: ${tool2.exportName}\n`);
// Test 3: Query package with tools
console.log('3. Querying package with tools...');
const packageWithTools = await prisma.package.findUnique({
where: { npmPackageName: '@test/hello' },
include: { tools: true },
});
console.log(`✅ Found package with ${packageWithTools?.tools.length} tools:`);
packageWithTools?.tools.forEach((t) => {
console.log(` - ${t.exportName}: ${t.description}`);
});
console.log();
// Test 4: Query tool with package
console.log('4. Querying tool with package...');
const toolWithPackage = await prisma.tool.findFirst({
where: {
package: { npmPackageName: '@test/hello' },
exportName: 'helloWorldTool',
},
include: { package: true },
});
console.log(`✅ Found tool: ${toolWithPackage?.exportName}`);
console.log(` Package: ${toolWithPackage?.package.npmPackageName}`);
console.log(` Category: ${toolWithPackage?.package.category}\n`);
// Test 5: Delete package (should cascade to tools)
console.log('5. Testing cascade delete...');
await prisma.package.delete({
where: { id: pkg.id },
});
const remainingTools = await prisma.tool.count({
where: { packageId: pkg.id },
});
console.log(`✅ Package deleted, remaining tools: ${remainingTools}`);
console.log(` (Should be 0 due to cascade delete)\n`);
console.log('✅ All schema tests passed!');
}
testSchema()
.then(() => process.exit(0))
.catch((error) => {
console.error('❌ Test failed:', error);
process.exit(1);
});