diff --git a/TPMJS_TALK.md b/TPMJS_TALK.md new file mode 100644 index 0000000..99142bd --- /dev/null +++ b/TPMJS_TALK.md @@ -0,0 +1,391 @@ +# TPMJS: The Missing Layer Between "LLMs Can Call Tools" and "Which Tool, Exactly?" + +--- + +## The Setup + +You're building an AI agent. It needs to do things in the world—scrape a webpage, send an email, query a database, generate an image. These capabilities come from **tools**. + +The problem isn't that tools don't exist. They do. Thousands of them. The problem is: + +- **You can't find them.** npm has 2 million packages. Which ones are AI-callable tools? Which ones actually work? +- **You can't trust them.** No schema. No examples. README says "AI-ready" but the function signature is `(opts: any) => Promise`. +- **You can't compare them.** Three packages do "web scraping." Which one handles JavaScript rendering? Which one returns structured data? Which one is maintained? + +Discovery is the bottleneck. Not capability—discovery. + +--- + +## What TPMJS Actually Is + +TPMJS is infrastructure. Specifically: + +1. **A registry** that indexes npm packages designed for AI tool use +2. **A metadata extraction pipeline** that pulls schemas directly from code +3. **A quality scoring system** that ranks tools by completeness and adoption +4. **A health monitoring system** that verifies tools actually work +5. **A playground** where you can test tools before integrating them + +It's not magic. It's plumbing. Good plumbing. + +--- + +## How It Works (The Technical Reality) + +### Discovery: Finding Tools in the Wild + +TPMJS runs three automated sync jobs: + +**1. npm Changes Feed (every 2 minutes)** +``` +npm registry → /_changes endpoint → filter for tpmjs-tool keyword → process +``` +This catches new packages and updates in near-real-time. We track sequence numbers so we never reprocess. + +**2. Keyword Search (every 15 minutes)** +``` +npm search "tpmjs-tool" → up to 250 results → validate → ingest +``` +Backup mechanism. Catches anything the changes feed missed. + +**3. Metrics Sync (hourly)** +``` +for each package → fetch download stats → recalculate quality scores → update health status +``` +Keeps the registry fresh. + +### The Publisher Contract + +To get indexed, a package needs two things: + +```json +{ + "name": "@acme/my-tool", + "keywords": ["tpmjs-tool"], + "tpmjs": { + "category": "web-scraping", + "description": "Scrapes URLs and returns structured markdown" + } +} +``` + +That's the minimum. Category + description. Everything else is either optional or auto-extracted. + +**Categories are fixed** (12 total): web-scraping, data-processing, file-operations, communication, database, api-integration, image-processing, text-analysis, automation, ai-ml, security, monitoring. + +Why fixed? Because agents need to filter. "Give me all database tools" has to mean something. + +### Schema Extraction: The Hard Part + +Here's what makes TPMJS different from a glorified npm search. + +When we ingest a package, we don't just read the README. We **execute it in a sandbox** and extract the actual schema: + +``` +1. Spin up isolated executor (Railway) +2. npm install the package +3. Import and inspect exports +4. Extract JSON Schema from TypeScript types +5. Store schema in database +``` + +The result: + +```json +{ + "name": "scrapeUrl", + "inputSchema": { + "type": "object", + "properties": { + "url": { "type": "string", "format": "uri" }, + "waitForSelector": { "type": "string" }, + "timeout": { "type": "number", "default": 30000 } + }, + "required": ["url"] + } +} +``` + +This isn't documentation. This is **extracted from the actual function signature**. It's ground truth. + +If the author provides a schema in the `tpmjs` field, we use that. If not, we extract it. Either way, every tool in the registry has a schema. + +### Quality Scoring: Ranking What Matters + +Every tool gets a score from 0.00 to 1.00: + +```typescript +// Base score from metadata completeness +const tierScore = tier === 'rich' ? 0.6 : 0.4; + +// Adoption signals +const downloadsScore = Math.min(0.2, Math.log10(downloads + 1) / 15); +const starsScore = Math.min(0.1, Math.log10(githubStars + 1) / 10); + +// Metadata richness bonus +let richnessScore = 0; +if (hasParameters) richnessScore += 0.04; +if (hasReturns) richnessScore += 0.03; +if (hasEnvVars) richnessScore += 0.03; +``` + +**Tier** is binary: +- **Minimal**: Just category + description (40% base) +- **Rich**: Has parameters, returns, env vars, or framework tags (60% base) + +The formula is deliberately simple. We're not trying to be clever. We're trying to surface tools that are well-documented and actually used. + +### Health Checks: Does It Actually Work? + +Two checks, run during sync and periodically: + +**1. Import Health** +``` +Can we require() this package without it exploding? +``` +You'd be surprised how many npm packages fail this. + +**2. Execution Health** +``` +Can we call the main function with minimal parameters without throwing? +``` +Not a full test suite. Just "does it run at all?" + +Results: `HEALTHY`, `BROKEN`, or `UNKNOWN`. + +Broken tools still appear in the registry (with a warning). We don't hide them—we label them. + +--- + +## The Data Model + +Here's what we actually store: + +### Package (npm package level) +``` +npmPackageName (unique) +npmVersion, npmDescription, npmRepository, npmLicense +npmKeywords[], npmReadme, npmAuthor +category (enum) +tier ('minimal' | 'rich') +discoveryMethod ('changes-feed' | 'keyword') +npmDownloadsLastMonth, githubStars +frameworks[] (vercel-ai, langchain, etc.) +env[] (required environment variables) +``` + +### Tool (individual callable within a package) +``` +packageId (FK) +name (export name: "scrapeUrl", "default", etc.) +description +inputSchema (JSON Schema) +schemaSource ('extracted' | 'author') +qualityScore (0.00-1.00) +importHealth, executionHealth (HEALTHY | BROKEN | UNKNOWN) +toolDiscoverySource ('auto' | 'manual') +``` + +One package can have multiple tools. `@acme/web-tools` might export `scrapeUrl`, `screenshotPage`, and `extractLinks`. Each is a separate tool with its own schema and health status. + +### Simulation (playground execution) +``` +toolId +userPrompt (what the user asked) +parameters (JSON, what was passed to the tool) +status (pending | running | success | error | timeout) +executionTimeMs, output, error +model, agentSteps +``` + +We track every playground execution. Not for surveillance—for debugging and improving the system. + +--- + +## The API + +### Search & Discovery + +``` +GET /api/tools + ?q=scrape + &category=web-scraping + &importHealth=HEALTHY + &executionHealth=HEALTHY + &limit=20 + &offset=0 + +→ Returns tools sorted by quality score +``` + +``` +GET /api/tools/search + ?q=I need to extract text from PDFs + +→ BM25-ranked semantic search +``` + +### Execution + +``` +POST /api/tools/execute/{toolId} +{ + "prompt": "Scrape the homepage of Hacker News", + "parameters": { "url": "https://news.ycombinator.com" } +} + +→ Server-Sent Events stream with: + - Agent reasoning steps + - Tool call results + - Final output +``` + +Rate limited: 10 requests per IP per hour. We're not a free compute platform. + +### Schema Operations + +``` +POST /api/tools/extract-schema +{ "packageName": "@acme/my-tool", "toolName": "scrapeUrl" } + +→ Forces re-extraction of schema from source +``` + +--- + +## The Playground + +A Next.js app where you can: + +1. **Browse tools** by category, health status, quality score +2. **Inspect schemas** before you commit to anything +3. **Test execution** with an AI agent +4. **See real responses** with actual latency and token usage + +It's not a demo. It's a debugging tool. "Does this tool do what I think it does?" Answer that question in 30 seconds instead of 30 minutes. + +--- + +## What This Enables + +### For Engineers Building Agents + +Before TPMJS: +``` +1. Search npm for "web scraper" +2. Get 500 results +3. Click through 20 of them +4. Read READMEs that say "easy to use!" +5. npm install three of them +6. Write test code for each +7. Find out two are broken +8. Pick the one that works +9. Hope it keeps working +``` + +After TPMJS: +``` +1. Search tpmjs.com for "web scraper" +2. Filter by HEALTHY status +3. Sort by quality score +4. Click top result +5. See exact input schema +6. Test in playground +7. Integrate +``` + +### For Tool Authors + +Before TPMJS: +``` +Publish to npm → hope someone finds it → no visibility into usage +``` + +After TPMJS: +``` +Publish to npm with tpmjs-tool keyword → indexed within 2 minutes → +schema auto-extracted → quality scored → discoverable by search → +execution stats tracked +``` + +Your tool becomes findable. Not just by humans grepping npm, but by agents querying the registry API. + +### For Agents (Yes, Really) + +Agents can query TPMJS at runtime: + +```typescript +const tools = await fetch('https://tpmjs.com/api/tools?' + new URLSearchParams({ + q: 'send email', + executionHealth: 'HEALTHY', + limit: '5' +})).then(r => r.json()); + +// Agent now has 5 working email tools with full schemas +// It can pick the best one for this specific task +``` + +This is the endgame. Not humans browsing a registry—agents dynamically selecting tools based on capability, health, and fit. + +--- + +## What TPMJS Is Not + +**Not a package manager.** We don't host packages. npm does that. We index and enrich. + +**Not an execution platform.** The playground runs tools for testing. Production execution is your responsibility. + +**Not a security guarantee.** We check if tools work. We don't audit them for malice. Same rules as npm: don't run untrusted code. + +**Not magic.** We're not using AI to understand what tools do. We're extracting schemas and running health checks. Boring, reliable, debuggable. + +--- + +## The Technical Stack + +- **Database**: PostgreSQL via Prisma +- **Web**: Next.js 16 (App Router) +- **Deployment**: Vercel (web) + Railway (sandbox executor) +- **Sync**: Vercel Cron + GitHub Actions backup +- **AI**: Vercel AI SDK for playground execution +- **Monorepo**: Turborepo + pnpm + +Key internal packages: +- `@tpmjs/npm-client` — npm registry integration +- `@tpmjs/package-executor` — sandbox execution client +- `@tpmjs/types` — schema validation and migration +- `@tpmjs/db` — Prisma client and models + +--- + +## Current State + +- **~100 tools indexed** (and growing with every npm publish) +- **12 categories** covering most agent use cases +- **Automated sync** running 24/7 +- **Health checks** on every tool +- **Schema extraction** working for TypeScript and JavaScript +- **Playground** functional for testing + +--- + +## The Pitch (Finally) + +Tools are the API surface of AI agents. The ecosystem is a mess. TPMJS is the index. + +We don't compete with npm—we sit on top of it. We don't replace tool authors—we make them discoverable. We don't build agents—we give agents a way to find their tools. + +Discovery is the bottleneck. We're fixing discovery. + +--- + +## Try It + +- **Browse**: https://tpmjs.com/tool-search +- **Playground**: https://tpmjs.com/playground +- **Publish**: Add `tpmjs-tool` keyword + `tpmjs` field to your package.json +- **API**: `GET https://tpmjs.com/api/tools` + +--- + +*Tools are inevitable. Discovery chaos isn't.* diff --git a/apps/tutorial/src/components/slides/DiscoverySlide.tsx b/apps/tutorial/src/components/slides/DiscoverySlide.tsx index 3033c6b..dd035ca 100644 --- a/apps/tutorial/src/components/slides/DiscoverySlide.tsx +++ b/apps/tutorial/src/components/slides/DiscoverySlide.tsx @@ -24,55 +24,54 @@ export function DiscoverySlide(): React.ReactElement { transition={{ type: 'spring', stiffness: 200, delay: 0.2 }} className="text-6xl mb-8" > - 🔍 + ⚙️ -

Two Users, One Surface

+

Schema Extraction

- Same registry. Different clients. + The hard part.{' '} + We run your code in a sandbox.

- {/* Two user types */} -
- -
👨‍💻
-
Engineers browsing
-
    -
  • "Show me the best extraction tools"
  • -
  • "I need one that supports screenshots"
  • -
  • "I need maintained + documented"
  • -
-
- - -
🤖
-
Agents selecting
-
    -
  • "I need to scrape a URL into markdown"
  • -
  • "I need to summarize a PDF with citations"
  • -
  • "I need a company research tool"
  • -
-
-
+ {/* Code block showing extraction */} + +
+ {/* comment */} + {'// We extract this from your actual code:'} +
+
inputSchema: {'{'}
+
+ type: "object", +
+
properties: {'{'}
+
+ url: {'{'} type: "string", format:{' '} + "uri" {'}'} +
+
+ timeout: {'{'} type: "number", + default: 30000 {'}'} +
+
{'}'}
+
{'}'}
+
{/* Key insight */} - Results are tool-shaped, not just package-shaped. + Not documentation.{' '} + + Ground truth from TypeScript types. + diff --git a/apps/tutorial/src/components/slides/GetStartedSlide.tsx b/apps/tutorial/src/components/slides/GetStartedSlide.tsx index b482b0f..49844ca 100644 --- a/apps/tutorial/src/components/slides/GetStartedSlide.tsx +++ b/apps/tutorial/src/components/slides/GetStartedSlide.tsx @@ -6,21 +6,21 @@ const links = [ { label: 'Browse Tools', href: 'https://tpmjs.com/tool-search', - description: 'Use as a discovery portal', + description: 'Search by name, category, quality', icon: '🔍', gradient: 'from-cyan-500 to-blue-500', }, { label: 'Publish a Tool', href: 'https://tpmjs.com/docs/publishing', - description: 'Clean metadata + examples', + description: 'Add tpmjs-tool keyword to npm', icon: '📦', gradient: 'from-purple-500 to-pink-500', }, { label: 'The Playground', href: 'https://tpmjs.com/playground', - description: 'Try tools before adopting', + description: 'Run tools in browser sandbox', icon: '🎮', gradient: 'from-emerald-500 to-teal-500', }, @@ -61,8 +61,10 @@ export function GetStartedSlide(): React.ReactElement { 🚀 -

Try It

-

Discovery chaos is optional

+

Get Started

+

+ Indexed automatically. Updated every 2 minutes. +

{/* CTA buttons */}
@@ -100,7 +102,13 @@ export function GetStartedSlide(): React.ReactElement { initial={{ x: -10 }} whileHover={{ x: 0 }} > - +
diff --git a/apps/tutorial/src/components/slides/HowItWorksSlide.tsx b/apps/tutorial/src/components/slides/HowItWorksSlide.tsx index c44924d..d07bccd 100644 --- a/apps/tutorial/src/components/slides/HowItWorksSlide.tsx +++ b/apps/tutorial/src/components/slides/HowItWorksSlide.tsx @@ -3,10 +3,10 @@ import { motion } from 'framer-motion'; const steps = [ - { icon: '📝', label: 'What it does', description: 'Description, category, tags' }, - { icon: '⚙️', label: 'How to call it', description: 'Inputs, schema, examples' }, - { icon: '🔑', label: 'What it needs', description: 'Env vars, auth hints' }, - { icon: '📊', label: 'Signals', description: 'Downloads, health, freshness' }, + { icon: '📦', label: 'npm publish', description: 'Add tpmjs-tool keyword' }, + { icon: '🔄', label: 'Auto-sync', description: 'Indexed in ~2 minutes' }, + { icon: '⚙️', label: 'Schema extracted', description: 'From actual code' }, + { icon: '✅', label: 'Health checked', description: 'Import + execution' }, ]; export function HowItWorksSlide(): React.ReactElement { @@ -21,13 +21,13 @@ export function HowItWorksSlide(): React.ReactElement { transition={{ duration: 0.8 }} className="relative z-10 max-w-5xl w-full" > -

What TPMJS Stores

-

For each tool, we expose the useful stuff

+

How It Works

+

Automated pipeline. No manual curation.

{/* Flow diagram */}
{steps.map((step, index) => ( -
+
{/* Step card */}
$1') - .replace(/('.*?')/g, '$1') - .replace(/(\/\/.*)/g, '$1') - .replace(/(\{|\}|\(|\))/g, '$1'); + // Simple text display without syntax highlighting to avoid dangerouslySetInnerHTML + const isComment = displayText.startsWith('//'); return (
+ className={`font-mono text-sm md:text-base leading-relaxed ${isComment ? 'text-white/30' : 'text-white/80'}`} + > + {displayText || '\u00A0'} +
); } @@ -73,11 +69,11 @@ export function IntegrationSlide(): React.ReactElement { transition={{ type: 'spring', stiffness: 200, delay: 0.2 }} className="text-6xl mb-8" > - 🔌 + 📊 -

Immediate Benefits

-

Even if you do nothing else

+

Quality Scoring

+

Deliberately simple. Not trying to be clever.

{/* Code block */} - {codeLines.map((line, index) => ( - + {codeLines.map((line) => ( + ))} - {/* Benefits badges */} + {/* Score factors */} - {['Less spelunking', 'Schemas + examples', 'Shared vocabulary', 'Tool visibility'].map((benefit) => ( + {['Tier: 40-60%', 'Downloads: 0-20%', 'Stars: 0-10%', 'Richness: 0-10%'].map((factor) => ( - {benefit} + {factor} ))} diff --git a/apps/tutorial/src/components/slides/ProblemSlide.tsx b/apps/tutorial/src/components/slides/ProblemSlide.tsx index 888d338..0f4e5ba 100644 --- a/apps/tutorial/src/components/slides/ProblemSlide.tsx +++ b/apps/tutorial/src/components/slides/ProblemSlide.tsx @@ -17,9 +17,9 @@ export function ProblemSlide(): React.ReactElement { return (
{/* Floating chaotic icons */} - {floatingIcons.map((item, index) => ( + {floatingIcons.map((item) => ( -

Tool Sprawl

+

The Problem

- In modern agent stacks, tools are the{' '} - real product surface area. + npm has 2 million packages.
- But discovery is the bottleneck. + Which ones are AI-callable tools? Which ones{' '} + actually work?

- {['Dozens of packages', 'Missing schemas', 'Vague READMEs', 'Unknown status'].map((word, i) => ( - - {word} - - ))} + {["Can't find them", "Can't trust them", "Can't compare them", 'No schemas'].map( + (word, i) => ( + + {word} + + ) + )}
diff --git a/apps/tutorial/src/components/slides/QualitySlide.tsx b/apps/tutorial/src/components/slides/QualitySlide.tsx index ed7b23a..e671548 100644 --- a/apps/tutorial/src/components/slides/QualitySlide.tsx +++ b/apps/tutorial/src/components/slides/QualitySlide.tsx @@ -28,7 +28,7 @@ function AnimatedGauge({ value, delay = 0 }: { value: number; delay?: number }) return (
- +

The Multipliers

-

Extra leverage when you're ready

+

Health Checks

+

+ We don't hide broken tools. We label them. +

{/* Main gauge */} -

TPMJS

+

What TPMJS Is

- A registry that indexes AI-tool packages and exposes{' '} - normalized, enriched information + A registry that indexes npm packages + for AI tool use.
- so agents and engineers can{' '} - discover the right tool faster. + Extracts schemas from code. Scores + quality. Checks health.

{/* Organized grid of icons */} @@ -46,9 +46,9 @@ export function SolutionSlide(): React.ReactElement { transition={{ delay: 0.5 }} className="grid grid-cols-4 md:grid-cols-6 gap-4 max-w-lg mx-auto" > - {gridItems.map((item, index) => ( + {gridItems.map((item) => ( - {['Catalog', 'Metadata layer', 'Search surface', 'Consistent contract'].map((word, i) => ( + {['Registry', 'Schema extraction', 'Quality scoring', 'Health checks'].map((word, i) => ( -

Tool-Shaped Results

-

Remove guesswork before you integrate

+

What We Store

+

+ Everything an agent needs to call a tool correctly +

{/* Mock tool card */} -
// Input Schema
-
- query: string +
{'// Extracted from actual code'}
+
+ inputSchema: JSON Schema
-
- limit?: number +
+ returnSchema: JSON Schema
-
- domains?: string[] +
+ envKeys: ["API_KEY"] +
+
+ tier: "rich"
diff --git a/apps/tutorial/src/components/slides/WelcomeSlide.tsx b/apps/tutorial/src/components/slides/WelcomeSlide.tsx index 54de016..86b6188 100644 --- a/apps/tutorial/src/components/slides/WelcomeSlide.tsx +++ b/apps/tutorial/src/components/slides/WelcomeSlide.tsx @@ -56,7 +56,8 @@ export function WelcomeSlide(): React.ReactElement { transition={{ duration: 0.8, delay: 0.5 }} className="mt-6 text-2xl md:text-3xl text-white/60 font-light max-w-2xl" > - Tool Discovery for AI Agents + The missing layer between "LLMs can call tools" and "which tool, + exactly?" {/* Decorative line */} @@ -79,7 +80,13 @@ export function WelcomeSlide(): React.ReactElement { transition={{ duration: 1.5, repeat: Number.POSITIVE_INFINITY, ease: 'easeInOut' }} className="text-white/40" > - +