Commit graph

49 commits

Author SHA1 Message Date
Ajax Davis
0413b9be0e feat: complete MCP Bridge implementation
- Add @tpmjs/mcp-client package for connecting to MCP servers
- Add @tpmjs/bridge CLI for bridging local MCP servers to TPMJS
- Add @tpmjs/test-file-writer test MCP server
- Add BridgeConnection and CollectionBridgeTool database models
- Add /api/bridge endpoints for bridge communication
- Add /api/collections/[id]/bridge-tools API for managing bridge tools
- Update MCP handlers to include bridge tools in tools/list
- Add bridge status UI at /dashboard/settings/bridge
- Add interactive bridge tutorial at /docs/tutorials/bridge
2026-01-13 00:59:34 +10:00
Ajax Davis
4307cd43e3 feat: add environment variables support for agents and collections
- Add envVars field to Agent and Collection models in Prisma schema
- Add envVars to UpdateAgentSchema and UpdateCollectionSchema types
- Implement env vars merging logic (agent overrides collection)
- Pass env vars through tool executor to sandbox
- Add UI for managing env vars in agent and collection settings pages
- Update API routes to handle envVars PATCH updates
- Refactor discord tool execute functions for biome compatibility

This allows users to configure tool-specific environment variables (like
DISCORD_BOT_TOKEN) at both the collection and agent level, with agent
settings taking precedence over collection settings.
2026-01-12 01:35:29 +10:00
Ajax Davis
91fc8799b7 chore: update dependencies with compatibility fixes
- Update all packages to latest versions via pnpm update --latest
- Downgrade Prisma 7 to 6 (v7 requires schema migration)
- Downgrade Tailwind CSS 4 to 3 (v4 requires PostCSS migration)
- Downgrade Storybook 10 to 8 (addons not available in v10)
- Pin cheerio to 1.0.0-rc.12 via pnpm override (type exports changed)
- Fix AI SDK tool definitions: parameters -> inputSchema
- Fix cheerio types in extract-meta and table-extract tools
- Add explicit type annotations to tool execute functions
- Migrate biome config to v2.3.11 schema

All type-checks, tests, and builds pass.
2026-01-09 22:49:41 +10:00
Ajax Davis
3ccacf9d5b feat: redesign public pages with virtualized tables
- Add sonner for toast notifications
- Create CopyButton component for simple copy actions
- Create CopyDropdown component with entity-specific copy options
- Create PackageManagerSelector with localStorage persistence
- Redesign tools page with TableVirtuoso, sort by downloads/likes/recent/name
- Redesign collections page with TableVirtuoso and infinite scroll
- Redesign agents page with TableVirtuoso and infinite scroll
- All tables have fast client-side filtering and copy functionality
2026-01-07 21:55:13 +10:00
Ajax Davis
77c4432d35 feat: add AI Agents feature with multi-provider support and documentation
- Add Agent, AgentCollection, AgentTool, UserApiKey, Conversation, Message models to Prisma schema
- Create agent types and Zod schemas in @tpmjs/types
- Implement AES-256 API key encryption utilities
- Add CRUD API endpoints for agents, tools, collections, and user API keys
- Create conversation streaming endpoint with SSE events
- Build agent tool builder to merge collections and individual tools
- Add dashboard pages: agents list, new agent form, agent detail/edit, chat interface
- Add API keys settings page for managing provider keys
- Add comprehensive Agents documentation section to /docs
- Update navigation to include Agents link in header and mobile menu
- Add new icons: terminal, puzzle, message, key, info, send

Supported providers: OpenAI, Anthropic, Google, Groq, Mistral
2026-01-02 20:08:52 +10:00
Ajax Davis
fcaa36001b feat: add MCP endpoint for collections
Add Model Context Protocol (MCP) server endpoint for collections,
allowing users to connect Claude Desktop, Cursor, and other MCP
clients to use tools from their collections.

- POST /api/collections/[id]/mcp - JSON-RPC endpoint for MCP
- GET /api/collections/[id]/mcp - Server info for discovery
- Tool name conversion (e.g., @tpmjs/hello → tpmjs-hello--helloWorldTool)
- Handlers for initialize, tools/list, tools/call methods
- Executes tools via existing sandbox executor

Only public collections are accessible (no auth required).
2026-01-02 04:25:33 +10:00
Ajax Davis
524c12406b fix: add Prisma client to web app for Vercel monorepo compatibility
- Add @prisma/client and prisma as direct dependencies
- Add postinstall and build scripts to generate Prisma client
- Points to schema in packages/db
2026-01-02 00:37:00 +10:00
Ajax Davis
745078f6cc feat: add Better Auth with email/password authentication
- Add User, Session, Account, Verification models to Prisma schema
- Create auth configuration with Prisma adapter
- Add Resend email integration for verification emails
- Create sign-in, sign-up, and verify-email pages
- Create user dashboard with profile display
- Add middleware for /dashboard route protection
- Add better-auth and resend dependencies

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-01 20:45:10 +10:00
Ajax Davis
4fccc796f5 feat: add /tool-ideas page with 10K AI-generated tools
- Add virtualized list using react-virtuoso for smooth scrolling
- Filter by category, verb, quality score, and search
- Expandable cards with parameters, returns, AI guidance
- Add redirect from /tools-ideas to /tool-ideas

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-01 17:13:46 +10:00
Ajax Davis
9a0fb5f2d5 feat: add 100+ official TPMJS tools
Implements a comprehensive suite of AI SDK v6 tools across multiple categories:

- Research (5): page-brief, compare-pages, source-credibility, claim-checklist, timeline-from-text
- Web (10): fetch-text, links-catalog, extract-meta, extract-json-ld, redirect-trace, sitemap-read, rss-read, table-extract, robots-policy, url-normalize
- Data (15): csv-parse, csv-stringify, json-repair, json-schema-validate, yaml-parse, yaml-stringify, text-chunk, normalize-whitespace, dedupe-by-key, pivot, rows-filter, rows-sort, rows-group-aggregate, rows-join, schema-infer
- Doc (12): toc-generate, glossary-build, faq-from-text, executive-brief, decision-record-adr, prd-outline, acceptance-criteria, style-rewrite
- Eng (12): diff-text-unified, env-var-docs-generate, dependency-audit-lite, conventional-commit-suggest, markdown-lint-basic, test-case-generate, stacktrace-parse, release-notes, changelog-entry, release-checklist
- Security (7): redact-secrets, secret-scan-text, url-risk-heuristic, csp-compose, hardening-checklist-web, access-control-matrix, data-classification-heuristic
- Stats (9): effect-size-suite, bootstrap-ci, permutation-test, multiple-testing-adjust, linear-regression-ols, logistic-regression, time-series-decompose-lite, anomaly-detect-mad
- Ops (7): slo-draft, runbook-draft, postmortem-draft, postmortem-action-extractor, error-log-triage, coverage-tracker, monitoring-gap-analysis
- Agent (15): prompt-to-workflow-skeleton, workflow-validate-io, workflow-explain, workflow-cost-estimate, tool-call-accuracy-score, eval-fixture-build, guardrail-policy-draft, workflow-auto-repair, tool-selection-plan, novelty-score-workflow, workflow-variant-generate, config-normalize, recipe-*
- Utility (8): base64-encode, base64-decode, hash-text, regex-extract, template-render, date-parse, json-path-query, url-parse
- HTML (3): html-sanitize, html-to-markdown, markdown-to-html

All tools follow AI SDK v6 pattern with tool() and jsonSchema<T>().

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-31 22:55:56 +10:00
Ajax Davis
1ea9b83039 feat: fully implement 5 research tools with production-ready functionality
Research tools now have complete implementations:

- **page-brief**: Uses @mozilla/readability + jsdom for content extraction,
  sbd for sentence parsing, comprehensive error handling with timeout and
  network error detection

- **compare-pages**: Uses natural library TF-IDF for text similarity,
  detects agreements via high-similarity sentences, identifies conflicts
  using negation pattern analysis

- **source-credibility**: Uses tldts for domain parsing, cheerio for HTML
  analysis, calculates 6 credibility signals (HTTPS, domain reputation,
  author, date, citations, contact info), weighted scoring system

- **claim-checklist**: Uses sbd for sentence boundary detection, regex
  patterns for claim identification (statistics, quotes, historical,
  scientific, factual), priority levels and evidence suggestions

- **timeline-from-text**: Uses chrono-node for date parsing, sbd for
  context extraction, calculates confidence scores, identifies gaps >30 days

All tools follow AI SDK v6 pattern (tool() + jsonSchema()), include proper
TypeScript types, input validation, comprehensive error handling, and
Node.js 18+ fetch requirement verification.

Updated blocks.yml with comprehensive philosophy and domain rules for
quality validation.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-31 21:04:07 +10:00
Ajax Davis
aaf3cdd424 feat: add 5 research tools and blocks framework setup
- Add @tpmjs/tools-page-brief for URL content extraction
- Add @tpmjs/tools-compare-pages for cross-source validation
- Add @tpmjs/tools-source-credibility for credibility scoring
- Add @tpmjs/tools-claim-checklist for factual claim extraction
- Add @tpmjs/tools-timeline-from-text for timeline generation
- Move createBlogPost to packages/tools/official/
- Add blocks.yml for Blocks framework validation
- Update pnpm-workspace.yaml to include official tools
2025-12-31 19:47:02 +10:00
Ajax Davis
e9a7a689a1 perf: reduce Neon compute usage with cron + caching optimizations
- Reduce cron frequency: changes 2min→4hr, keyword 15min→6hr, metrics hourly→daily
- Add Prisma directUrl for connection pooling support
- Add Vercel KV caching to /api/tools endpoint (graceful degradation if not configured)
- Add X-Cache header to indicate cache hit/miss
- Add NEON_COMPUTE_OPTIMIZATION.md with full strategy guide

These changes should reduce Neon CU usage from 100+ to ~20-30 CU-hrs/month.
2025-12-31 12:14:00 +10:00
Ajax Davis
06029fbc60 feat: add interactive tutorial slideshow app
Create apps/tutorial - a beautiful, animated Next.js slideshow that explains
TPMJS concepts from basic to advanced:

Slides:
- Welcome: Gradient TPMJS title with animated floating orbs
- Problem: Chaotic floating icons showing fragmented AI tools
- Solution: Icons organizing into a neat grid
- How It Works: Flow diagram (npm → Discovery → Registry → AI Agent)
- Discovery: Animated counters showing real-time sync stats
- Integration: Typewriter code animation showing SDK usage
- Quality Scoring: Animated gauge and progress bars
- Tool Detail: Mock tool card with all metadata
- Get Started: CTA buttons linking to tpmjs.com

Features:
- Framer Motion animations with spring physics
- Keyboard navigation (arrows, space, enter)
- Mouse navigation (arrows, progress dots)
- Touch-friendly swipe support
- Dark theme with cyan/purple gradient accents
- Responsive design
2025-12-28 23:41:15 +10:00
Ajax Davis
87b200a735 feat: add build-time OG image generation with OpenAI
- Create generate-og-images.ts script for build-time image generation
- Generate 13 static page OG images using gpt-image-1-mini
- Update API route to serve pre-generated images from public/og/
- Add 30-day cache headers and fallback to default image
- Images regenerate if older than 30 days

Run with: pnpm --filter=@tpmjs/web generate-og

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-28 20:45:02 +10:00
Ajax Davis
75a386359c feat: add AI-generated OG images with OpenAI and Vercel Blob caching
- Add /api/og/[...path] endpoint for dynamic OG image generation
- Use OpenAI gpt-image-1 for image generation with page-specific prompts
- Cache images in Vercel Blob storage with 30-day TTL
- Extract page content for contextual prompts (static pages, tool details)
- Update all page metadata to use dynamic OG image URLs
- Refactor tool detail page to server component for generateMetadata support
- Fall back to static /public/og-image.png on generation errors
2025-12-28 18:21:11 +10:00
Ajax Davis
b4f6eb46b8 feat: add comprehensive stats dashboard with D3 charts
- Add /stats page with animated D3 visualizations
- Create reusable chart components: AnimatedCounter, DonutChart, BarChart, AreaChart
- Expand stats API endpoints: /api/stats, /api/stats/health, /api/stats/executions, /api/stats/sync, /api/stats/tools
- Add Stats link to desktop and mobile navigation
- Add AI SDK v6 integration tests with vitest

Dashboard displays:
- Registry overview metrics with count-up animations
- Health distribution donut charts (import/execution)
- Quality score distribution
- Package tier breakdown
- Execution trends area chart with success/error series
- Token usage statistics
- Top categories bar chart
- Recent sync operations status
2025-12-28 17:08:19 +10:00
Ajax Davis
52ea479ad9 chore: upgrade AI SDK from beta to stable v6
- ai: 6.0.0-beta.124 → 6.0.3
- @ai-sdk/openai: 3.0.0-beta.74 → 3.0.1
- @ai-sdk/react: 3.0.0-beta.131 → 3.0.3

Breaking changes addressed:
- CoreMessage renamed to ModelMessage
- convertToModelMessages is now async (added await)
2025-12-28 16:17:41 +10:00
Ajax Davis
d2b22201aa feat: add Vercel Analytics to web and playground apps
- Install @vercel/analytics in both apps
- Add <Analytics /> component to root layouts
- Enables automatic page view tracking on Vercel
2025-12-17 11:04:02 +10:00
Ajax Davis
ff17182e04 chore: update dependencies and add unsandbox package 2025-12-16 12:00:22 +10:00
Ajax Davis
db482855d3 fix(createblogpost): convert to proper AI SDK tool
- Wrap function with tool() from 'ai' package
- Add jsonSchema() for input validation
- Rename export from createBlogPost to createBlogPostTool
- Update package.json with proper tpmjs.tools format
- Bump version to 0.3.0

Fixes executor error: 'Cannot destructure property title of t'
2025-12-12 10:38:46 +10:00
Ajax Davis
53d30f7eca feat(sdk): replace ASCII diagram with interactive D3 visualization
- Create SDKFlowDiagram component with animated D3 graphics
- Add flowing particle animations along connection paths
- Add hover interactions with tooltips for each node
- Add entrance animations with staggered timing
- Add subtle glow effects and shadows
- Responsive design that adapts to screen width
- Sleek minimal black and white aesthetic with depth effects
2025-12-12 09:43:40 +10:00
Ajax Davis
463889b7c4 feat: add @tpmjs/registrySearch and @tpmjs/registryExecute SDK packages
- Create @tpmjs/registrySearch package for searching tool registry
- Create @tpmjs/registryExecute package for executing tools via sandbox
- Support self-hosted registries via TPMJS_API_URL and TPMJS_EXECUTOR_URL env vars
- Add /sdk documentation page with usage examples and architecture
- Add SDK link to navigation menu
- Include design doc for registry SDK architecture

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-12 07:29:06 +10:00
Ajax Davis
256699099b fix(security): update Next.js to 16.0.8 to address CVE-2025-66478 2025-12-11 11:14:43 +10:00
Ajax Davis
79547ff52e feat: add markdown-formatter package with defensive patterns
- Create @tpmjs/markdown-formatter with 2 tools:
  - markdownToPlainText: Convert markdown to plain text
  - formatMarkdownTable: Format and align markdown tables
- Includes defensive parameter validation
- Uses AI SDK v6 beta with Zod 4 schemas
- Published v0.2.0 to npm

Testing the full end-to-end workflow:
- Package creation following generator patterns
- Changesets for version management
- npm publishing
- TPMJS registry auto-discovery
2025-12-05 01:31:03 +10:00
Ajax Davis
3cfc141944 chore: release emoji-magic v0.2.0 and create-basic-tools v1.0.4 2025-12-05 00:10:18 +10:00
Ajax Davis
051e58c23d feat: add @tpmjs/create-basic-tools CLI generator
- Interactive CLI generator for scaffolding TPMJS tool packages
- Generates packages with minimum 2 tools (ideally 2-3)
- Zod 4 schemas - uses Zod directly (not jsonSchema wrapper)
- One file per tool in src/tools/<toolName>.ts
- TPMJS validated against official schemas from @tpmjs/types
- Complete package generation ready to publish to npm
- Works both standalone and in monorepo packages/ folders

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 23:34:37 +10:00
Ajax Davis
9abff1f203 feat: add reusable ToolHealthBadge and ToolHealthBanner components
- Create ToolHealthBadge component in @tpmjs/ui for broken tool indicator
- Create ToolHealthBanner component in @tpmjs/ui for detailed health warnings
- Integrate both components into playground ToolsSidebar:
  - Badge shows in tool cards in left sidebar
  - Banner shows in tool detail modal
- Update search-registry to include health fields in API responses
- Add package.json exports for new health components

These components provide consistent UI for displaying broken tool status
across the application (tool search, tool detail pages, playground).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 19:12:07 +10:00
Ajax Davis
1642ec07d9 feat: add automated Vercel AI registry sync with OpenAI
Add hourly GitHub Action that syncs tools from Vercel's AI SDK registry:

Features:
- Fetches Vercel AI registry from their GitHub
- Uses OpenAI GPT-4 to intelligently convert tool metadata
- Handles multi-export packages (multiple tools per npm package)
- Automatically commits new tools to manual-tools.ts
- Sends Discord notifications with detailed stats
- Extensive logging at every step

Files added:
- sync-vercel-registry.ts - Main sync script with AI conversion
- .github/workflows/sync-vercel-registry.yml - Hourly GitHub Action
- docs/vercel-registry-sync.md - Complete documentation

Requires OPENAI_API_KEY secret in GitHub repository settings.
2025-12-04 17:43:17 +10:00
Ajax Davis
8b5757c9aa fix: migrate playground to AI SDK v6 beta and resolve TypeScript errors
- Update @ai-sdk/react to v3.0.0-beta.131 for compatibility with AI SDK v6
- Fix tool execute method calls with type assertions in API routes
- Update chat components to use UIMessage types from AI SDK
- Move body option into DefaultChatTransport constructor
- Remove deprecated onResponse option from useChat hook
- Remove unused isCoreTool type guard function
- Fix Button variant from 'primary' to 'default'

Resolves all TypeScript compilation errors and build passes successfully.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 13:15:21 +10:00
Ajax Davis
2158ee6dfd feat: add dynamic tool loading system with Railway executor
Implements a complete dynamic tool loading system that allows the playground to discover and load tools from the TPMJS registry at runtime.

**Architecture:**
- Search tool package (@tpmjs/search-registry) - Searches registry for tools
- Search API endpoint (/api/tools/search) - Text-based search with scoring
- Pre-flight tool loading - Automatically searches and loads tools on every message
- Railway executor service (Deno) - Loads tools from esm.sh via HTTP imports
- Dynamic tool loader - Calls Railway to load and execute tools remotely

**Key Components:**

1. Railway Executor (apps/railway-executor/)
   - Deno-based service that natively supports HTTP imports
   - Endpoints: /load-and-describe, /execute-tool, /cache/stats, /cache/clear
   - Deploys to Railway with deno run --allow-net --allow-env server.ts

2. Search Tool Package (packages/tools/search-registry/)
   - AI SDK v6 tool for searching TPMJS registry
   - Uses jsonSchema + inputSchema pattern
   - Searches /api/tools/search endpoint

3. Search API (apps/web/src/app/api/tools/search/)
   - Text-based search with composite scoring
   - Scores: text relevance + quality boost + download boost
   - Returns tool metadata with importUrl for dynamic loading

4. Dynamic Tool Loader (apps/playground/src/lib/dynamic-tool-loader.ts)
   - Calls Railway service to load tools from esm.sh
   - Creates tool wrappers that execute remotely
   - Process-level module cache + per-conversation tracking

5. Pre-flight Loading (apps/playground/src/app/api/chat/route.ts)
   - Automatically searches for tools on every user message
   - Loads top 5 matching tools before agent processes request
   - Merges with static tools for seamless experience

**Technical Decisions:**
- Deno over Node.js: Native HTTP import support without flags
- Remote execution: Tools run in Railway sandbox, not Vercel
- Pre-flight loading: Better UX than two-turn search pattern
- Text search: BM25 had dependency issues, simple scoring works well

**Environment Variables:**
- RAILWAY_SERVICE_URL: https://endearing-commitment-production.up.railway.app

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 08:25:53 +10:00
Ajax Davis
0612eac5e2 feat: implement multi-tool package architecture with manual tool registry
BREAKING CHANGE: Complete refactoring from single-tool to multi-tool package support

Database Schema:
- Split Tool model into Package (1) and Tool (many) with one-to-many relationship
- Package stores npm metadata and package-level tpmjs fields (category, env, frameworks, tier)
- Tool stores individual tool exports with tool-level metadata (exportName, description, parameters, returns, aiAgent)
- Unique constraint on (packageId, exportName) to prevent duplicate tools
- Cascade deletes when packages are removed

Type System:
- Updated tpmjs field schema to support tools array
- Each tool has exportName, description, parameters, returns, aiAgent
- Package-level fields: category, env, frameworks shared across all tools
- Backward compatible with legacy single-tool format (auto-migrates to exportName: "default")

API Updates:
- Updated all /api/tools routes to query Tool model with Package relations
- Updated /api/tools/[slug] to accept package/export path segments
- Updated tool-executor-agent to use actual exportName instead of hardcoded "default"
- Updated metrics sync to calculate quality scores per Tool

Frontend Updates:
- Updated tool search page to display exportName as primary heading
- Updated tool detail pages to show package name as secondary info
- Removed tag-based filtering (tags moved to package level)

Manual Tool Registry:
- Added manual-tools.ts with 23 curated tools from major providers
- Created sync-manual-tools.ts script to sync manual tools to database
- Added MANUAL_TOOLS.md documentation for manual tool system
- Added GitHub workflow for automated daily sync
- Includes tools from: Vercel, Exa, Firecrawl, AWS Bedrock, Perplexity, Tavily, Superagent, Valyu

Playground Updates:
- Updated tool loader to load multiple tools per package
- Added sanitizeToolName for OpenAI API compatibility

Sync System Updates:
- Updated changes feed sync to handle multi-tool packages
- Updated keyword sync to upsert multiple tools per package
- Added orphaned tool deletion when tools removed from package.json

Migration Strategy:
- Database uses same Neon instance for dev and prod
- Schema updated via prisma db push (no migration files yet)
- All data repopulates from npm via sync system

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 06:39:58 +10:00
Ajax Davis
635fc96cac feat: implement playground app with AI SDK v6 tool execution
- Create new Next.js app at apps/playground for testing TPMJS tools
- Implement AI SDK v6 patterns with DefaultChatTransport and UIMessage format
- Create template tool package at packages/tools/hello with hello-world and hello-name tools
- Use tool() and jsonSchema() helpers to avoid Zod 4 conversion issues with OpenAI
- Add static tool loading system with switch statement (Next.js/webpack compatible)
- Implement chat interface with tool call visualization showing inputs/outputs
- Support multi-step tool execution with stepCountIs(5)
- Stream responses with toUIMessageStreamResponse() for full tool support
- Add sidebar showing available tools (static list)
- Use parts-based message rendering for text and tool calls
- Integrate firecrawl-aisdk tools (scrape, crawl, search)
- Add theme toggle in header (defaults to light mode)
- Fix responsive layout with max-width for message bubbles
- Use biome-ignore comments for legitimate any types in tool loading

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 02:51:02 +10:00
Ajax Davis
2a05e68aa3 fix: preserve streamed output and add markdown rendering to playground
**Problem:**
- Tool playground was only showing partial output
- When 'complete' event arrived, it replaced streamed text with final output
- Output was displayed as plain text instead of formatted markdown

**Changes:**
- Fixed streaming bug in ToolPlayground component
  - Removed line that overwrote accumulated text on 'complete' event
  - Now preserves all streamed chunks for full output display
- Added react-markdown with GitHub Flavored Markdown support
  - Install react-markdown and remark-gfm packages
  - Added @tailwindcss/typography plugin for prose styling
  - Replaced plain <pre> with <ReactMarkdown> component
  - Applied prose classes for proper markdown formatting
- Added type="button" to button elements for accessibility

**Files changed:**
- apps/web/src/components/ToolPlayground.tsx
  - Comment out setOutput(data.output) on complete event
  - Import ReactMarkdown and remarkGfm
  - Replace pre element with ReactMarkdown component
  - Add prose styling classes
  - Add type="button" to buttons
- apps/web/tailwind.config.ts
  - Add @tailwindcss/typography plugin

**Result:**
- Full streamed output now displays correctly
- Markdown is rendered with proper formatting (headings, lists, code blocks, etc.)
- Better UX for AI-generated tool responses

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-30 19:28:53 +10:00
Ajax Davis
78c72b85f3 feat: upgrade to AI SDK v6 beta and Zod v4 to fix tool schema errors
OpenAI was rejecting tool definitions with error "schema must be a JSON Schema of 'type: "object"'". This was caused by AI SDK v5 not properly converting Zod schemas to JSON Schema format.

**Changes:**
- Upgrade AI SDK from v5.0.104 to v6.0.0-beta.124
- Upgrade @ai-sdk/openai from v2.0.74 to v3.0.0-beta.22
- Upgrade Zod from v3.25.76 to v4.1.13 across all packages

**AI SDK v6 breaking changes:**
- Tool definition API: `parameters` renamed to `inputSchema`
- Removed `aiTool()` wrapper - use plain object with description, inputSchema, execute
- Streaming API: Use `textStream` async iterator instead of onChunk callback
- Zod schemas now properly converted to JSON Schema for OpenAI

**Zod v4 breaking changes:**
- `z.record()` now requires two arguments: `z.record(keySchema, valueSchema)`
- `z.enum()` params changed: `errorMap` removed, use `message` instead
- Type system improvements require explicit type parameters
- Fixed type errors in @tpmjs/env, @tpmjs/npm-client, @tpmjs/types

**Files changed:**
- apps/web/src/lib/ai-agent/tool-executor-agent.ts
  - Updated tool definition to use `inputSchema` instead of `parameters`
  - Removed `aiTool()` wrapper
  - Fixed streaming to use `textStream` iterator
- packages/env/src/index.ts
  - Updated type constraint from `z.ZodRawShape` to `Record<string, z.ZodTypeAny>`
- packages/npm-client/src/package.ts
  - Fixed `z.record()` calls to include both key and value schemas
  - Added type assertions for record indexing
- packages/types/src/tpmjs.ts
  - Changed `errorMap` to `message` in z.enum() calls

**Testing:**
-  Type-check passes
-  Production build succeeds
-  All routes compile correctly

This fixes the tool execution error where OpenAI rejected tool schemas with invalid format.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-30 19:14:01 +10:00
Ajax Davis
8f3c6a2910 fix: sanitize npm package names for OpenAI tool names and update lockfile
Problem: OpenAI tool names must match ^[a-zA-Z0-9_-]+ but npm package names like @tpmjs/createblogpost contain @ and /

Solution: Add sanitizeToolName() function and update pnpm-lock.yaml

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-30 02:28:42 +10:00
Ajax Davis
d73db1c333 fix: remove VM2 sandboxing to resolve Next.js build errors
**VM2 Removal:**
- Remove VM2 dependency from package-executor
- Rewrite executor to use direct package execution with require()
- Add TODO comment for future sandboxing implementation

**Why this change:**
- VM2 requires runtime filesystem access to bridge.js which doesn't work with Next.js Turbopack bundling
- Even marking as serverExternalPackages fails because VM2 uses hardcoded file paths
- Direct execution allows builds to complete while we find Next.js-compatible sandboxing solution

**Next Steps:**
- Implement proper sandboxing with isolated-vm or similar Next.js-compatible solution
- Add security measures for package execution
- Consider moving package execution to separate microservice

This unblocks CI/CD while maintaining playground functionality.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-30 00:57:32 +10:00
Ajax Davis
6eb51d1371 feat: add interactive tool playground with AI-powered execution
Implement comprehensive tool testing environment with real package execution, AI agents, and token tracking.

**New Features:**
- Interactive playground UI with 4 tabs (Input, Output, Logs, Token Usage)
- Real npm package execution in VM2 sandbox with security constraints
- AI-powered tool execution using AI SDK v5 and GPT-4 Turbo
- Server-Sent Events (SSE) streaming for real-time progress updates
- Comprehensive 4-category token tracking (Input, Tool Description, Schema, Output)
- Visual token breakdown with colored progress bars
- IP-based rate limiting (10 executions/hour per IP)
- Database persistence of all simulations with full metadata

**Database Schema:**
- New `Simulation` model for execution records
- New `TokenUsage` model for detailed token metrics
- New `ExecutionLog` model for execution event tracking
- Added simulations relation to Tool model

**Package Executor (@tpmjs/package-executor):**
- VM2 sandbox with 5-second timeout
- Blocked dangerous modules (fs, net, http, https, child_process)
- LRU file system cache in /tmp/.tpmjs-cache
- Package installation and caching strategy

**AI Agent Service:**
- TPMJS parameter to Zod schema conversion
- AI SDK tool definition generation
- Token counting using tiktoken library
- GPT-4 Turbo pricing estimation
- Streaming text execution with callbacks

**API Endpoints:**
- POST /api/tools/[...slug]/execute - SSE streaming execution
- GET /api/tools/[...slug]/simulations - Execution history
- Rate limit headers (X-RateLimit-Limit, X-RateLimit-Remaining)

**Frontend Components:**
- ToolPlayground - Main playground UI with tabs
- TokenBreakdown - Visual token metrics with colored bars
- Integrated above README section on tool detail pages

**Security:**
- VM2 sandboxing prevents filesystem/network access
- Rate limiting prevents abuse
- IP tracking for usage monitoring
- Timeout protection (60s max API duration, 5s VM timeout)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-30 00:36:05 +10:00
Ajax Davis
91f950d453 feat: add syntax highlighting and improve README readability
**Syntax Highlighting:**
- Add react-syntax-highlighter with Solarized Light theme
- Proper language detection from markdown code blocks
- Beautiful syntax highlighting for all code examples
- Improved inline code styling with subtle borders

**Enhanced Readability:**
- Larger base typography with prose-lg
- Better contrast for text colors (zinc-700/zinc-300)
- Improved heading spacing and hierarchy
- Enhanced table styling with hover effects and better spacing
- Table headers with uppercase, bold styling
- Table cells with generous padding (px-6 py-4/py-3)
- Row hover effects for better interaction
- Better blockquote styling with blue accents
- Improved list spacing with leading-relaxed
- Enhanced image borders and shadows

**Table Improvements:**
- Professional header styling with background colors
- Better cell padding and spacing
- Hover effects on rows
- Improved borders and shadows
- Responsive overflow handling

All code blocks now have beautiful Solarized Light syntax highlighting, and the overall typography is more readable and professional.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-29 23:21:40 +10:00
Ajax Davis
f9ba1c094e feat: add README rendering and enhanced package metadata display
**Database Schema:**
- Add npmReadme, npmKeywords, npmAuthor, npmMaintainers fields to Tool model

**NPM Client:**
- Add fetchLatestPackageWithMetadata() function to fetch README and top-level metadata
- Export new PackageVersionWithReadme type

**Sync Workers:**
- Update keyword and changes sync to fetch and store README content
- Store author, maintainers, and keywords from package.json

**UI Components:**
- Create Markdown component using react-markdown with GitHub Flavored Markdown
- Add rehype-sanitize for security and remark-gfm for tables/strikethrough support

**Tool Detail Page:**
- Convert from createElement to JSX for better maintainability
- Display README in a dedicated card with proper markdown rendering
- Show NPM keywords, author, and maintainers in sidebar
- Add ThemeToggle to header
- Improve layout with better spacing and organization

This brings the tool detail pages much closer to NPM's package pages,
providing users with comprehensive information about each tool including
the full README documentation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-29 22:41:50 +10:00
Ajax Davis
b58df449da feat(tools): create @tpmjs/createblogpost test tool package
Add test tool package to verify sync system can discover and process tools:

Package Features:
- Creates structured blog posts with frontmatter and metadata
- Supports both Markdown and MDX output formats
- Automatic slug generation from title
- Word count and reading time calculation
- SEO-friendly metadata generation

TPMJS Integration:
- Rich-tier tpmjs field with all optional metadata
- Tagged with 'tpmjs-tool' keyword for NPM discovery
- Complete parameter and return type documentation
- AI agent guidance for optimal LLM usage
- Pricing, authentication, and framework metadata

Implementation:
- Full TypeScript implementation with exported types
- Proper tsconfig and tsup build configuration
- Comprehensive README with usage examples
- Changeset for publishing workflow
- Added packages/tools/* to pnpm workspace

This package serves as an end-to-end test of the sync workers to verify:
1. NPM keyword search discovers the package
2. Changes feed picks up updates
3. tpmjs field validation works correctly
4. Rich-tier metadata is properly extracted
5. Quality score calculation functions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-28 03:10:31 +10:00
Ajax Davis
3b95502577 feat(sync): implement Phase 3 Sync Workers with Vercel Cron integration
Add complete sync worker system for NPM package discovery and metrics:

**Sync Workers:**
- Changes Feed Sync (/api/sync/changes) - Polls NPM changes every 2 min
- Keyword Search Sync (/api/sync/keyword) - Searches tpmjs-tool keyword every 15 min
- Metrics Sync (/api/sync/metrics) - Updates downloads & quality scores hourly

**Features:**
- Secure CRON_SECRET authentication for all sync endpoints
- Comprehensive error handling with sync logs and checkpoints
- Smart package validation and filtering (skip invalid tpmjs fields)
- Automatic tool upsert with discovery method tracking
- Quality score calculation based on tier, downloads, and GitHub stars
- 5-minute timeout support for long-running sync operations

**Dependencies:**
- Add @tpmjs/npm-client to web app for NPM API integration
- Use barrel exports from npm-client package (no subpath imports)
- Add CRON_SECRET env variable validation
- Add ~/src path alias to tsconfig

**Infrastructure:**
- Configure Vercel Cron jobs in vercel.json for automated syncing
- Add publishedAt field to PackageVersion schema
- Fix Prisma JSON field handling (use undefined instead of null)
- Proper null checks for fetchLatestPackageVersion return values

**Type Safety:**
- Cast TpmjsField union type to access optional rich-tier properties
- Handle searchByKeyword array return type correctly
- Fix fetchDownloadStats to return number directly

All API routes follow Next.js 16 conventions with proper type checking.
Type-check and full build successful.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-28 02:57:54 +10:00
Ajax Davis
3ae9ced6fa feat(db): create database package with Prisma schema for NPM registry
- Add @tpmjs/db package with Prisma ORM setup
- Define Tool model with NPM metadata and TPMJS fields
- Define SyncCheckpoint model for tracking sync worker progress
- Define SyncLog model for audit trail of sync operations
- Add Prisma client singleton with dev logging
- Add seed script for initializing sync checkpoints
- Include comprehensive README with setup instructions

Package includes:
- Complete Prisma schema matching NPM_MIRROR.md spec
- Three models: Tool, SyncCheckpoint, SyncLog
- Indexes for performance on key fields
- TypeScript support via @tpmjs/tsconfig
- Scripts for db:migrate, db:push, db:studio, db:seed

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-28 02:07:39 +10:00
Ajax Davis
ded939e643 feat(ui): add comprehensive form component library with playground showcase
Implemented 8 production-ready form components with full accessibility:

Components Added:
- Textarea: Multi-line text input with character counter
- Checkbox: Custom styled with indeterminate state support
- Radio & RadioGroup: Context-based radio button groups
- Switch: Toggle with animated thumb and loading state
- Select: Native select with custom styling and option groups
- Slider: Range input with marks, value display, cross-browser support
- FormField: Wrapper component with label, error, and helper text

Features:
- Full accessibility (ARIA attributes, semantic HTML)
- Controlled/uncontrolled patterns via useControlled hook
- Dark mode support with semantic tokens
- Design tokens and shared variant system
- Comprehensive test coverage (856 tests passing)
- Form-specific design tokens (formTokens)
- Shared form variant base classes (formVariants)

Playground Updates:
- Added comprehensive Forms section showcasing all components
- Interactive examples with state management
- Complete form composition example
- All components fully functional and themed

Test Coverage:
- 10+ describe blocks per component
- All edge cases covered
- Accessibility testing
- Cross-browser compatibility

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-27 01:56:54 +10:00
Ajax Davis
cbed8f34cc fix: add eslint as devDependency to packages that lint
Packages/apps that run `eslint .` need eslint installed as a
devDependency. While eslint is in @tpmjs/eslint-config, it needs to
be available in the local node_modules/.bin for the lint script to run.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-26 18:42:39 +10:00
Ajax Davis
378b068e38 feat: add quality gate tools and Node 22 setup
- Add type-coverage, knip, and dependency-cruiser for code quality
- Set up Node 22 (LTS) with .nvmrc file
- Configure knip for dead code detection across monorepo
- Configure dependency-cruiser with sensible architecture rules
- Add ts-reset for better TypeScript built-in types
- Fix Tabs component type exports for Storybook
- Recreate eslint react.js config that was missing
- Add quality gates documentation

All quality checks pass with 0 errors (only informational warnings).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-26 17:33:45 +10:00
Ajax Davis
a6753cd8e6 feat: add light/dark theme system and full homepage with comprehensive UI components
Implement theme system:
- Add next-themes for theme management with light mode as default
- Swap CSS variable definitions in globals.css (light in :root, dark in .dark class)
- Create ThemeProvider component wrapping next-themes
- Create ThemeToggle component with sun/moon icons
- Update root layout with ThemeProvider

Build full homepage:
- Add Header component with navigation and theme toggle
- Create hero section with gradient background and search
- Add Featured Tools section with 6 tool cards (icons, badges, usage stats)
- Add Browse by Category section with 12 colored category tiles
- Add Platform Statistics section with 4 metric cards
- Add footer with copyright and links
- Create homePageData.ts with comprehensive mock data

Add Storybook stories:
- Icon.stories.ts (AllIcons, AllSizes, WithColors)
- Badge.stories.ts (Default, AllVariants, AllSizes, WithLongText)
- Input.stories.ts (Default, WithLabel, AllStates, AllSizes, PasswordInput)
- ProgressBar.stories.ts (Default, AllVariants, AllSizes, WithLabel, edge cases)
- CodeBlock.stories.ts (Default, multiple languages, LongCode)
- Tabs.stories.ts (Default, AllSizes, WithCounts, ManyTabs)

Fix production build issues:
- Export IconName type from Icon.ts
- Add sun and moon icons to icons.ts
- Fix Icon prop naming (icon not name)
- Fix Icon size constraints (remove xl, use lg)
- Update ThemeProvider types using ComponentProps
- Add explicit return types where needed
- Remove unused imports
- Replace forEach with for...of in variants system
- Fix template literals in storybook stories

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-26 01:29:28 +10:00
Ajax Davis
aaf6d788c3 chore: version packages 2025-11-25 22:23:10 +10:00
Ajax Davis
78e6b9f4b5 feat: add complete monorepo structure and all packages
This completes the initial monorepo setup with all packages and configuration.

Packages added:
- @tpmjs/ui - React component library with .ts-only components
- @tpmjs/utils - Utility functions (cn, format)
- @tpmjs/types - TypeScript types and Zod schemas
- @tpmjs/env - Environment variable validation
- @tpmjs/test - Shared Vitest configuration
- @tpmjs/mocks - MSW mock server
- @tpmjs/config - Shared configurations (Biome, ESLint, Tailwind, TypeScript)

Applications added:
- @tpmjs/web - Next.js 16 App Router

Infrastructure:
- Turborepo configuration with build pipeline
- pnpm workspace setup
- GitHub Actions CI/CD
- Changesets for versioning
- Lefthook pre-commit hooks
- VSCode workspace settings
- Comprehensive documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-25 21:59:40 +10:00