Commit graph

455 commits

Author SHA1 Message Date
Ajax Davis
5926373be6 chore: update ai package to 6.0.49 across all packages
- Updated all packages from ai@6.0.23 to ai@6.0.49
- Added pnpm override to ensure consistent version
- Created changeset for publishing affected packages
2026-01-25 11:33:03 +10:00
Ajax Davis
dff523fa99 fix(omega): move registry packages to transpilePackages for ESM support
- @tpmjs/registry-search and @tpmjs/registry-execute use ESM syntax
- Move from serverExternalPackages to transpilePackages so Next.js
  properly bundles and transpiles them
- Revert to static imports now that packages are bundled correctly
2026-01-25 05:52:52 +10:00
Ajax Davis
69162682d1 fix(omega): lazy load registry tools to fix serverless import issues 2026-01-25 05:34:12 +10:00
Ajax Davis
88c4848ab6 fix(omega): resolve module import crash and improve error handling
- Move @ai-sdk/devtools import to lazy dynamic import to prevent
  module load failures in production environment
- Add try-catch around response.json() in client to handle
  non-JSON error responses gracefully
- Display proper error message when server returns non-JSON response
2026-01-25 05:15:40 +10:00
Ajax Davis
bfd45341bb style: update design system to clean, minimal aesthetic
- Light mode: pure white backgrounds (#FFFFFF) with high contrast
  near-black text (#0A0A0A) for better readability
- Dark mode: clean neutral blacks (#0A0A0A base) instead of warm browns
- Borders use neutral grays instead of warm-tinted grays
- Status colors (success, warning, error, info) more saturated and vibrant
- Maintains copper accent color (#A6592D) as brand element
- Overall cleaner, more minimal feel with stronger contrast
2026-01-25 04:57:12 +10:00
Ajax Davis
209fd275f3 feat(skills): use Streamdown to render markdown in question detail page 2026-01-25 04:45:08 +10:00
Ajax Davis
b5731cf1a1 feat(skills): add clickable questions with detail pages and browse view
- Create /api/skills/questions endpoint for listing questions with pagination
- Create /api/skills/questions/[id] endpoint for individual question details
- Add questions list page with filtering by skill
- Add question detail page with full answer, related tools, and similar questions
- Make activity feed cards clickable links to question detail
- Add "View all" link in SkillsSection
2026-01-25 04:22:05 +10:00
Ajax Davis
9995d73052 refactor: remove UseCasesSection from collection pages
Scenarios now replace use cases as the primary way to demonstrate
collection capabilities. Removes unused state and callback handlers.
2026-01-25 04:13:59 +10:00
Ajax Davis
9f0d7a2b17 fix(skills): refactor skills section to use design system tokens
- Replace hardcoded purple/blue colors with primary/10 backgrounds
- Use text-primary for icons instead of hardcoded colors
- Use error tokens for error states instead of red-*
- Use ProgressBar component instead of custom div progress bars
- Use EmptyState component for empty activity feed
- Use Card components consistently with proper padding
- Fix TypeScript types in integration tests
2026-01-25 03:52:20 +10:00
Ajax Davis
234548c6b2 chore: sync remaining RealSkills updates and config changes
- Update skills route, activity feed, and stats components
- Update skills embedding and response generator modules
- Update Omega conversation messages route
- Update PRD documentation
- Refresh CLI oclif manifest
- Update gitignore and package configs
2026-01-25 03:34:27 +10:00
Ajax Davis
a44f38eda9 test: add comprehensive tests for CLI and scenarios
Add unit tests for:
- CLI TpmClient (api-client.test.ts) - 26 tests covering authentication,
  tool execution, agent/collection/scenario management, error handling
- Scenario evaluation (evaluate.test.ts) - 16 tests for regex assertions
  and verdict determination
- Cosine similarity (similarity.test.ts) - 16 tests for embedding comparison

Add integration tests for scenarios:
- CRUD operations (scenarios-crud.integration.test.ts)
- Run execution and quota management (scenarios-run.integration.test.ts)

Configure vitest for CLI package with @tpmjs/test shared config.
2026-01-25 03:28:28 +10:00
Ajax Davis
9c731d582e fix(web): use username/slug URL format for collection links on /collections page 2026-01-25 02:57:57 +10:00
Ajax Davis
1118463e6b feat(skills): add RealSkills living endpoint for agent Q&A
Implements a skills endpoint that evolves through agent conversations:

- GET /:username/collections/:slug/skills - Returns markdown skill summary
- POST /:username/collections/:slug/skills - Ask questions, get RAG+LLM responses

Features:
- OpenAI text-embedding-3-large (3072 dims) for semantic similarity
- GPT-4.1-mini for response generation with RAG context
- Lazy seeding of synthetic questions on first access
- Cache hits for >95% similar questions
- Real-time skill graph updates (emergent skill taxonomy)
- Session support for multi-turn conversations
- Activity feed and stats APIs for UI

Database models: SkillQuestion, Skill, SkillSession, SkillQuestionSkill, SkillQuestionTool
2026-01-25 02:37:02 +10:00
Ajax Davis
0489646e55 feat(ui): add ToolRenderer system for Omega chat
Add abstract tool rendering infrastructure to @tpmjs/ui:
- ToolRenderer component with registry-based renderer lookup
- DefaultJsonRenderer as fallback with collapsible JSON display
- RegistrySearchRenderer for registrySearchTool results
- RegistryExecuteRenderer for registryExecuteTool results
- registerBuiltInRenderers() for idempotent initialization

Update Omega chat page to use new ToolRenderer:
- Replace inline ToolCallCard with ToolRenderer component
- Add helper functions to convert between ToolCall and ToolPart
- Remove unused expandedToolCalls state (managed internally)

Also add video file extensions to .gitignore
2026-01-23 22:35:17 +10:00
Ajax Davis
ca6f908ebd fix(web): use npm versions for registry packages instead of workspace 2026-01-23 18:27:37 +10:00
Ajax Davis
fb06055f80 fix(web): remove registry packages from transpilePackages to fix conflict 2026-01-23 18:19:05 +10:00
Ajax Davis
c4ea363497 fix(web): configure ESM exports for registry packages 2026-01-23 18:11:55 +10:00
Ajax Davis
c4430fb1bf fix(web): add registry packages to transpilePackages config 2026-01-23 18:02:48 +10:00
Ajax Davis
0d3395b32f feat(omega): add environment variable management for tool API keys
- Add /api/omega/settings/env-vars API endpoints for CRUD operations
- Add /omega/settings page with env var management UI
- Pass user env vars to tool execution in messages route
- Detect and warn about missing required env vars via SSE events
- Add EnvVarWarningBanner component to chat UI
- Update system prompt to document registry tools usage
- Improve auth flow on landing page with sign-in redirect
2026-01-23 17:43:22 +10:00
Ajax Davis
19a905d9d1 refactor(omega): use BM25 auto-loading instead of meta-tools 2026-01-23 10:54:02 +10:00
Ajax Davis
3259e038fb feat: add Omega AI agent chat feature
- Add Prisma models for conversations, messages, participants, tool runs, and user settings
- Create API endpoints for conversation CRUD and SSE message streaming
- Build landing page with sample prompts at /omega
- Build chat interface with real-time streaming at /omega/[conversationId]
- Integrate @tpmjs/registry-search and @tpmjs/registry-execute packages
- Use OpenAI GPT-4.1 Mini as the default model
2026-01-23 09:03:10 +10:00
Ajax Davis
d2740c3f69 feat: add usage.md endpoint for collection scenarios
- Create /:username/collections/:slug/usage.md endpoint
- Shows real-world usage patterns from test scenarios
- Filters out health check scenarios automatically
- Groups examples by tags for better organization
- Update skills.md to reference usage.md for usage examples
2026-01-23 06:05:40 +10:00
Ajax Davis
5d00f2a711 feat: add chunked skills.md generation for large collections
Implements batched generation to avoid Vercel 120s timeout for collections
with 60+ tools. Uses per-tool caching and recursive serverless invocations.

- Add ToolSkillsCache and SkillsGenerationJob models to schema
- Create tool-skills-generator.ts for per-tool markdown generation
- Create skills-summary-generator.ts for final pass summary/intro
- Update route handler with chunked generation logic
- Small collections (<20 tools) use original monolithic approach
- Large collections use 10-tool batches with progress tracking
2026-01-21 16:26:06 +10:00
Ajax Davis
514ea3c0db chore: increase max file size to 100KB for package source fetcher 2026-01-21 15:30:03 +10:00
Ajax Davis
9ab8946c9d feat: add skills.md endpoint and CLI run command
- Add /:username/collections/:slug/skills.md endpoint for AI-generated skills docs
- Analyze npm package source code to generate comprehensive capability contracts
- Add `tpm run` CLI command to execute tools from collections via MCP
- Cache generated skills.md in database with 1-week TTL
2026-01-21 14:50:07 +10:00
Ajax Davis
c5a17a0a69 Add architecture docs and humanize use case copy 2026-01-21 07:37:04 +10:00
Ajax Davis
ad974239fd feat: add interactive architecture diagram page 2026-01-20 19:52:31 +10:00
Ajax Davis
474e7f9cda Fix review response typing and usage date rendering 2026-01-20 17:49:28 +10:00
Ajax Davis
48c41e8733 feat: add use cases marketing product with AI-generated content
Transform qualifying scenarios into marketing-ready use cases with:
- AI-generated titles, descriptions, ROI estimates, business value
- Persona/industry/category taxonomy for targeting
- Browseable feed with filtering and ranking
- SEO-optimized case study pages
- Daily cron job for generation and ranking

Database:
- Add Persona, Industry, Category lookup tables
- Add UseCase model with marketing content fields
- Add junction tables for personas/industries/categories
- Add SocialProof model for cached metrics

API:
- GET /api/use-cases - Global directory with filtering
- GET /api/use-cases/[id] - Individual use case details
- GET /api/public/users/[username]/collections/[slug]/use-cases
- POST /api/cron/use-cases - Nightly generation job

Frontend:
- /use-cases - Global feed with persona dropdown
- /use-cases/[slug] - SEO case study page
- /[username]/collections/[slug]/use-cases - Collection feed
- UseCasesFeed component - Sortable table component
- UseCaseCaseStudy component - Full case study layout
2026-01-20 16:17:35 +10:00
Ajax Davis
5fafc90e3e fix: reorder collection page sections and add layout to docs page
- Move use cases section above scenarios section on collection detail pages
- Add AppHeader and AppFooter to /docs/developers/guide page for consistent layout
2026-01-20 14:15:16 +10:00
Ajax Davis
92e4f5c467 chore: remove old vitest .ts config files
Replaced with .mjs versions for proper ESM support.
2026-01-20 13:54:25 +10:00
Ajax Davis
01b7295daa fix: resolve build errors and clean up scenario page
- Remove undefined variable reference (run.conversation) in header
- Extract ExpandedRunDetails component to reduce JSX nesting
- Fix vitest configs: rename to .mjs and add ESM-compatible __dirname
- Inline tailwind base config to avoid module resolution issues
- Remove unused imports (Streamdown, viewMode state)

This fixes the Turbopack parsing error that was preventing the build.
2026-01-20 13:54:24 +10:00
Ajax Davis
f5d7364c96 fix: resolve tailwind config import issues
- Inline tailwind config to resolve Turbopack import issues with @tpmjs/config
- Add exports field to @tpmjs/config package.json for proper module resolution
- Add @tpmjs/config to transpilePackages in Next.js config

Note: page.tsx has a pre-existing Turbopack parsing error at line 604
that needs to be addressed separately (it existed before these changes).
2026-01-20 12:12:16 +10:00
Ajax Davis
ffc5807ba0 feat: add E2B sandbox tools and update documentation navigation
- Add new e2b tool package for cloud sandbox code execution
- Update docs navigation with descriptions and developers guide section
- Configure blocks.yml for e2b tool discovery
2026-01-20 08:05:19 +10:00
Ajax Davis
a3436ca5a5 feat: add developers guide to documentation menu
- Add new /docs/developers/guide page with comprehensive scenarios documentation
- Explain what scenarios are, why use them, and how they work
- Cover developer use cases (CI/CD, local testing, quality monitoring)
- Include comparison with traditional testing approaches
- Add to AppHeader developers dropdown menu
2026-01-20 06:49:09 +10:00
Ajax Davis
cf2fd3ad9e feat: add conversation history and view mode toggle to scenario runs
- Add Streamdown for markdown rendering
- Add Message interface for conversation typing
- Add viewMode state (chat/debug) for switching views
- Add conversation history section in expanded run details
- Show conversation in chat format (USER/ASSISTANT/TOOL messages)
- Add view mode toggle to switch between chat and raw JSON views
- Improve usage stats section to show — when data is missing
2026-01-20 06:30:33 +10:00
Ajax Davis
8964e0c237 fix: handle null/undefined evaluator object in scenario runs
- Make evaluator field nullable in TypeScript interface
- Add optional chaining for run.evaluator?.verdict
- Add optional chaining for run.evaluator?.model
- Prevents TypeError when evaluator object is missing
2026-01-20 06:11:23 +10:00
Ajax Davis
2d939ea11b fix: handle null/undefined usage object in scenario runs
- Make usage field nullable in TypeScript interface
- Add optional chaining for run.usage?.executionTimeMs
- Add optional chaining for run.usage?.totalTokens
- Prevents TypeError when usage object is missing
2026-01-20 06:04:58 +10:00
Ajax Davis
474e11fc13 fix: handle null timestamps in scenario details page
- Add optional chaining for run.timestamps?.createdAt
- Make timestamps.createdAt optional in interface
- Add null check for evaluator.reason
- Prevents TypeError when accessing undefined properties
2026-01-20 05:40:33 +10:00
Ajax Davis
ac6fdb87e7 feat(web): add tweet button and OG tags to collection page
- Add ShareButton component for Twitter/X sharing
- Refactor collection page to server component with generateMetadata
- Add proper OpenGraph and Twitter Card meta tags for social sharing
- Extract client-side logic to CollectionDetailClient component
2026-01-19 18:16:17 +10:00
Ajax Davis
3c5c218207 feat(web): add scenarios UI - explorer, detail, and homepage section
- Add global scenarios explorer page at /scenarios
- Add scenario detail page with run history at /scenarios/[id]
- Add collection-scoped scenario detail page
- Add featured scenarios section to homepage
- Add useScenarios hook for data fetching
- Regenerate CLI manifest
2026-01-19 05:57:29 +10:00
Ajax Davis
ad2f8a629b feat(web): add ScenariosSection component to collection page
- Create ScenariosSection component with scenario list, status badges, and metrics
- Allow generating new scenarios with AI (owner only)
- Allow running scenarios and showing run progress
- Display quality scores and pass/fail streaks
- Link to scenario detail pages for run history
2026-01-19 05:43:43 +10:00
Ajax Davis
ee3dd3dcd0 feat(scenarios): implement real agent execution with collection tools
- Replace simulated execution with real AI SDK generateText() calls
- Build tools from collection with executor config cascade
- Support multi-step tool execution with MAX_TOOL_STEPS limit
- Capture full conversation history, token usage, and execution time
- Use gpt-4.1-mini for consistent execution model
2026-01-19 05:37:59 +10:00
Ajax Davis
a1f45fa33a fix: switch scenario evaluator to gpt-4.1-mini
Changed default evaluator from claude-3-5-haiku-latest to gpt-4.1-mini
since OPENAI_API_KEY is configured in production but ANTHROPIC_API_KEY
is not.
2026-01-19 05:20:05 +10:00
Ajax Davis
c52e6eaf24 docs: add scenarios user guide and API reference
- Add /docs/scenarios page with comprehensive user guide
  - Overview of what scenarios are
  - Common archetypes for different tool types
  - CLI commands: generate, list, run, test, info
  - Quality scoring explanation
  - CI/CD integration examples
  - Rate limits documentation

- Add /docs/api/scenarios page with API reference
  - List scenarios endpoint
  - Get scenario details
  - List collection scenarios
  - Create scenario
  - Generate scenarios with AI
  - Run scenario
  - Get run history
  - Check prompt similarity
  - Featured scenarios
  - Error responses
2026-01-18 09:53:36 +10:00
Ajax Davis
ea742c9386 feat: add scenarios system for collection testing
Implements the complete scenarios feature for TPMJS:

API endpoints:
- GET/POST /api/scenarios - list/create scenarios
- GET/PATCH/DELETE /api/scenarios/:id - scenario CRUD
- POST /api/scenarios/:id/run - execute scenario
- GET /api/scenarios/:id/runs - run history
- POST /api/scenarios/check-similarity - vector similarity check
- GET /api/scenarios/featured - featured scenarios
- POST /api/collections/:id/scenarios/generate - AI scenario generation
- GET /api/collections/:id/scenarios - list collection scenarios

Services:
- generate-prompt.ts - AI prompt generation using GPT-4o-mini
- similarity.ts - vector embedding and cosine similarity
- evaluate.ts - LLM-based success evaluation
- execute.ts - scenario execution orchestration

CLI commands:
- tpm scenario list [collection] - list scenarios
- tpm scenario run <collection> - run all scenarios
- tpm scenario test <id> - run single scenario
- tpm scenario generate <collection> - generate scenarios
- tpm scenario info <id> - scenario details

Database:
- Scenario, ScenarioEmbedding, ScenarioRun, ScenarioQuota models
- Streak-based quality scoring
- Daily quota management

Migration script included for converting existing useCases.
2026-01-18 09:31:31 +10:00
Ajax Davis
154f000505 feat(cli): add @tpmjs/cli package and browser auth page
- Create comprehensive CLI with oclif framework
- Add commands: auth, tool, agent, collection, mcp, publish
- Add doctor, playground, and update commands
- Add /cli/auth page for browser-based OAuth flow
- Publish to npm as @tpmjs/cli@0.1.2
2026-01-18 06:12:34 +10:00
Ajax Davis
aa438078f9 feat: add hidden /features page for development 2026-01-18 02:13:23 +10:00
Ajax Davis
028c87056a chore: temporarily disable FeaturesSection 2026-01-18 02:11:18 +10:00
Ajax Davis
5abc85dafc fix: replace non-existent icon names to fix production crash
Changed invalid icon names that were causing viewBox undefined error:
- shield → key
- zap → star
- users → user
- code → terminal
2026-01-18 02:06:17 +10:00