Commit graph

18 commits

Author SHA1 Message Date
Ajax Davis
4fa8344b34 feat: replace hardcoded stats with real DB data and add view tracking
- Add PageView model for daily-bucketed view tracking
- Add viewCount fields to Tool, Collection, Agent models
- Add social proof fields to StatsSnapshot
- Add POST /api/track/view endpoint with IP-based dedup
- Add GET /api/activity/public endpoint for real activity stream
- Add /api/sync/view-rollup daily cron for aggregating views
- Expand stats-snapshot cron with new social proof queries
- Replace hardcoded homepage stats with real DB-driven props
- Add downloads to hero metrics strip
- Add PublicActivityStream component fetching real UserActivity
- Add useTrackView hook for tool, collection, agent detail pages
- Add forkCount column to collections and agents listings
- Add views/reviews to tool detail statistics sidebar
- Remove deprecated hardcoded statistics and categories from homePageData
2026-02-10 02:27:14 +10:00
Thomas Davis
198f9f7d1e fix: separate sync discovery from enrichment and harden Railway executor
Sync system was timing out because discovery endpoints (keyword, changes)
also ran schema extraction (~10-15s per tool). Now discovery is fast
(npm metadata + DB writes only) and a new /api/sync/enrich endpoint
handles schema extraction in time-budgeted chunks.

Railway executor was crashing without restarting due to unhandled promise
rejections, no restart policy, and no health checks. Added crash
protection, graceful shutdown, cache size limits, railway.toml with
ALWAYS restart policy, and upgraded Deno from 1.39 to 2.1.9.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 04:49:54 +10:00
Ajax Davis
48c41e8733 feat: add use cases marketing product with AI-generated content
Transform qualifying scenarios into marketing-ready use cases with:
- AI-generated titles, descriptions, ROI estimates, business value
- Persona/industry/category taxonomy for targeting
- Browseable feed with filtering and ranking
- SEO-optimized case study pages
- Daily cron job for generation and ranking

Database:
- Add Persona, Industry, Category lookup tables
- Add UseCase model with marketing content fields
- Add junction tables for personas/industries/categories
- Add SocialProof model for cached metrics

API:
- GET /api/use-cases - Global directory with filtering
- GET /api/use-cases/[id] - Individual use case details
- GET /api/public/users/[username]/collections/[slug]/use-cases
- POST /api/cron/use-cases - Nightly generation job

Frontend:
- /use-cases - Global feed with persona dropdown
- /use-cases/[slug] - SEO case study page
- /[username]/collections/[slug]/use-cases - Collection feed
- UseCasesFeed component - Sortable table component
- UseCaseCaseStudy component - Full case study layout
2026-01-20 16:17:35 +10:00
Ajax Davis
938a382aab feat: add environment variables support for agents and collections
- Add envVars field to Agent and Collection models in Prisma schema
- Add envVars to UpdateAgentSchema and UpdateCollectionSchema types
- Implement env vars merging logic (agent overrides collection)
- Pass env vars through tool executor to sandbox
- Add UI for managing env vars in agent and collection settings pages
- Update API routes to handle envVars PATCH updates
- Refactor discord tool execute functions for biome compatibility

This allows users to configure tool-specific environment variables (like
DISCORD_BOT_TOKEN) at both the collection and agent level, with agent
settings taking precedence over collection settings.
2026-01-12 01:35:29 +10:00
Ajax Davis
bc5eb61086 revert: restore framework=nextjs in vercel.json 2026-01-09 18:18:04 +10:00
Ajax Davis
849802bfeb fix: set framework to null for monorepo Vercel deployment 2026-01-09 18:11:26 +10:00
Ajax Davis
d6de3e025a feat: implement user activity stream and multiple API improvements
Activity Stream:
- Add UserActivity model with ActivityType enum to track user actions
- Create activity logging service with fire-and-forget pattern
- Add /api/user/activity endpoint with cursor pagination
- Add cleanup cron job for 90-day activity retention
- Integrate activity logging into 18 mutation API routes
- Add DashboardActivityStream component with virtualized rendering

API Improvements:
- Add distributed rate limiting via Vercel KV (with in-memory fallback)
- Add rate limiting to chat endpoint (30 req/min)
- Optimize BM25 search with database-level pre-filtering
- Fix JSON parse crash in search endpoint
- Add pagination to collection tools response (toolsLimit/toolsOffset)
- Add take limits to agent detail query to prevent excessive data fetch
- Fix hardcoded tool count calculation in agents dashboard

Schema Extraction:
- Add schemaExtractionAttemptAt and schemaExtractionError fields
- Separate rate limiting for failed (1 min) vs successful (1 hour) attempts
- Allow retry of failed extractions

Standardization:
- Create api-response.ts with standardized response helpers
- Add apiSuccess, apiError, apiNotFound, apiForbidden, etc.
- Update agents routes to use standardized format

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-08 04:11:01 +10:00
Ajax Davis
1d771efef6 fix: add build config to root vercel.json for tpmjs project 2026-01-03 14:22:07 +10:00
Ajax Davis
fd4520dbf6 perf: reduce Neon compute usage with cron + caching optimizations
- Reduce cron frequency: changes 2min→4hr, keyword 15min→6hr, metrics hourly→daily
- Add Prisma directUrl for connection pooling support
- Add Vercel KV caching to /api/tools endpoint (graceful degradation if not configured)
- Add X-Cache header to indicate cache hit/miss
- Add NEON_COMPUTE_OPTIMIZATION.md with full strategy guide

These changes should reduce Neon CU usage from 100+ to ~20-30 CU-hrs/month.
2025-12-31 12:14:00 +10:00
Ajax Davis
61c89824f9 feat: add historical stats tracking with daily snapshots
- Add StatsSnapshot model to store daily registry metrics
- Create /api/sync/stats-snapshot endpoint for daily cron captures
- Add GET endpoint to retrieve historical snapshots (up to 365 days)
- Update stats page with Historical Trends section showing:
  - Tools & packages growth over time
  - Health status trends
  - Daily executions history
  - NPM downloads trends
- Add cron job running at midnight UTC daily
- Fix PostgreSQL GROUP BY issue in quality distribution query
2025-12-28 17:19:46 +10:00
Ajax Davis
c20ee3df04 feat: add HN launch readiness features
Launch checklist implementation:
- Add Privacy Policy page (/privacy) with GDPR compliance
- Add Terms of Service page (/terms)
- Add custom 404 and error pages with helpful navigation
- Add FAQ page (/faq) covering common questions
- Add SEO meta tags with OpenGraph/Twitter cards
- Add JSON-LD structured data (Organization, WebSite, SoftwareApplication)
- Add sitemap.ts and robots.ts for search engines
- Add security headers (HSTS, CSP, X-Frame-Options) in vercel.json
- Add security.txt at /.well-known/security.txt
- Add API rate limiting (100 req/min default, 20 req/min strict)
- Add empty states in tool search for better UX
- Update AppHeader with FAQ link
- Update AppFooter with Privacy/Terms links
- Update biome.json to allow dangerouslySetInnerHTML for JSON-LD in page files

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-14 11:56:42 +10:00
Ajax Davis
958697f788 feat: add GitHub Action for daily health check cron and backfill script
- Create .github/workflows/health-check.yml to run daily at 2am UTC
- Add scripts/backfill-health-checks.ts to populate health data for existing tools
- Remove health-check from vercel.json crons (now using GitHub Actions)

The GitHub Action workflow follows the same pattern as other sync operations
and calls the /api/sync/health-check endpoint with proper authentication.

The backfill script:
- Fetches all tools from database
- Runs batch health checks with concurrency control (5 tools at a time)
- Shows detailed progress and summary statistics
- Lists broken tools with error details

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 16:36:55 +10:00
Ajax Davis
e2af4cfd6a feat: implement health check system (Phase 1 & 2)
Add comprehensive health monitoring for TPMJS tools that tracks both
import and execution health via Railway executor service.

## Database Schema

- Add HealthStatus enum (UNKNOWN, HEALTHY, BROKEN)
- Add HealthCheckType enum (IMPORT, EXECUTION, FULL)
- Add health fields to Tool model:
  - importHealth: tracks if tool can be loaded
  - executionHealth: tracks if tool can execute
  - lastHealthCheck: timestamp of last check
  - healthCheckError: stores error message
- Add HealthCheck audit table for full history

## Core Service

Create health-check-service.ts with 5 functions:
1. checkImportHealth() - Tests tool loading via /load-and-describe
2. checkExecutionHealth() - Tests execution via /execute-tool
3. generateTestParameters() - Creates minimal test params by type
4. performHealthCheck() - Full check with database updates
5. performBatchHealthCheck() - Processes tools in batches

Features:
- 30-second timeout per check
- Skips execution if import fails
- Batch processing (5 concurrent, 1s delays)
- Full audit trail in HealthCheck table

## API Endpoints

/api/sync/health-check (POST):
- Daily cron job at 2am UTC
- Checks all tools in database
- Requires CRON_SECRET auth
- Logs results to SyncLog table
- Max duration: 5 minutes

/api/tools/broken (GET):
- Lists all tools with broken health status
- Filters by importHealth='BROKEN' OR executionHealth='BROKEN'
- Includes package metadata
- Orders by lastHealthCheck DESC

## Configuration

- Add RAILWAY_EXECUTOR_URL to env.ts
- Add daily cron job to vercel.json
- Use db:push for schema changes (existing production data)

Next: Manual trigger endpoint + health filtering + UI components

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 16:03:21 +10:00
Ajax Davis
9be3af7427 fix(routing): move www redirect from vercel.json to Next.js config
Root cause: The redirect rule in /vercel.json was intercepting ALL requests
(including /api/*) before they reached the Next.js app, causing API routes
to timeout without generating any logs.

Changes:
- Move www → non-www redirect to apps/web/next.config.ts async redirects()
- Remove redirects array from root vercel.json
- Keep cron job configuration in root vercel.json

Why this fixes the issue:
- Next.js handles redirects AFTER route resolution (pages vs API routes)
- API routes execute before redirect logic is applied
- Redirects still work for pages as expected
- More idiomatic for Next.js applications

Testing:
- API routes should now respond: /api/health, /api/tools
- www.tpmjs.com should still redirect to tpmjs.com
- All existing functionality preserved

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-28 20:25:52 +10:00
Ajax Davis
f3af742207 fix(config): redirect www subdomain to apex domain
- Add redirect rule in vercel.json to redirect www.tpmjs.com → tpmjs.com
- Use permanent (301) redirect for SEO benefits
- Preserves all paths and query parameters with /:path* pattern
- Ensures canonical URL is always non-www

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-28 05:13:55 +10:00
Ajax Davis
3b95502577 feat(sync): implement Phase 3 Sync Workers with Vercel Cron integration
Add complete sync worker system for NPM package discovery and metrics:

**Sync Workers:**
- Changes Feed Sync (/api/sync/changes) - Polls NPM changes every 2 min
- Keyword Search Sync (/api/sync/keyword) - Searches tpmjs-tool keyword every 15 min
- Metrics Sync (/api/sync/metrics) - Updates downloads & quality scores hourly

**Features:**
- Secure CRON_SECRET authentication for all sync endpoints
- Comprehensive error handling with sync logs and checkpoints
- Smart package validation and filtering (skip invalid tpmjs fields)
- Automatic tool upsert with discovery method tracking
- Quality score calculation based on tier, downloads, and GitHub stars
- 5-minute timeout support for long-running sync operations

**Dependencies:**
- Add @tpmjs/npm-client to web app for NPM API integration
- Use barrel exports from npm-client package (no subpath imports)
- Add CRON_SECRET env variable validation
- Add ~/src path alias to tsconfig

**Infrastructure:**
- Configure Vercel Cron jobs in vercel.json for automated syncing
- Add publishedAt field to PackageVersion schema
- Fix Prisma JSON field handling (use undefined instead of null)
- Proper null checks for fetchLatestPackageVersion return values

**Type Safety:**
- Cast TpmjsField union type to access optional rich-tier properties
- Handle searchByKeyword array return type correctly
- Fix fetchDownloadStats to return number directly

All API routes follow Next.js 16 conventions with proper type checking.
Type-check and full build successful.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-28 02:57:54 +10:00
Ajax Davis
e95bea6847 fix(lint): configure Biome to properly ignore build directories
- Add root biome.json that extends packages/config/biome.json
- Add explicit ignore patterns for dist/, build/, .next/, .turbo/, etc.
- Make a11y lints warnings instead of errors (non-blocking)
- Remove invalid biome-ignore comment from Section.tsx
- Apply formatting to all source files (155 files checked, 129 fixed)

This fixes the CI format check that was failing with 2064+ errors
because Biome was checking generated files in dist/ and .turbo/.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-26 22:49:45 +10:00
Ajax Davis
99998e5186 feat: configure Vercel to wait for CI before deploying
- Update GitHub Actions CI workflow to use Node 22
- Add architecture and deadcode checks to CI pipeline
- Create Vercel deployment guard script
- Add comprehensive deployment documentation
- Update README with CI badge and quality gates info
- Configure vercel.json for deployment settings

Vercel will now only deploy to production after all CI checks pass:
- Linting & formatting
- Type checking
- Tests
- Build verification
- Architecture validation
- Dead code detection

See DEPLOYMENT.md for configuration instructions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-26 17:42:23 +10:00