From ef95e10a1ef4a1691a9e452f1d4e658f21254fda Mon Sep 17 00:00:00 2001 From: Ajax Davis Date: Tue, 20 Jan 2026 04:19:16 +1000 Subject: [PATCH] feat: add tool-request pipeline for automated tool creation - Add pipeline spec at .claude/pipelines/tool-request.md - Add workflow to trigger Claude on 'tool-request' label - Add auto-close workflow for published issues (24h) - Update claude.yml with write permissions and NPM_TOKEN - Create labels: tool-request, claude-working, validation-failed, published, escalated Pipeline flow: 1. Maintainer applies 'tool-request' label to issue 2. Claude analyzes, designs, implements, validates, publishes 3. Issue auto-closes 24h after successful publish --- .claude/pipelines/tool-request.md | 248 +++++++++++++++++++++ .github/workflows/auto-close-published.yml | 66 ++++++ .github/workflows/claude.yml | 13 +- .github/workflows/tool-request.yml | 75 +++++++ 4 files changed, 397 insertions(+), 5 deletions(-) create mode 100644 .claude/pipelines/tool-request.md create mode 100644 .github/workflows/auto-close-published.yml create mode 100644 .github/workflows/tool-request.yml diff --git a/.claude/pipelines/tool-request.md b/.claude/pipelines/tool-request.md new file mode 100644 index 0000000..16a8359 --- /dev/null +++ b/.claude/pipelines/tool-request.md @@ -0,0 +1,248 @@ +# Tool Request Pipeline Specification + +Automated pipeline for creating TPMJS tools from GitHub issues using Claude. + +## Overview + +When a maintainer applies the `tool-request` label to an issue, Claude automatically: +1. Analyzes the tool idea and designs the implementation +2. Determines the best package (existing or new) +3. Implements the tool with AI SDK v6 +4. Validates using blocks CLI +5. Creates an auto-merge PR +6. Publishes to npm +7. Syncs to tpmjs.com registry + +## Trigger + +| Setting | Value | +|---------|-------| +| Label | `tool-request` | +| Who can apply | Maintainers only | +| Trigger mechanism | Label application triggers workflow, which comments `@claude` | +| Concurrency | Parallel execution allowed | +| Rate limit | None (trust maintainers) | + +## Input Requirements + +| Setting | Value | +|---------|-------| +| Input format | Accept vague ideas - Claude designs autonomously | +| Structured template | Not required | +| Clarification | Claude fills gaps autonomously, doesn't ask first | +| Mid-flight edits | Incorporate edits - check for changes at each step | + +## Package Organization + +| Setting | Value | +|---------|-------| +| Strategy | Hybrid - default to categories, allow functional cohesion exceptions | +| Package selection | Analyze all existing tools in candidate packages to find best fit | +| New vs existing | Claude decides based on functional cohesion analysis | +| blocks.yml access | Full access - Claude adds entries as part of workflow | + +### Decision Logic for Package Selection + +1. Search existing packages for functionally related tools +2. If strong match found (>70% conceptual overlap), add to existing package +3. If no match or tool is foundational for a new domain, create new package +4. Exception: tightly coupled tools (e.g., e2b-*) stay together regardless of category + +## Validation & Iteration + +| Setting | Value | +|---------|-------| +| Max attempts | 3 before escalating to human review | +| On failure | Iterate in-issue - Claude fixes and retries | +| Runtime test | Execute with sample inputs, capture output as screenshot | +| Tool restrictions | None - any valid tool that passes validation is allowed | + +### Validation Steps + +1. `pnpm blocks run ` - domain rules and output measures +2. TypeScript compilation check +3. Execute tool with generated sample inputs +4. Verify output structure matches schema +5. Capture execution output as proof in issue comment + +## Publishing + +| Setting | Value | +|---------|-------| +| Branch strategy | Auto-merge PR - create for visibility, auto-merge if CI passes | +| Version bump | Minor (0.X.0) - new functionality = minor version | +| NPM auth | Use existing `NPM_TOKEN` secret | +| On publish failure | Comment explaining failure, wait for human to fix and re-trigger | + +### PR Template + +```markdown +## Tool: `` + +**Package:** `@tpmjs/tools-` +**Version:** `0.X.0` -> `0.Y.0` + +### Description + + +### Implementation +- [ ] blocks.yml entry added +- [ ] Package files created +- [ ] Validation passed +- [ ] Runtime test passed + +### Test Output + + +--- +Auto-generated by Claude from # +``` + +## Post-Publish + +| Setting | Value | +|---------|-------| +| Registry sync | Auto-sync - call `/api/sync/keyword` after publish | +| Verify listing | Confirm tool appears on tpmjs.com before reporting success | +| Collections | Standalone only - no auto-add | +| Duplicates | Propose enhancement to existing tool if duplicate detected | + +## Status Tracking + +### Labels (managed by Claude) + +| Label | Meaning | +|-------|---------| +| `tool-request` | Initial trigger (applied by maintainer) | +| `claude-working` | Claude is actively processing | +| `validation-failed` | Validation failed, iterating | +| `published` | Successfully published to npm | +| `escalated` | Requires human intervention | + +### Issue Lifecycle + +1. Maintainer applies `tool-request` label +2. Workflow triggers, adds `claude-working` label +3. On validation failure: add `validation-failed`, retry (max 3x) +4. On success: remove other labels, add `published` +5. Keep issue open 24h for feedback +6. Auto-close after 24h + +## Success Report + +Full changelog posted to issue: + +```markdown +## Tool Published Successfully + +**Package:** `@tpmjs/tools-@` +**NPM:** https://www.npmjs.com/package/@tpmjs/tools- +**Registry:** https://tpmjs.com/tool/@tpmjs/tools-/ + +### Changes +- Added `` tool +- Updated blocks.yml +- Bumped version from X.Y.Z to X.Y+1.0 + +### Validation Results + + +### Test Execution + + +### Files Changed + + +--- +This issue will auto-close in 24 hours. Reply if you have feedback. +``` + +## Error Handling + +| Scenario | Action | +|----------|--------| +| Validation fails 3x | Add `escalated` label, assign to maintainer with diagnostic info | +| NPM publish fails | Comment explaining failure, wait for human fix | +| Duplicate detected | Comment explaining existing tool, propose enhancement instead | +| blocks.yml conflict | Rebase and retry automatically | +| Issue edited mid-work | Detect changes, incorporate into implementation | + +## Context & Memory + +| Setting | Value | +|---------|-------| +| State tracking | Full conversation - Claude remembers entire issue thread | +| Previous attempts | Tracked within issue context | +| Cross-issue | No memory between different issues | + +## Workflow File Structure + +```yaml +name: Tool Request Pipeline + +on: + issues: + types: [labeled] + +jobs: + trigger-claude: + if: github.event.label.name == 'tool-request' + runs-on: ubuntu-latest + steps: + - name: Add working label + uses: actions/github-script@v7 + with: + script: | + await github.rest.issues.addLabels({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: context.issue.number, + labels: ['claude-working'] + }); + + - name: Comment to trigger Claude + uses: actions/github-script@v7 + with: + script: | + await github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: context.issue.number, + body: '@claude Please implement this tool request following the tool-request pipeline specification at `.claude/pipelines/tool-request.md`.' + }); +``` + +## Claude Instructions + +When triggered, Claude should: + +1. **Read this spec** at `.claude/pipelines/tool-request.md` +2. **Analyze the issue** - extract tool name, description, intended functionality +3. **Check for duplicates** - search existing tools for similar functionality +4. **Select package** - analyze existing packages, decide new vs existing +5. **Design the tool** - define inputs, outputs, implementation approach +6. **Implement** - create/update blocks.yml, create package files +7. **Validate** - run `pnpm blocks run ` in packages/tools/official +8. **Test** - execute with sample inputs, capture output +9. **Create PR** - feature branch, include all changes +10. **Publish** - after CI passes, `npm publish` +11. **Sync** - trigger registry sync +12. **Report** - full changelog to issue +13. **Cleanup** - update labels, schedule auto-close + +## Security Considerations + +- Only maintainers can apply trigger label +- NPM_TOKEN is existing secret, not exposed in logs +- Tool code is reviewed via PR (even if auto-merged) +- No restrictions on tool types - trust validation + maintainer judgment +- Full audit trail in issue comments + +## Dry Run + +No dry run mode. Validation is sufficient safeguard. If testing needed, create a test issue and manually delete artifacts after. + +--- + +*Specification created: 2026-01-19* +*Interview conducted with: @ajax* diff --git a/.github/workflows/auto-close-published.yml b/.github/workflows/auto-close-published.yml new file mode 100644 index 0000000..3e11f5e --- /dev/null +++ b/.github/workflows/auto-close-published.yml @@ -0,0 +1,66 @@ +name: Auto-Close Published Issues + +on: + schedule: + # Run every hour to check for issues to close + - cron: '0 * * * *' + workflow_dispatch: # Allow manual trigger + +jobs: + auto-close: + runs-on: ubuntu-latest + permissions: + issues: write + + steps: + - name: Close published issues older than 24h + uses: actions/github-script@v7 + with: + script: | + const { data: issues } = await github.rest.issues.listForRepo({ + owner: context.repo.owner, + repo: context.repo.repo, + labels: 'published', + state: 'open', + per_page: 100 + }); + + const now = new Date(); + const twentyFourHoursAgo = new Date(now.getTime() - 24 * 60 * 60 * 1000); + + for (const issue of issues) { + // Find when 'published' label was added + const { data: events } = await github.rest.issues.listEvents({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: issue.number, + per_page: 100 + }); + + const publishedEvent = events + .filter(e => e.event === 'labeled' && e.label?.name === 'published') + .sort((a, b) => new Date(b.created_at) - new Date(a.created_at))[0]; + + if (publishedEvent) { + const labeledAt = new Date(publishedEvent.created_at); + + if (labeledAt < twentyFourHoursAgo) { + console.log(`Closing issue #${issue.number} - published ${labeledAt.toISOString()}`); + + await github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: issue.number, + body: 'Auto-closing after 24 hours. The tool has been published successfully. Reopen if you encounter any issues.' + }); + + await github.rest.issues.update({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: issue.number, + state: 'closed', + state_reason: 'completed' + }); + } + } + } diff --git a/.github/workflows/claude.yml b/.github/workflows/claude.yml index d300267..28dea23 100644 --- a/.github/workflows/claude.yml +++ b/.github/workflows/claude.yml @@ -19,11 +19,11 @@ jobs: (github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude'))) runs-on: ubuntu-latest permissions: - contents: read - pull-requests: read - issues: read + contents: write # Push branches, create commits + pull-requests: write # Create and manage PRs + issues: write # Manage labels, close issues id-token: write - actions: read # Required for Claude to read CI results on PRs + actions: read # Read CI results on PRs steps: - name: Checkout repository uses: actions/checkout@v4 @@ -33,10 +33,13 @@ jobs: - name: Run Claude Code id: claude uses: anthropics/claude-code-action@v1 + env: + NPM_TOKEN: ${{ secrets.NPM_TOKEN }} + CRON_SECRET: ${{ secrets.CRON_SECRET }} with: claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} - # This is an optional setting that allows Claude to read CI results on PRs + # Allow Claude to read CI results and manage issues/PRs additional_permissions: | actions: read diff --git a/.github/workflows/tool-request.yml b/.github/workflows/tool-request.yml new file mode 100644 index 0000000..3e57a96 --- /dev/null +++ b/.github/workflows/tool-request.yml @@ -0,0 +1,75 @@ +name: Tool Request Pipeline + +on: + issues: + types: [labeled] + +jobs: + trigger-claude: + # Only run when 'tool-request' label is added + if: github.event.label.name == 'tool-request' + runs-on: ubuntu-latest + permissions: + issues: write + + steps: + - name: Add working label + uses: actions/github-script@v7 + with: + script: | + // Add 'claude-working' label to indicate processing has started + await github.rest.issues.addLabels({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: context.issue.number, + labels: ['claude-working'] + }); + + - name: Comment to trigger Claude + uses: actions/github-script@v7 + with: + script: | + const body = `@claude Please implement this tool request. + +**Instructions:** +1. Read the pipeline specification at \`.claude/pipelines/tool-request.md\` +2. Follow all steps: analyze, design, implement, validate, test, publish +3. Update labels as you progress (remove \`claude-working\`, add \`published\` or \`validation-failed\`) +4. Post full changelog when complete +5. This issue will auto-close 24h after successful publish + +**Issue Context:** +- Issue #${context.issue.number} +- Author: @${context.payload.issue.user.login} +- Created: ${context.payload.issue.created_at} + +Begin implementation.`; + + await github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: context.issue.number, + body: body + }); + + # Auto-close published issues after 24 hours + auto-close: + runs-on: ubuntu-latest + if: github.event.label.name == 'published' + permissions: + issues: write + steps: + - name: Schedule auto-close + uses: actions/github-script@v7 + with: + script: | + // Add a comment about auto-close + await github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: context.issue.number, + body: '**Note:** This issue will auto-close in 24 hours. Reply if you have feedback or issues with the published tool.' + }); + + # Separate workflow handles the actual auto-close via scheduled job + # See: .github/workflows/auto-close-published.yml