engineering/tpmjs
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2.0k

feat: add tool-request pipeline for automated tool creation

Browse source 
- Add pipeline spec at .claude/pipelines/tool-request.md
- Add workflow to trigger Claude on 'tool-request' label
- Add auto-close workflow for published issues (24h)
- Update claude.yml with write permissions and NPM_TOKEN
- Create labels: tool-request, claude-working, validation-failed, published, escalated

Pipeline flow:
1. Maintainer applies 'tool-request' label to issue
2. Claude analyzes, designs, implements, validates, publishes
3. Issue auto-closes 24h after successful publish
This commit is contained in:
Ajax Davis 2026-01-20 04:19:16 +10:00
parent 3acb6f49e5
commit ef95e10a1e
4 changed files with 397 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

248
.claude/pipelines/tool-request.md Normal file
View file

@ -0,0 +1,248 @@
# Tool Request Pipeline Specification
Automated pipeline for creating TPMJS tools from GitHub issues using Claude.
## Overview
When a maintainer applies the `tool-request` label to an issue, Claude automatically:
1. Analyzes the tool idea and designs the implementation
2. Determines the best package (existing or new)
3. Implements the tool with AI SDK v6
4. Validates using blocks CLI
5. Creates an auto-merge PR
6. Publishes to npm
7. Syncs to tpmjs.com registry
## Trigger
| Setting | Value |
|---------|-------|
| Label | `tool-request` |
| Who can apply | Maintainers only |
| Trigger mechanism | Label application triggers workflow, which comments `@claude` |
| Concurrency | Parallel execution allowed |
| Rate limit | None (trust maintainers) |
## Input Requirements
| Setting | Value |
|---------|-------|
| Input format | Accept vague ideas - Claude designs autonomously |
| Structured template | Not required |
| Clarification | Claude fills gaps autonomously, doesn't ask first |
| Mid-flight edits | Incorporate edits - check for changes at each step |
## Package Organization
| Setting | Value |
|---------|-------|
| Strategy | Hybrid - default to categories, allow functional cohesion exceptions |
| Package selection | Analyze all existing tools in candidate packages to find best fit |
| New vs existing | Claude decides based on functional cohesion analysis |
| blocks.yml access | Full access - Claude adds entries as part of workflow |
### Decision Logic for Package Selection
1. Search existing packages for functionally related tools
2. If strong match found (>70% conceptual overlap), add to existing package
3. If no match or tool is foundational for a new domain, create new package
4. Exception: tightly coupled tools (e.g., e2b-*) stay together regardless of category
## Validation & Iteration
| Setting | Value |
|---------|-------|
| Max attempts | 3 before escalating to human review |
| On failure | Iterate in-issue - Claude fixes and retries |
| Runtime test | Execute with sample inputs, capture output as screenshot |
| Tool restrictions | None - any valid tool that passes validation is allowed |
### Validation Steps
1. `pnpm blocks run <tool-name>` - domain rules and output measures
2. TypeScript compilation check
3. Execute tool with generated sample inputs
4. Verify output structure matches schema
5. Capture execution output as proof in issue comment
## Publishing
| Setting | Value |
|---------|-------|
| Branch strategy | Auto-merge PR - create for visibility, auto-merge if CI passes |
| Version bump | Minor (0.X.0) - new functionality = minor version |
| NPM auth | Use existing `NPM_TOKEN` secret |
| On publish failure | Comment explaining failure, wait for human to fix and re-trigger |
### PR Template
```markdown
## Tool: `<tool-name>`
**Package:** `@tpmjs/tools-<package>`
**Version:** `0.X.0` -> `0.Y.0`
### Description
<tool description>
### Implementation
- [ ] blocks.yml entry added
- [ ] Package files created
- [ ] Validation passed
- [ ] Runtime test passed
### Test Output
<screenshot of tool execution>
---
Auto-generated by Claude from #<issue-number>
```
## Post-Publish
| Setting | Value |
|---------|-------|
| Registry sync | Auto-sync - call `/api/sync/keyword` after publish |
| Verify listing | Confirm tool appears on tpmjs.com before reporting success |
| Collections | Standalone only - no auto-add |
| Duplicates | Propose enhancement to existing tool if duplicate detected |
## Status Tracking
### Labels (managed by Claude)
| Label | Meaning |
|-------|---------|
| `tool-request` | Initial trigger (applied by maintainer) |
| `claude-working` | Claude is actively processing |
| `validation-failed` | Validation failed, iterating |
| `published` | Successfully published to npm |
| `escalated` | Requires human intervention |
### Issue Lifecycle
1. Maintainer applies `tool-request` label
2. Workflow triggers, adds `claude-working` label
3. On validation failure: add `validation-failed`, retry (max 3x)
4. On success: remove other labels, add `published`
5. Keep issue open 24h for feedback
6. Auto-close after 24h
## Success Report
Full changelog posted to issue:
```markdown
## Tool Published Successfully
**Package:** `@tpmjs/tools-<package>@<version>`
**NPM:** https://www.npmjs.com/package/@tpmjs/tools-<package>
**Registry:** https://tpmjs.com/tool/@tpmjs/tools-<package>/<tool-name>
### Changes
- Added `<tool-name>` tool
- Updated blocks.yml
- Bumped version from X.Y.Z to X.Y+1.0
### Validation Results
<validation output>
### Test Execution
<screenshot of tool running with sample inputs>
### Files Changed
<file diff summary>
---
This issue will auto-close in 24 hours. Reply if you have feedback.
```
## Error Handling
| Scenario | Action |
|----------|--------|
| Validation fails 3x | Add `escalated` label, assign to maintainer with diagnostic info |
| NPM publish fails | Comment explaining failure, wait for human fix |
| Duplicate detected | Comment explaining existing tool, propose enhancement instead |
| blocks.yml conflict | Rebase and retry automatically |
| Issue edited mid-work | Detect changes, incorporate into implementation |
## Context & Memory
| Setting | Value |
|---------|-------|
| State tracking | Full conversation - Claude remembers entire issue thread |
| Previous attempts | Tracked within issue context |
| Cross-issue | No memory between different issues |
## Workflow File Structure
```yaml
name: Tool Request Pipeline
on:
issues:
types: [labeled]
jobs:
trigger-claude:
if: github.event.label.name == 'tool-request'
runs-on: ubuntu-latest
steps:
- name: Add working label
uses: actions/github-script@v7
with:
script: |
await github.rest.issues.addLabels({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
labels: ['claude-working']
});
- name: Comment to trigger Claude
uses: actions/github-script@v7
with:
script: |
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: '@claude Please implement this tool request following the tool-request pipeline specification at `.claude/pipelines/tool-request.md`.'
});
```
## Claude Instructions
When triggered, Claude should:
1. **Read this spec** at `.claude/pipelines/tool-request.md`
2. **Analyze the issue** - extract tool name, description, intended functionality
3. **Check for duplicates** - search existing tools for similar functionality
4. **Select package** - analyze existing packages, decide new vs existing
5. **Design the tool** - define inputs, outputs, implementation approach
6. **Implement** - create/update blocks.yml, create package files
7. **Validate** - run `pnpm blocks run <tool>` in packages/tools/official
8. **Test** - execute with sample inputs, capture output
9. **Create PR** - feature branch, include all changes
10. **Publish** - after CI passes, `npm publish`
11. **Sync** - trigger registry sync
12. **Report** - full changelog to issue
13. **Cleanup** - update labels, schedule auto-close
## Security Considerations
- Only maintainers can apply trigger label
- NPM_TOKEN is existing secret, not exposed in logs
- Tool code is reviewed via PR (even if auto-merged)
- No restrictions on tool types - trust validation + maintainer judgment
- Full audit trail in issue comments
## Dry Run
No dry run mode. Validation is sufficient safeguard. If testing needed, create a test issue and manually delete artifacts after.
---
*Specification created: 2026-01-19*
*Interview conducted with: @ajax*

66
.github/workflows/auto-close-published.yml vendored Normal file
View file

@ -0,0 +1,66 @@
name: Auto-Close Published Issues
on:
schedule:
# Run every hour to check for issues to close
- cron: '0 * * * *'
workflow_dispatch: # Allow manual trigger
jobs:
auto-close:
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- name: Close published issues older than 24h
uses: actions/github-script@v7
with:
script: |
const { data: issues } = await github.rest.issues.listForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
labels: 'published',
state: 'open',
per_page: 100
});
const now = new Date();
const twentyFourHoursAgo = new Date(now.getTime() - 24 * 60 * 60 * 1000);
for (const issue of issues) {
// Find when 'published' label was added
const { data: events } = await github.rest.issues.listEvents({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue.number,
per_page: 100
});
const publishedEvent = events
.filter(e => e.event === 'labeled' && e.label?.name === 'published')
.sort((a, b) => new Date(b.created_at) - new Date(a.created_at))[0];
if (publishedEvent) {
const labeledAt = new Date(publishedEvent.created_at);
if (labeledAt < twentyFourHoursAgo) {
console.log(`Closing issue #${issue.number} - published ${labeledAt.toISOString()}`);
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue.number,
body: 'Auto-closing after 24 hours. The tool has been published successfully. Reopen if you encounter any issues.'
});
await github.rest.issues.update({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue.number,
state: 'closed',
state_reason: 'completed'
});
}
}
}

13
.github/workflows/claude.yml vendored
View file

@ -19,11 +19,11 @@ jobs:
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
issues: read
contents: write # Push branches, create commits
pull-requests: write # Create and manage PRs
issues: write # Manage labels, close issues
id-token: write
actions: read # Required for Claude to read CI results on PRs
actions: read # Read CI results on PRs
steps:
- name: Checkout repository
uses: actions/checkout@v4
@ -33,10 +33,13 @@ jobs:
- name: Run Claude Code
id: claude
uses: anthropics/claude-code-action@v1
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
CRON_SECRET: ${{ secrets.CRON_SECRET }}
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# This is an optional setting that allows Claude to read CI results on PRs
# Allow Claude to read CI results and manage issues/PRs
additional_permissions: |
actions: read

75
.github/workflows/tool-request.yml vendored Normal file
View file

@ -0,0 +1,75 @@
name: Tool Request Pipeline
on:
issues:
types: [labeled]
jobs:
trigger-claude:
# Only run when 'tool-request' label is added
if: github.event.label.name == 'tool-request'
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- name: Add working label
uses: actions/github-script@v7
with:
script: |
// Add 'claude-working' label to indicate processing has started
await github.rest.issues.addLabels({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
labels: ['claude-working']
});
- name: Comment to trigger Claude
uses: actions/github-script@v7
with:
script: |
const body = `@claude Please implement this tool request.
**Instructions:**
1. Read the pipeline specification at \`.claude/pipelines/tool-request.md\`
2. Follow all steps: analyze, design, implement, validate, test, publish
3. Update labels as you progress (remove \`claude-working\`, add \`published\` or \`validation-failed\`)
4. Post full changelog when complete
5. This issue will auto-close 24h after successful publish
**Issue Context:**
- Issue #${context.issue.number}
- Author: @${context.payload.issue.user.login}
- Created: ${context.payload.issue.created_at}
Begin implementation.`;
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: body
});
# Auto-close published issues after 24 hours
auto-close:
runs-on: ubuntu-latest
if: github.event.label.name == 'published'
permissions:
issues: write
steps:
- name: Schedule auto-close
uses: actions/github-script@v7
with:
script: |
// Add a comment about auto-close
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: '**Note:** This issue will auto-close in 24 hours. Reply if you have feedback or issues with the published tool.'
});
# Separate workflow handles the actual auto-close via scheduled job
# See: .github/workflows/auto-close-published.yml