engineering/tpmjs
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2.2k

feat: add tool-request pipeline for automated tool creation

Browse source 
- Add pipeline spec at .claude/pipelines/tool-request.md
- Add workflow to trigger Claude on 'tool-request' label
- Add auto-close workflow for published issues (24h)
- Update claude.yml with write permissions and NPM_TOKEN
- Create labels: tool-request, claude-working, validation-failed, published, escalated

Pipeline flow:
1. Maintainer applies 'tool-request' label to issue
2. Claude analyzes, designs, implements, validates, publishes
3. Issue auto-closes 24h after successful publish
This commit is contained in:
Ajax Davis 2026-01-20 04:19:16 +10:00
parent 3acb6f49e5
commit ef95e10a1e
4 changed files with 397 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

248
.claude/pipelines/tool-request.md Normal file
View file

@ -0,0 +1,248 @@
# Tool Request Pipeline Specification
Automated pipeline for creating TPMJS tools from GitHub issues using Claude.
## Overview
When a maintainer applies the `tool-request` label to an issue, Claude automatically:
1. Analyzes the tool idea and designs the implementation
2. Determines the best package (existing or new)
3. Implements the tool with AI SDK v6
4. Validates using blocks CLI
5. Creates an auto-merge PR
6. Publishes to npm
7. Syncs to tpmjs.com registry
## Trigger
| Setting | Value |
|---------|-------|
| Label | `tool-request` |
| Who can apply | Maintainers only |
| Trigger mechanism | Label application triggers workflow, which comments `@claude` |
| Concurrency | Parallel execution allowed |
| Rate limit | None (trust maintainers) |
## Input Requirements
| Setting | Value |
|---------|-------|
| Input format | Accept vague ideas - Claude designs autonomously |
| Structured template | Not required |
| Clarification | Claude fills gaps autonomously, doesn't ask first |
| Mid-flight edits | Incorporate edits - check for changes at each step |
## Package Organization
| Setting | Value |
|---------|-------|
| Strategy | Hybrid - default to categories, allow functional cohesion exceptions |
| Package selection | Analyze all existing tools in candidate packages to find best fit |
| New vs existing | Claude decides based on functional cohesion analysis |
| blocks.yml access | Full access - Claude adds entries as part of workflow |
### Decision Logic for Package Selection
1. Search existing packages for functionally related tools
2. If strong match found (>70% conceptual overlap), add to existing package
3. If no match or tool is foundational for a new domain, create new package
4. Exception: tightly coupled tools (e.g., e2b-*) stay together regardless of category
## Validation & Iteration
| Setting | Value |
|---------|-------|
| Max attempts | 3 before escalating to human review |
| On failure | Iterate in-issue - Claude fixes and retries |
| Runtime test | Execute with sample inputs, capture output as screenshot |
| Tool restrictions | None - any valid tool that passes validation is allowed |
### Validation Steps
1. `pnpm blocks run <tool-name>` - domain rules and output measures
2. TypeScript compilation check
3. Execute tool with generated sample inputs
4. Verify output structure matches schema
5. Capture execution output as proof in issue comment
## Publishing
| Setting | Value |
|---------|-------|
| Branch strategy | Auto-merge PR - create for visibility, auto-merge if CI passes |
| Version bump | Minor (0.X.0) - new functionality = minor version |
| NPM auth | Use existing `NPM_TOKEN` secret |
| On publish failure | Comment explaining failure, wait for human to fix and re-trigger |
### PR Template
```markdown
## Tool: `<tool-name>`
**Package:** `@tpmjs/tools-<package>`
**Version:** `0.X.0` -> `0.Y.0`
### Description
<tool description>
### Implementation
- [ ] blocks.yml entry added
- [ ] Package files created
- [ ] Validation passed
- [ ] Runtime test passed
### Test Output
<screenshot of tool execution>
---
Auto-generated by Claude from #<issue-number>
```
## Post-Publish
| Setting | Value |
|---------|-------|
| Registry sync | Auto-sync - call `/api/sync/keyword` after publish |
| Verify listing | Confirm tool appears on tpmjs.com before reporting success |
| Collections | Standalone only - no auto-add |
| Duplicates | Propose enhancement to existing tool if duplicate detected |
## Status Tracking
### Labels (managed by Claude)
| Label | Meaning |
|-------|---------|
| `tool-request` | Initial trigger (applied by maintainer) |
| `claude-working` | Claude is actively processing |
| `validation-failed` | Validation failed, iterating |
| `published` | Successfully published to npm |
| `escalated` | Requires human intervention |
### Issue Lifecycle
1. Maintainer applies `tool-request` label
2. Workflow triggers, adds `claude-working` label
3. On validation failure: add `validation-failed`, retry (max 3x)
4. On success: remove other labels, add `published`
5. Keep issue open 24h for feedback
6. Auto-close after 24h
## Success Report
Full changelog posted to issue:
```markdown
## Tool Published Successfully
**Package:** `@tpmjs/tools-<package>@<version>`
**NPM:** https://www.npmjs.com/package/@tpmjs/tools-<package>
**Registry:** https://tpmjs.com/tool/@tpmjs/tools-<package>/<tool-name>
### Changes
- Added `<tool-name>` tool
- Updated blocks.yml
- Bumped version from X.Y.Z to X.Y+1.0
### Validation Results
<validation output>
### Test Execution
<screenshot of tool running with sample inputs>
### Files Changed
<file diff summary>
---
This issue will auto-close in 24 hours. Reply if you have feedback.
```
## Error Handling
| Scenario | Action |
|----------|--------|
| Validation fails 3x | Add `escalated` label, assign to maintainer with diagnostic info |
| NPM publish fails | Comment explaining failure, wait for human fix |
| Duplicate detected | Comment explaining existing tool, propose enhancement instead |
| blocks.yml conflict | Rebase and retry automatically |
| Issue edited mid-work | Detect changes, incorporate into implementation |
## Context & Memory
| Setting | Value |
|---------|-------|
| State tracking | Full conversation - Claude remembers entire issue thread |
| Previous attempts | Tracked within issue context |
| Cross-issue | No memory between different issues |
## Workflow File Structure
```yaml
name: Tool Request Pipeline
on:
issues:
types: [labeled]
jobs:
trigger-claude:
if: github.event.label.name == 'tool-request'
runs-on: ubuntu-latest
steps:
- name: Add working label
uses: actions/github-script@v7
with:
script: |
await github.rest.issues.addLabels({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
labels: ['claude-working']
});
- name: Comment to trigger Claude
uses: actions/github-script@v7
with:
script: |
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: '@claude Please implement this tool request following the tool-request pipeline specification at `.claude/pipelines/tool-request.md`.'
});
```
## Claude Instructions
When triggered, Claude should:
1. **Read this spec** at `.claude/pipelines/tool-request.md`
2. **Analyze the issue** - extract tool name, description, intended functionality
3. **Check for duplicates** - search existing tools for similar functionality
4. **Select package** - analyze existing packages, decide new vs existing
5. **Design the tool** - define inputs, outputs, implementation approach
6. **Implement** - create/update blocks.yml, create package files
7. **Validate** - run `pnpm blocks run <tool>` in packages/tools/official
8. **Test** - execute with sample inputs, capture output
9. **Create PR** - feature branch, include all changes
10. **Publish** - after CI passes, `npm publish`
11. **Sync** - trigger registry sync
12. **Report** - full changelog to issue
13. **Cleanup** - update labels, schedule auto-close
## Security Considerations
- Only maintainers can apply trigger label
- NPM_TOKEN is existing secret, not exposed in logs
- Tool code is reviewed via PR (even if auto-merged)
- No restrictions on tool types - trust validation + maintainer judgment
- Full audit trail in issue comments
## Dry Run
No dry run mode. Validation is sufficient safeguard. If testing needed, create a test issue and manually delete artifacts after.
---
*Specification created: 2026-01-19*
*Interview conducted with: @ajax*