feat: add 100+ official TPMJS tools

Implements a comprehensive suite of AI SDK v6 tools across multiple categories:

- Research (5): page-brief, compare-pages, source-credibility, claim-checklist, timeline-from-text
- Web (10): fetch-text, links-catalog, extract-meta, extract-json-ld, redirect-trace, sitemap-read, rss-read, table-extract, robots-policy, url-normalize
- Data (15): csv-parse, csv-stringify, json-repair, json-schema-validate, yaml-parse, yaml-stringify, text-chunk, normalize-whitespace, dedupe-by-key, pivot, rows-filter, rows-sort, rows-group-aggregate, rows-join, schema-infer
- Doc (12): toc-generate, glossary-build, faq-from-text, executive-brief, decision-record-adr, prd-outline, acceptance-criteria, style-rewrite
- Eng (12): diff-text-unified, env-var-docs-generate, dependency-audit-lite, conventional-commit-suggest, markdown-lint-basic, test-case-generate, stacktrace-parse, release-notes, changelog-entry, release-checklist
- Security (7): redact-secrets, secret-scan-text, url-risk-heuristic, csp-compose, hardening-checklist-web, access-control-matrix, data-classification-heuristic
- Stats (9): effect-size-suite, bootstrap-ci, permutation-test, multiple-testing-adjust, linear-regression-ols, logistic-regression, time-series-decompose-lite, anomaly-detect-mad
- Ops (7): slo-draft, runbook-draft, postmortem-draft, postmortem-action-extractor, error-log-triage, coverage-tracker, monitoring-gap-analysis
- Agent (15): prompt-to-workflow-skeleton, workflow-validate-io, workflow-explain, workflow-cost-estimate, tool-call-accuracy-score, eval-fixture-build, guardrail-policy-draft, workflow-auto-repair, tool-selection-plan, novelty-score-workflow, workflow-variant-generate, config-normalize, recipe-*
- Utility (8): base64-encode, base64-decode, hash-text, regex-extract, template-render, date-parse, json-path-query, url-parse
- HTML (3): html-sanitize, html-to-markdown, markdown-to-html

All tools follow AI SDK v6 pattern with tool() and jsonSchema<T>().

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Ajax Davis 2025-12-31 22:55:56 +10:00
parent 2d9b06020d
commit 5d2096fb5d
499 changed files with 50782 additions and 238 deletions

View file

@ -0,0 +1,152 @@
# @tpmjs/tools-normalize-whitespace
Normalize whitespace in text by trimming lines, collapsing spaces, and standardizing line endings.
## Installation
```bash
npm install @tpmjs/tools-normalize-whitespace
```
## Usage
```typescript
import { normalizeWhitespaceTool } from '@tpmjs/tools-normalize-whitespace';
// Use with AI SDK
const result = await normalizeWhitespaceTool.execute({
text: ' Hello World \n This is a test \r\n',
options: {
trimLines: true,
collapseSpaces: true,
normalizeLineEndings: true
}
});
console.log(result.text);
// "Hello World\nThis is a test"
console.log(result.changes);
// {
// linesTrimmed: 2,
// spacesCollapsed: 5,
// lineEndingsNormalized: 1,
// originalLength: 44,
// normalizedLength: 24
// }
```
## Features
- **Trim Lines**: Remove leading and trailing whitespace from each line
- **Collapse Spaces**: Replace multiple consecutive spaces with a single space
- **Normalize Line Endings**: Convert CRLF (`\r\n`) to LF (`\n`)
- **Change Tracking**: Reports detailed statistics about transformations applied
- **Configurable**: Enable/disable each normalization option independently
## Input Schema
| Parameter | Type | Required | Description |
|-----------|--------|----------|--------------------------------|
| text | string | Yes | The text to normalize |
| options | object | No | Normalization options (see below) |
### Options
| Option | Type | Default | Description |
|---------------------|---------|---------|------------------------------------------|
| trimLines | boolean | true | Trim whitespace from start/end of lines |
| collapseSpaces | boolean | true | Collapse multiple spaces into one |
| normalizeLineEndings| boolean | true | Convert CRLF to LF |
## Output Schema
```typescript
interface NormalizeWhitespaceResult {
text: string; // The normalized text
changes: {
linesTrimmed: number; // Number of lines that were trimmed
spacesCollapsed: number; // Number of spaces removed by collapsing
lineEndingsNormalized: number; // Number of CRLF converted to LF
originalLength: number; // Character count before normalization
normalizedLength: number; // Character count after normalization
};
}
```
## Examples
### Trim Lines Only
```typescript
const result = await normalizeWhitespaceTool.execute({
text: ' Hello \n World ',
options: {
trimLines: true,
collapseSpaces: false,
normalizeLineEndings: false
}
});
console.log(result.text);
// "Hello\nWorld"
```
### Collapse Spaces Only
```typescript
const result = await normalizeWhitespaceTool.execute({
text: 'Hello World Test',
options: {
trimLines: false,
collapseSpaces: true,
normalizeLineEndings: false
}
});
console.log(result.text);
// "Hello World Test"
```
### Normalize Line Endings Only
```typescript
const result = await normalizeWhitespaceTool.execute({
text: 'Line 1\r\nLine 2\r\nLine 3',
options: {
trimLines: false,
collapseSpaces: false,
normalizeLineEndings: true
}
});
console.log(result.text);
// "Line 1\nLine 2\nLine 3"
```
### All Options Enabled (Default)
```typescript
const result = await normalizeWhitespaceTool.execute({
text: ' Hello World \r\n Line 2 '
});
console.log(result.text);
// "Hello World\nLine 2"
console.log(result.changes);
// {
// linesTrimmed: 2,
// spacesCollapsed: 5,
// lineEndingsNormalized: 1,
// originalLength: 35,
// normalizedLength: 18
// }
```
## Use Cases
- **Data Cleaning**: Normalize text data from various sources
- **Configuration Files**: Clean up YAML/JSON/config file content
- **User Input**: Sanitize and normalize user-submitted text
- **Text Processing**: Prepare text for analysis or comparison
- **Code Formatting**: Normalize whitespace in code snippets
## License
MIT

View file

@ -0,0 +1,66 @@
{
"name": "@tpmjs/tools-normalize-whitespace",
"version": "0.1.0",
"description": "Normalize whitespace in text by trimming, collapsing spaces, and standardizing line endings",
"type": "module",
"keywords": ["tpmjs", "text", "data", "whitespace", "normalize"],
"exports": {
".": {
"types": "./dist/index.d.ts",
"default": "./dist/index.js"
}
},
"files": ["dist"],
"scripts": {
"build": "tsup",
"dev": "tsup --watch",
"type-check": "tsc --noEmit",
"clean": "rm -rf dist .turbo"
},
"devDependencies": {
"@tpmjs/tsconfig": "workspace:*",
"tsup": "^8.3.5",
"typescript": "^5.9.3"
},
"publishConfig": {
"access": "public"
},
"repository": {
"type": "git",
"url": "https://github.com/anthropics/tpmjs.git",
"directory": "packages/tools/official/normalize-whitespace"
},
"homepage": "https://tpmjs.com",
"license": "MIT",
"tpmjs": {
"category": "data",
"frameworks": ["vercel-ai"],
"tools": [
{
"name": "normalizeWhitespaceTool",
"description": "Normalize whitespace in text by trimming, collapsing spaces, and standardizing line endings",
"parameters": [
{
"name": "text",
"type": "string",
"description": "The text to normalize",
"required": true
},
{
"name": "options",
"type": "object",
"description": "Normalization options",
"required": false
}
],
"returns": {
"type": "NormalizeWhitespaceResult",
"description": "Object with normalized text and change statistics"
}
}
]
},
"dependencies": {
"ai": "6.0.0-beta.124"
}
}

View file

@ -0,0 +1,182 @@
/**
* Normalize Whitespace Tool for TPMJS
* Normalizes whitespace in text by trimming lines, collapsing spaces,
* and standardizing line endings
*/
import { jsonSchema, tool } from 'ai';
/**
* Options for whitespace normalization
*/
export interface NormalizeOptions {
trimLines?: boolean;
collapseSpaces?: boolean;
normalizeLineEndings?: boolean;
}
/**
* Statistics about changes made during normalization
*/
export interface WhitespaceChanges {
linesTrimmed: number;
spacesCollapsed: number;
lineEndingsNormalized: number;
originalLength: number;
normalizedLength: number;
}
/**
* Output interface for normalize whitespace result
*/
export interface NormalizeWhitespaceResult {
text: string;
changes: WhitespaceChanges;
}
type NormalizeWhitespaceInput = {
text: string;
options?: NormalizeOptions;
};
/**
* Default normalization options
*/
const DEFAULT_OPTIONS: Required<NormalizeOptions> = {
trimLines: true,
collapseSpaces: true,
normalizeLineEndings: true,
};
/**
* Normalizes line endings to \n (LF)
*/
function normalizeLineEndings(text: string): { text: string; count: number } {
let count = 0;
const normalized = text.replace(/\r\n/g, () => {
count++;
return '\n';
});
return { text: normalized, count };
}
/**
* Trims whitespace from the start and end of each line
*/
function trimLines(text: string): { text: string; count: number } {
const lines = text.split('\n');
let count = 0;
const trimmed = lines.map((line) => {
const before = line.length;
const trimmedLine = line.trim();
if (trimmedLine.length < before) {
count++;
}
return trimmedLine;
});
return { text: trimmed.join('\n'), count };
}
/**
* Collapses multiple consecutive spaces into a single space
*/
function collapseSpaces(text: string): { text: string; count: number } {
let count = 0;
const collapsed = text.replace(/ {2,}/g, (match) => {
count += match.length - 1;
return ' ';
});
return { text: collapsed, count };
}
/**
* Normalize Whitespace Tool
* Normalizes whitespace in text with configurable options
*/
export const normalizeWhitespaceTool = tool({
description:
'Normalize whitespace in text by trimming lines, collapsing multiple spaces, and standardizing line endings. Useful for cleaning up text data, formatting content, or preparing text for processing.',
inputSchema: jsonSchema<NormalizeWhitespaceInput>({
type: 'object',
properties: {
text: {
type: 'string',
description: 'The text to normalize',
},
options: {
type: 'object',
description: 'Normalization options',
properties: {
trimLines: {
type: 'boolean',
description: 'Trim whitespace from start and end of each line (default: true)',
},
collapseSpaces: {
type: 'boolean',
description: 'Collapse multiple consecutive spaces into one (default: true)',
},
normalizeLineEndings: {
type: 'boolean',
description: 'Convert all line endings to LF (\\n) (default: true)',
},
},
additionalProperties: false,
},
},
required: ['text'],
additionalProperties: false,
}),
async execute({ text, options = {} }): Promise<NormalizeWhitespaceResult> {
// Validate input
if (typeof text !== 'string') {
throw new Error('Text must be a string');
}
// Merge with default options
const opts: Required<NormalizeOptions> = {
...DEFAULT_OPTIONS,
...options,
};
// Track changes
const changes: WhitespaceChanges = {
linesTrimmed: 0,
spacesCollapsed: 0,
lineEndingsNormalized: 0,
originalLength: text.length,
normalizedLength: 0,
};
let normalized = text;
// Apply normalizations in order
if (opts.normalizeLineEndings) {
const result = normalizeLineEndings(normalized);
normalized = result.text;
changes.lineEndingsNormalized = result.count;
}
if (opts.collapseSpaces) {
const result = collapseSpaces(normalized);
normalized = result.text;
changes.spacesCollapsed = result.count;
}
if (opts.trimLines) {
const result = trimLines(normalized);
normalized = result.text;
changes.linesTrimmed = result.count;
}
changes.normalizedLength = normalized.length;
return {
text: normalized,
changes,
};
},
});
export default normalizeWhitespaceTool;

View file

@ -0,0 +1,11 @@
{
"extends": "@tpmjs/tsconfig/base.json",
"compilerOptions": {
"outDir": "dist",
"rootDir": "src",
"incremental": false,
"composite": false
},
"include": ["src"],
"exclude": ["node_modules", "dist"]
}

View file

@ -0,0 +1,10 @@
import { defineConfig } from 'tsup';
export default defineConfig({
entry: ['src/index.ts'],
format: ['esm'],
dts: true,
clean: true,
treeshake: true,
splitting: false,
});