5d2096fb5d
Implements a comprehensive suite of AI SDK v6 tools across multiple categories: - Research (5): page-brief, compare-pages, source-credibility, claim-checklist, timeline-from-text - Web (10): fetch-text, links-catalog, extract-meta, extract-json-ld, redirect-trace, sitemap-read, rss-read, table-extract, robots-policy, url-normalize - Data (15): csv-parse, csv-stringify, json-repair, json-schema-validate, yaml-parse, yaml-stringify, text-chunk, normalize-whitespace, dedupe-by-key, pivot, rows-filter, rows-sort, rows-group-aggregate, rows-join, schema-infer - Doc (12): toc-generate, glossary-build, faq-from-text, executive-brief, decision-record-adr, prd-outline, acceptance-criteria, style-rewrite - Eng (12): diff-text-unified, env-var-docs-generate, dependency-audit-lite, conventional-commit-suggest, markdown-lint-basic, test-case-generate, stacktrace-parse, release-notes, changelog-entry, release-checklist - Security (7): redact-secrets, secret-scan-text, url-risk-heuristic, csp-compose, hardening-checklist-web, access-control-matrix, data-classification-heuristic - Stats (9): effect-size-suite, bootstrap-ci, permutation-test, multiple-testing-adjust, linear-regression-ols, logistic-regression, time-series-decompose-lite, anomaly-detect-mad - Ops (7): slo-draft, runbook-draft, postmortem-draft, postmortem-action-extractor, error-log-triage, coverage-tracker, monitoring-gap-analysis - Agent (15): prompt-to-workflow-skeleton, workflow-validate-io, workflow-explain, workflow-cost-estimate, tool-call-accuracy-score, eval-fixture-build, guardrail-policy-draft, workflow-auto-repair, tool-selection-plan, novelty-score-workflow, workflow-variant-generate, config-normalize, recipe-* - Utility (8): base64-encode, base64-decode, hash-text, regex-extract, template-render, date-parse, json-path-query, url-parse - HTML (3): html-sanitize, html-to-markdown, markdown-to-html All tools follow AI SDK v6 pattern with tool() and jsonSchema<T>(). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
3.1 KiB
3.1 KiB
@tpmjs/tools-url-normalize
Normalize URLs with configurable options for consistent URL handling.
Installation
npm install @tpmjs/tools-url-normalize
Usage
import { urlNormalizeTool } from '@tpmjs/tools-url-normalize';
// Use with AI SDK
const result = await urlNormalizeTool.execute({
url: 'HTTPS://WWW.Example.COM:443/path/?z=1&a=2#section',
options: {
sortParams: true,
removeHash: true,
lowercase: true,
removeTrailingSlash: true,
removeDefaultPort: true,
removeWWW: true,
},
});
console.log(result.normalized);
// 'https://example.com/path?a=2&z=1'
console.log(result.changes);
// [
// { type: 'lowercase', description: '...', before: 'HTTPS:', after: 'https:' },
// { type: 'lowercase', description: '...', before: 'WWW.Example.COM', after: 'www.example.com' },
// { type: 'removeWWW', description: '...', before: 'www.example.com', after: 'example.com' },
// { type: 'removeDefaultPort', description: '...', before: '443', after: '' },
// { type: 'sortParams', description: '...', before: '?z=1&a=2', after: '?a=2&z=1' },
// { type: 'removeHash', description: '...', before: '#section', after: '' },
// ]
Features
- Lowercase conversion: Converts protocol and hostname to lowercase
- Trailing slash removal: Removes trailing slashes from pathnames
- Query parameter sorting: Sorts query parameters alphabetically
- Hash removal: Optionally removes URL fragments
- Default port removal: Removes standard ports (80, 443, 21)
- WWW removal: Optionally removes www subdomain
- Change tracking: Reports all transformations applied
Parameters
url(string, required): The URL to normalizeoptions(object, optional): Normalization optionssortParams(boolean, default: true): Sort query parameters alphabeticallyremoveHash(boolean, default: false): Remove URL fragment/hashlowercase(boolean, default: true): Convert protocol and hostname to lowercaseremoveTrailingSlash(boolean, default: true): Remove trailing slash from pathnameremoveDefaultPort(boolean, default: true): Remove default portsremoveWWW(boolean, default: false): Remove www subdomain
Returns
{
normalized: string;
original: string;
changes: Array<{
type: string;
description: string;
before: string;
after: string;
}>;
metadata: {
protocol: string;
hostname: string;
pathname: string;
hasQueryParams: boolean;
hasHash: boolean;
};
}
Use Cases
- URL deduplication in web crawlers
- Canonical URL generation for SEO
- Link comparison and matching
- Database indexing of URLs
- Analytics tracking consistency
Example with AI Agent
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';
import { urlNormalizeTool } from '@tpmjs/tools-url-normalize';
const result = await generateText({
model: openai('gpt-4'),
tools: {
urlNormalize: urlNormalizeTool,
},
prompt: 'Normalize this URL and remove the hash: https://Example.com/PATH/?b=2&a=1#top',
});
License
MIT