tpmjs/packages/tools/official/diff-in-diff/README.md
Ajax Davis 5d2096fb5d feat: add 100+ official TPMJS tools
Implements a comprehensive suite of AI SDK v6 tools across multiple categories:

- Research (5): page-brief, compare-pages, source-credibility, claim-checklist, timeline-from-text
- Web (10): fetch-text, links-catalog, extract-meta, extract-json-ld, redirect-trace, sitemap-read, rss-read, table-extract, robots-policy, url-normalize
- Data (15): csv-parse, csv-stringify, json-repair, json-schema-validate, yaml-parse, yaml-stringify, text-chunk, normalize-whitespace, dedupe-by-key, pivot, rows-filter, rows-sort, rows-group-aggregate, rows-join, schema-infer
- Doc (12): toc-generate, glossary-build, faq-from-text, executive-brief, decision-record-adr, prd-outline, acceptance-criteria, style-rewrite
- Eng (12): diff-text-unified, env-var-docs-generate, dependency-audit-lite, conventional-commit-suggest, markdown-lint-basic, test-case-generate, stacktrace-parse, release-notes, changelog-entry, release-checklist
- Security (7): redact-secrets, secret-scan-text, url-risk-heuristic, csp-compose, hardening-checklist-web, access-control-matrix, data-classification-heuristic
- Stats (9): effect-size-suite, bootstrap-ci, permutation-test, multiple-testing-adjust, linear-regression-ols, logistic-regression, time-series-decompose-lite, anomaly-detect-mad
- Ops (7): slo-draft, runbook-draft, postmortem-draft, postmortem-action-extractor, error-log-triage, coverage-tracker, monitoring-gap-analysis
- Agent (15): prompt-to-workflow-skeleton, workflow-validate-io, workflow-explain, workflow-cost-estimate, tool-call-accuracy-score, eval-fixture-build, guardrail-policy-draft, workflow-auto-repair, tool-selection-plan, novelty-score-workflow, workflow-variant-generate, config-normalize, recipe-*
- Utility (8): base64-encode, base64-decode, hash-text, regex-extract, template-render, date-parse, json-path-query, url-parse
- HTML (3): html-sanitize, html-to-markdown, markdown-to-html

All tools follow AI SDK v6 pattern with tool() and jsonSchema<T>().

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-31 22:55:56 +10:00

3.8 KiB

Difference-in-Differences (DiD)

Causal inference estimator for measuring treatment effects using before/after comparison with a control group.

Installation

npm install @tpmjs/tools-diff-in-diff

Usage

import { diffInDiffTool } from '@tpmjs/tools-diff-in-diff';

// Example: Evaluate impact of a policy intervention
// Treatment group: Cities that implemented the policy
// Control group: Cities that did not implement the policy
const result = await diffInDiffTool.execute({
  treatmentBefore: [100, 105, 98, 102],   // Before policy
  treatmentAfter: [120, 125, 118, 122],    // After policy
  controlBefore: [95, 100, 92, 98],        // Before (no policy)
  controlAfter: [98, 103, 95, 101],        // After (no policy)
  confidenceLevel: 0.95,
});

console.log(result);
// {
//   effect: 17.5,              // Treatment caused 17.5 unit increase
//   standardError: 2.1,
//   tStatistic: 8.33,
//   pValue: 0.0001,
//   significant: true,
//   confidenceInterval: {
//     lower: 13.2,
//     upper: 21.8,
//     level: 0.95
//   },
//   interpretation: "The treatment effect is 17.5 (increased by 17.5 units)...",
//   groupMeans: {
//     treatmentBefore: 101.25,
//     treatmentAfter: 121.25,
//     controlBefore: 96.25,
//     controlAfter: 99.25
//   },
//   differences: {
//     treatmentDiff: 20.0,     // Treatment group change
//     controlDiff: 3.0         // Control group change
//   }
// }

API

Input

  • treatmentBefore (required): Treatment group values before intervention
  • treatmentAfter (required): Treatment group values after intervention
  • controlBefore (required): Control group values before intervention
  • controlAfter (required): Control group values after intervention
  • confidenceLevel (optional): Confidence level (default: 0.95)

Output

  • effect: Estimated causal treatment effect (DiD estimator)
  • standardError: Standard error of the estimate
  • tStatistic: Test statistic for significance testing
  • pValue: Two-tailed p-value
  • significant: Whether effect is statistically significant
  • confidenceInterval: Confidence interval for the effect
  • interpretation: Plain English interpretation
  • groupMeans: Mean values for all four groups
  • differences: Within-group changes over time

Algorithm

The DiD estimator removes time-invariant confounders by differencing:

Formula: DiD = (T_after - T_before) - (C_after - C_before)

Where:

  • T = Treatment group
  • C = Control group

This double-differencing removes:

  1. Time trends (via control group)
  2. Group differences (via before/after comparison)

Key Assumption: Parallel trends - Without treatment, both groups would have changed similarly.

Use Cases

  • Policy evaluation: Measure impact of new regulations
  • Marketing: Test effectiveness of campaigns
  • Medicine: Clinical trials with before/after measurements
  • Economics: Evaluate economic interventions
  • Education: Assess program effectiveness

Example: Minimum Wage Study

// States that raised minimum wage (treatment)
// vs states that didn't (control)
const result = await diffInDiffTool.execute({
  treatmentBefore: [5.2, 5.5, 5.1, 5.4], // Employment before
  treatmentAfter: [5.1, 5.3, 5.0, 5.2],  // Employment after
  controlBefore: [5.3, 5.4, 5.2, 5.5],
  controlAfter: [5.4, 5.5, 5.3, 5.6],
});

// Effect tells us the causal impact on employment

Interpretation

A positive effect means treatment increased the outcome. A negative effect means treatment decreased the outcome.

Statistical significance (p < 0.05) suggests the effect is real, not due to chance.

Limitations

  • Requires parallel trends assumption
  • Can't control for time-varying confounders
  • Sensitive to outliers
  • Needs sufficient sample size

License

MIT