docs: add comprehensive NPM package sync documentation and GitHub Actions workflow
**Documentation (CLAUDE.md):** - Document all three sync endpoints (changes feed, keyword search, metrics) - Explain Vercel Cron configuration and schedules - Detail quality score calculation algorithm - Add database schema documentation for sync tables - Provide manual sync trigger examples with curl - Include monitoring and debugging instructions - Document error handling patterns (partial/complete failures) - Add package discovery flow diagram - List future improvements and potential enhancements **GitHub Actions Workflow (.github/workflows/sync.yml):** - Add backup sync automation via GitHub Actions cron - Support manual trigger via workflow_dispatch with sync type selection - Run changes feed every 2 minutes - Run keyword search every 15 minutes - Run metrics sync every hour - Use concurrency control to prevent overlapping runs - Call production Vercel endpoints with CRON_SECRET auth **Key Features:** - Dual automation strategy: Vercel Cron (primary) + GitHub Actions (backup) - Idempotent endpoints allow both systems to run simultaneously - Manual trigger capability for debugging and testing - Comprehensive documentation for future maintenance 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
61ac2aa5dd
commit
0b4a314bd1
2 changed files with 491 additions and 1 deletions
71
.github/workflows/sync.yml
vendored
Normal file
71
.github/workflows/sync.yml
vendored
Normal file
|
|
@ -0,0 +1,71 @@
|
|||
name: NPM Package Sync
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# Changes feed: Every 2 minutes
|
||||
- cron: '*/2 * * * *'
|
||||
# Keyword search: Every 15 minutes
|
||||
- cron: '*/15 * * * *'
|
||||
# Metrics: Every hour
|
||||
- cron: '0 * * * *'
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
sync_type:
|
||||
description: 'Type of sync to run'
|
||||
required: true
|
||||
type: choice
|
||||
options:
|
||||
- changes
|
||||
- keyword
|
||||
- metrics
|
||||
- all
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.inputs.sync_type || 'scheduled' }}
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
sync-changes:
|
||||
runs-on: ubuntu-latest
|
||||
# Run on schedule every 2 minutes OR manual trigger with 'changes' or 'all'
|
||||
if: |
|
||||
github.event.schedule == '*/2 * * * *' ||
|
||||
github.event.inputs.sync_type == 'changes' ||
|
||||
github.event.inputs.sync_type == 'all'
|
||||
steps:
|
||||
- name: Trigger changes feed sync
|
||||
run: |
|
||||
curl -X POST "${{ secrets.VERCEL_PRODUCTION_URL }}/api/sync/changes" \
|
||||
-H "Authorization: Bearer ${{ secrets.CRON_SECRET }}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-f -s -S -w "\nHTTP Status: %{http_code}\n"
|
||||
|
||||
sync-keyword:
|
||||
runs-on: ubuntu-latest
|
||||
# Run on schedule every 15 minutes OR manual trigger with 'keyword' or 'all'
|
||||
if: |
|
||||
github.event.schedule == '*/15 * * * *' ||
|
||||
github.event.inputs.sync_type == 'keyword' ||
|
||||
github.event.inputs.sync_type == 'all'
|
||||
steps:
|
||||
- name: Trigger keyword search sync
|
||||
run: |
|
||||
curl -X POST "${{ secrets.VERCEL_PRODUCTION_URL }}/api/sync/keyword" \
|
||||
-H "Authorization: Bearer ${{ secrets.CRON_SECRET }}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-f -s -S -w "\nHTTP Status: %{http_code}\n"
|
||||
|
||||
sync-metrics:
|
||||
runs-on: ubuntu-latest
|
||||
# Run on schedule every hour OR manual trigger with 'metrics' or 'all'
|
||||
if: |
|
||||
github.event.schedule == '0 * * * *' ||
|
||||
github.event.inputs.sync_type == 'metrics' ||
|
||||
github.event.inputs.sync_type == 'all'
|
||||
steps:
|
||||
- name: Trigger metrics sync
|
||||
run: |
|
||||
curl -X POST "${{ secrets.VERCEL_PRODUCTION_URL }}/api/sync/metrics" \
|
||||
-H "Authorization: Bearer ${{ secrets.CRON_SECRET }}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-f -s -S -w "\nHTTP Status: %{http_code}\n"
|
||||
Loading…
Add table
Add a link
Reference in a new issue