408102602c
**Documentation (CLAUDE.md):** - Document all three sync endpoints (changes feed, keyword search, metrics) - Explain Vercel Cron configuration and schedules - Detail quality score calculation algorithm - Add database schema documentation for sync tables - Provide manual sync trigger examples with curl - Include monitoring and debugging instructions - Document error handling patterns (partial/complete failures) - Add package discovery flow diagram - List future improvements and potential enhancements **GitHub Actions Workflow (.github/workflows/sync.yml):** - Add backup sync automation via GitHub Actions cron - Support manual trigger via workflow_dispatch with sync type selection - Run changes feed every 2 minutes - Run keyword search every 15 minutes - Run metrics sync every hour - Use concurrency control to prevent overlapping runs - Call production Vercel endpoints with CRON_SECRET auth **Key Features:** - Dual automation strategy: Vercel Cron (primary) + GitHub Actions (backup) - Idempotent endpoints allow both systems to run simultaneously - Manual trigger capability for debugging and testing - Comprehensive documentation for future maintenance 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
71 lines
2.3 KiB
YAML
71 lines
2.3 KiB
YAML
name: NPM Package Sync
|
|
|
|
on:
|
|
schedule:
|
|
# Changes feed: Every 2 minutes
|
|
- cron: '*/2 * * * *'
|
|
# Keyword search: Every 15 minutes
|
|
- cron: '*/15 * * * *'
|
|
# Metrics: Every hour
|
|
- cron: '0 * * * *'
|
|
workflow_dispatch:
|
|
inputs:
|
|
sync_type:
|
|
description: 'Type of sync to run'
|
|
required: true
|
|
type: choice
|
|
options:
|
|
- changes
|
|
- keyword
|
|
- metrics
|
|
- all
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.event.inputs.sync_type || 'scheduled' }}
|
|
cancel-in-progress: false
|
|
|
|
jobs:
|
|
sync-changes:
|
|
runs-on: ubuntu-latest
|
|
# Run on schedule every 2 minutes OR manual trigger with 'changes' or 'all'
|
|
if: |
|
|
github.event.schedule == '*/2 * * * *' ||
|
|
github.event.inputs.sync_type == 'changes' ||
|
|
github.event.inputs.sync_type == 'all'
|
|
steps:
|
|
- name: Trigger changes feed sync
|
|
run: |
|
|
curl -X POST "${{ secrets.VERCEL_PRODUCTION_URL }}/api/sync/changes" \
|
|
-H "Authorization: Bearer ${{ secrets.CRON_SECRET }}" \
|
|
-H "Content-Type: application/json" \
|
|
-f -s -S -w "\nHTTP Status: %{http_code}\n"
|
|
|
|
sync-keyword:
|
|
runs-on: ubuntu-latest
|
|
# Run on schedule every 15 minutes OR manual trigger with 'keyword' or 'all'
|
|
if: |
|
|
github.event.schedule == '*/15 * * * *' ||
|
|
github.event.inputs.sync_type == 'keyword' ||
|
|
github.event.inputs.sync_type == 'all'
|
|
steps:
|
|
- name: Trigger keyword search sync
|
|
run: |
|
|
curl -X POST "${{ secrets.VERCEL_PRODUCTION_URL }}/api/sync/keyword" \
|
|
-H "Authorization: Bearer ${{ secrets.CRON_SECRET }}" \
|
|
-H "Content-Type: application/json" \
|
|
-f -s -S -w "\nHTTP Status: %{http_code}\n"
|
|
|
|
sync-metrics:
|
|
runs-on: ubuntu-latest
|
|
# Run on schedule every hour OR manual trigger with 'metrics' or 'all'
|
|
if: |
|
|
github.event.schedule == '0 * * * *' ||
|
|
github.event.inputs.sync_type == 'metrics' ||
|
|
github.event.inputs.sync_type == 'all'
|
|
steps:
|
|
- name: Trigger metrics sync
|
|
run: |
|
|
curl -X POST "${{ secrets.VERCEL_PRODUCTION_URL }}/api/sync/metrics" \
|
|
-H "Authorization: Bearer ${{ secrets.CRON_SECRET }}" \
|
|
-H "Content-Type: application/json" \
|
|
-f -s -S -w "\nHTTP Status: %{http_code}\n"
|