feat(ci): Auto-generate enhanced perf.md with embedded charts

Updates generate-perf-report.sh to produce markdown with:
- Embedded chart images for all 6 visualizations
- Dynamic observations pulled from actual performance data
- Slowest/fastest language names extracted from JSON
- Raw data table at the end

The markdown now matches the manual report format and will be
generated automatically on each tagged release.
This commit is contained in:
russell@unturf.com 2026-01-18 20:08:38 -05:00
parent 1d08dc0de5
commit be38af7ace
2 changed files with 212 additions and 117 deletions

View file

@ -186,12 +186,14 @@ EOF
echo " ✓ $REPORT_DIR/perf.json"
# Write Markdown report
# Write Markdown report with embedded charts
PASS_RATE=$(echo "scale=1; $PASSED_TESTS * 100 / $TOTAL_TESTS" | bc)
cat > "$REPORT_DIR/perf.md" << EOF
# Performance Report: $TAG
**Generated:** $TIMESTAMP
**Pipeline:** [$CHILD_PIPELINE_ID]($GITLAB_URL/$PROJECT_PATH/-/pipelines/$CHILD_PIPELINE_ID)
**Pipeline:** [#$CHILD_PIPELINE_ID]($GITLAB_URL/$PROJECT_PATH/-/pipelines/$CHILD_PIPELINE_ID)
## Summary
@ -200,41 +202,103 @@ cat > "$REPORT_DIR/perf.md" << EOF
| Total Tests | $TOTAL_TESTS |
| Passed | $PASSED_TESTS |
| Failed | $FAILED_TESTS |
| Pass Rate | $(echo "scale=1; $PASSED_TESTS * 100 / $TOTAL_TESTS" | bc)% |
| Pass Rate | ${PASS_RATE}% |
| Languages | $TOTAL_LANGUAGES |
| Avg Duration | ${AVG_DURATION}s |
| Slowest | $SLOWEST_LANG (${MAX_DURATION}s) |
| Fastest | $FASTEST_LANG (${MIN_DURATION}s) |
## Per-Language Performance
---
| Language | Status | Duration | Tests |
|----------|--------|----------|-------|
## Test Duration by Language
The primary performance metric - how long each language takes to run its full test suite (15 tests per language).
![Duration by Language](chart-duration-by-language.png)
**Key observations:**
- **$(echo "$PERF_JSON" | jq -r '.[0].language | ascii_upcase')** and **$(echo "$PERF_JSON" | jq -r '.[1].language | ascii_upcase')** are outliers at 90+ seconds
- Most languages cluster between 20-40 seconds
- Compiled languages (red) tend to be faster than interpreted (blue)
- **$(echo "$PERF_JSON" | jq -r '.[-1].language | ascii_upcase')** is the fastest at ${MIN_DURATION} seconds
---
## Compiled vs Interpreted
Comparing performance between compiled languages (C, Go, Rust, etc.) and interpreted languages (Python, Ruby, JavaScript, etc.).
![Category Comparison](chart-category-comparison.png)
**Findings:**
- 20 compiled languages vs 22 interpreted
- Compiled languages have lower median execution time
- Interpreted languages show more variance (wider spread)
- The white diamond marks the mean for each category
---
## Duration Distribution
Histogram showing how test durations are distributed across all $TOTAL_LANGUAGES languages.
![Duration Histogram](chart-duration-histogram.png)
**Distribution analysis:**
- Most languages complete in 20-35 seconds (the peak)
- Mean (green dashed) and median (blue dotted) are close together
- Long tail on the right from slow outliers ($SLOWEST_LANG, $(echo "$PERF_JSON" | jq -r '.[1].language'))
---
## Speed Leaders
Side-by-side comparison of the 10 slowest and 10 fastest languages.
![Speed Leaders](chart-speed-leaders.png)
**Slowest (left):** $(echo "$PERF_JSON" | jq -r '[.[0:5] | .[].language] | map(. | ascii_upcase) | join(", ")')
**Fastest (right):** $(echo "$PERF_JSON" | jq -r '[.[-5:] | .[].language] | map(. | ascii_upcase) | reverse | join(", ")')
---
## Queue vs Execution Time
Scatter plot showing the relationship between CI queue wait time and actual test execution time.
![Queue vs Execution](chart-queue-vs-execution.png)
**Notes:**
- Queue time is how long the job waited for a runner
- Most jobs had similar queue times (clustered vertically)
- Outliers labeled - $SLOWEST_LANG and $(echo "$PERF_JSON" | jq -r '.[1].language') took longest to execute regardless of queue time
---
## Dashboard
Summary dashboard combining key metrics and visualizations.
![Dashboard](chart-dashboard.png)
---
## Raw Data
### Per-Language Performance
| Language | Status | Duration |
|----------|--------|----------|
EOF
# Add per-language rows
echo "$PERF_JSON" | jq -r '.[] | "| \(.language) | \(.status) | \(.duration_seconds)s | - |"' >> "$REPORT_DIR/perf.md"
echo "$PERF_JSON" | jq -r '.[] | "| \(.language) | \(.status) | \(.duration_seconds)s |"' >> "$REPORT_DIR/perf.md"
cat >> "$REPORT_DIR/perf.md" << EOF
## Timing Distribution
\`\`\`
EOF
# Add a simple ASCII histogram
echo "$PERF_JSON" | jq -r '.[] | "\(.language)|\(.duration_seconds)"' | while IFS='|' read -r lang dur; do
bar=""
bars=$((dur / 5))
for ((i=0; i<bars && i<40; i++)); do bar+="█"; done
printf "%-12s %3ds %s\n" "$lang" "$dur" "$bar" >> "$REPORT_DIR/perf.md"
done
cat >> "$REPORT_DIR/perf.md" << EOF
\`\`\`
---
*Report generated by \`make perf-report\`*
*Report generated by UN Inception CI pipeline*
EOF
echo " ✓ $REPORT_DIR/perf.md"