feat(ci): Auto-generate enhanced perf.md with embedded charts
Updates generate-perf-report.sh to produce markdown with: - Embedded chart images for all 6 visualizations - Dynamic observations pulled from actual performance data - Slowest/fastest language names extracted from JSON - Raw data table at the end The markdown now matches the manual report format and will be generated automatically on each tagged release.
This commit is contained in:
parent
1d08dc0de5
commit
be38af7ace
2 changed files with 212 additions and 117 deletions
|
|
@ -186,12 +186,14 @@ EOF
|
|||
|
||||
echo " ✓ $REPORT_DIR/perf.json"
|
||||
|
||||
# Write Markdown report
|
||||
# Write Markdown report with embedded charts
|
||||
PASS_RATE=$(echo "scale=1; $PASSED_TESTS * 100 / $TOTAL_TESTS" | bc)
|
||||
|
||||
cat > "$REPORT_DIR/perf.md" << EOF
|
||||
# Performance Report: $TAG
|
||||
|
||||
**Generated:** $TIMESTAMP
|
||||
**Pipeline:** [$CHILD_PIPELINE_ID]($GITLAB_URL/$PROJECT_PATH/-/pipelines/$CHILD_PIPELINE_ID)
|
||||
**Pipeline:** [#$CHILD_PIPELINE_ID]($GITLAB_URL/$PROJECT_PATH/-/pipelines/$CHILD_PIPELINE_ID)
|
||||
|
||||
## Summary
|
||||
|
||||
|
|
@ -200,41 +202,103 @@ cat > "$REPORT_DIR/perf.md" << EOF
|
|||
| Total Tests | $TOTAL_TESTS |
|
||||
| Passed | $PASSED_TESTS |
|
||||
| Failed | $FAILED_TESTS |
|
||||
| Pass Rate | $(echo "scale=1; $PASSED_TESTS * 100 / $TOTAL_TESTS" | bc)% |
|
||||
| Pass Rate | ${PASS_RATE}% |
|
||||
| Languages | $TOTAL_LANGUAGES |
|
||||
| Avg Duration | ${AVG_DURATION}s |
|
||||
| Slowest | $SLOWEST_LANG (${MAX_DURATION}s) |
|
||||
| Fastest | $FASTEST_LANG (${MIN_DURATION}s) |
|
||||
|
||||
## Per-Language Performance
|
||||
---
|
||||
|
||||
| Language | Status | Duration | Tests |
|
||||
|----------|--------|----------|-------|
|
||||
## Test Duration by Language
|
||||
|
||||
The primary performance metric - how long each language takes to run its full test suite (15 tests per language).
|
||||
|
||||

|
||||
|
||||
**Key observations:**
|
||||
- **$(echo "$PERF_JSON" | jq -r '.[0].language | ascii_upcase')** and **$(echo "$PERF_JSON" | jq -r '.[1].language | ascii_upcase')** are outliers at 90+ seconds
|
||||
- Most languages cluster between 20-40 seconds
|
||||
- Compiled languages (red) tend to be faster than interpreted (blue)
|
||||
- **$(echo "$PERF_JSON" | jq -r '.[-1].language | ascii_upcase')** is the fastest at ${MIN_DURATION} seconds
|
||||
|
||||
---
|
||||
|
||||
## Compiled vs Interpreted
|
||||
|
||||
Comparing performance between compiled languages (C, Go, Rust, etc.) and interpreted languages (Python, Ruby, JavaScript, etc.).
|
||||
|
||||

|
||||
|
||||
**Findings:**
|
||||
- 20 compiled languages vs 22 interpreted
|
||||
- Compiled languages have lower median execution time
|
||||
- Interpreted languages show more variance (wider spread)
|
||||
- The white diamond marks the mean for each category
|
||||
|
||||
---
|
||||
|
||||
## Duration Distribution
|
||||
|
||||
Histogram showing how test durations are distributed across all $TOTAL_LANGUAGES languages.
|
||||
|
||||

|
||||
|
||||
**Distribution analysis:**
|
||||
- Most languages complete in 20-35 seconds (the peak)
|
||||
- Mean (green dashed) and median (blue dotted) are close together
|
||||
- Long tail on the right from slow outliers ($SLOWEST_LANG, $(echo "$PERF_JSON" | jq -r '.[1].language'))
|
||||
|
||||
---
|
||||
|
||||
## Speed Leaders
|
||||
|
||||
Side-by-side comparison of the 10 slowest and 10 fastest languages.
|
||||
|
||||

|
||||
|
||||
**Slowest (left):** $(echo "$PERF_JSON" | jq -r '[.[0:5] | .[].language] | map(. | ascii_upcase) | join(", ")')
|
||||
**Fastest (right):** $(echo "$PERF_JSON" | jq -r '[.[-5:] | .[].language] | map(. | ascii_upcase) | reverse | join(", ")')
|
||||
|
||||
---
|
||||
|
||||
## Queue vs Execution Time
|
||||
|
||||
Scatter plot showing the relationship between CI queue wait time and actual test execution time.
|
||||
|
||||

|
||||
|
||||
**Notes:**
|
||||
- Queue time is how long the job waited for a runner
|
||||
- Most jobs had similar queue times (clustered vertically)
|
||||
- Outliers labeled - $SLOWEST_LANG and $(echo "$PERF_JSON" | jq -r '.[1].language') took longest to execute regardless of queue time
|
||||
|
||||
---
|
||||
|
||||
## Dashboard
|
||||
|
||||
Summary dashboard combining key metrics and visualizations.
|
||||
|
||||

|
||||
|
||||
---
|
||||
|
||||
## Raw Data
|
||||
|
||||
### Per-Language Performance
|
||||
|
||||
| Language | Status | Duration |
|
||||
|----------|--------|----------|
|
||||
EOF
|
||||
|
||||
# Add per-language rows
|
||||
echo "$PERF_JSON" | jq -r '.[] | "| \(.language) | \(.status) | \(.duration_seconds)s | - |"' >> "$REPORT_DIR/perf.md"
|
||||
echo "$PERF_JSON" | jq -r '.[] | "| \(.language) | \(.status) | \(.duration_seconds)s |"' >> "$REPORT_DIR/perf.md"
|
||||
|
||||
cat >> "$REPORT_DIR/perf.md" << EOF
|
||||
|
||||
## Timing Distribution
|
||||
|
||||
\`\`\`
|
||||
EOF
|
||||
|
||||
# Add a simple ASCII histogram
|
||||
echo "$PERF_JSON" | jq -r '.[] | "\(.language)|\(.duration_seconds)"' | while IFS='|' read -r lang dur; do
|
||||
bar=""
|
||||
bars=$((dur / 5))
|
||||
for ((i=0; i<bars && i<40; i++)); do bar+="█"; done
|
||||
printf "%-12s %3ds %s\n" "$lang" "$dur" "$bar" >> "$REPORT_DIR/perf.md"
|
||||
done
|
||||
|
||||
cat >> "$REPORT_DIR/perf.md" << EOF
|
||||
\`\`\`
|
||||
|
||||
---
|
||||
*Report generated by \`make perf-report\`*
|
||||
|
||||
*Report generated by UN Inception CI pipeline*
|
||||
EOF
|
||||
|
||||
echo " ✓ $REPORT_DIR/perf.md"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue