Perf Regressions

The CI perf regression advisory job protects the terminal rendering differentiators from silent regressions.

CI workflow: .github/workflows/ci.yml
Job name: perf regression advisory

PR Gate Scope

The gate runs a reduced, deterministic benchmark profile:

• suite: terminal
• framework: rezi-native
• io mode: stub (default for CI determinism)
• required scenarios:
  • terminal-rerender
  • terminal-frame-fill (dirtyLines=1 and dirtyLines=40)
  • terminal-full-ui
  • terminal-strict-ui
  • terminal-virtual-list
  • terminal-table

Profile source: packages/bench/profiles/ci.json

Coverage guardrail:

• scripts/run-bench-ci.mjs now validates terminal scenario manifest coverage.
• Every terminal-* scenario in the registry must be either:
  • listed in requiredScenarios, or
  • listed in explicitExclusions with a reason.

Local Commands

From a clean checkout, the root bench wrapper builds @rezi-ui/bench and its TypeScript references automatically:

npm ci
npm run bench:ci -- --output-dir .artifacts/bench/ci-local

This reduced CI profile uses io=stub, so it does not require npm run build:native.

If you bypass the npm wrapper and invoke node scripts/run-bench-ci.mjs directly, build the bench package first:

npm run bench:build
node scripts/run-bench-ci.mjs --output-dir .artifacts/bench/ci-local

Compare to baseline:

npm run bench:ci:compare -- \
  --baseline benchmarks/ci-baseline/results.json \
  --current .artifacts/bench/ci-local/results.json \
  --config benchmarks/ci-baseline/config.json \
  --report-json .artifacts/bench/ci-local/compare.json \
  --report-md .artifacts/bench/ci-local/compare.md

Without --output-dir, the profile writes its reports to out/bench-ci/.

Threshold Model

Threshold config: benchmarks/ci-baseline/config.json

Rules are max(relative_regression * baseline, absolute_regression) per metric:

• timing.mean is a hard gate (fails CI when exceeded).
• timing.p95 is advisory by default (recorded in report, non-fatal).

This dual threshold prevents false failures from tiny absolute timing shifts on very fast scenarios, while still failing meaningful slowdowns.

Artifacts

The CI job uploads artifacts on both success and failure:

• .artifacts/bench/ci/results.json
• .artifacts/bench/ci/results.md
• .artifacts/bench/ci/compare.json
• .artifacts/bench/ci/compare.md
• .artifacts/bench/ci/manifest.json
• .artifacts/bench/ci/profile.json

compare.json and compare.md include actionable rows with scenario, metric, baseline, current, delta, and threshold.

Intentional Baseline Updates

Baseline path:

• benchmarks/ci-baseline/results.json
• benchmarks/ci-baseline/results.md

Update process:

1. Prefer CI-host refresh: download perf-regression-artifacts from the PR/job run and copy results.json/results.md into benchmarks/ci-baseline/.
2. Re-run compare against the updated baseline to verify zero hard regressions.
3. Keep threshold changes (benchmarks/ci-baseline/config.json) explicit in the same PR only when justified.
4. In PR description, include why baseline movement is intentional (feature/perf tradeoff, infra change, or expected engine shift).