perf(tooling): add an animation cost profiler and document the harness

Adds `bun run profile:animation`: it serves an isolated fixture and measures each animation variant directly, so comparing techniques takes seconds instead of an application rebuild plus a streamed response. The result is unambiguous and does not vary with element count, measured from 1 to 32: transform, opacity and filter cost zero extra style recalculations, while the individual rotate property, background-position, border-color and box-shadow each recalculate style 60 times a second, and geometry properties add layout on top. Notably `rotate: 360deg` is not a cheap synonym for `transform: rotate(360deg)`, and will-change, wrapper elements, containment and stepped timing do not make a non-composited property cheap. `scripts/perf/DOCUMENTATION.md` documents all four capture commands, how to stand up a production build to measure against, how to read the artifacts, the validity guarantees the scripts enforce, and the methodology rules, so this can be handed to an agent as the entry point for measuring performance. It is linked from the root guide's documentation anchors. The theme skill gains an animation contract carrying the measured table, and the performance skill points at the tooling documentation.
2026-08-03 18:38:58 +03:00
parent 632fc09e18
commit ca31157584
7 changed files with 478 additions and 2 deletions
@@ -197,13 +197,19 @@ A cache inside an `O(consumers × entities × candidates)` loop is a mitigation,
 ## Repository Tooling
-Three unattended capture commands exist; prefer them over ad-hoc timing code,
+`scripts/perf/DOCUMENTATION.md` is the entry point: it covers every capture
 command, how to stand up a production build to measure against, how to read the
 artifacts, and the validity guarantees these scripts enforce. Read it before
 measuring.
 Four unattended capture commands exist; prefer them over ad-hoc timing code,
 and extend them when a scenario is missing rather than measuring by hand.
 | Command | Answers |
 |---|---|
 | `bun run profile:idle` | What the app does while nobody interacts with it. Supports `--session`, `--tab`, `--then-tab`, `--panel`, `--expand-projects` to reach a specific mounted state, plus `--baseline` and `--budget-*` for regression gating. |
 | `bun run profile:session` | What a streaming assistant response costs. Creates a session, dispatches a prompt through the `openchamber session` CLI, and records until the session reports idle. Reports the long-task distribution, a timeline-trace breakdown, running animations, and output-normalised metrics. |
 | `bun run profile:animation` | What a CSS animation costs, isolated from the app. Animate only `transform` and `opacity`; everything else recalculates style every frame. |
 | `bun run profile:browser` | A manually driven capture when the interaction cannot be scripted. |
 Both automated commands fail loudly rather than reporting a clean result when
@@ -71,8 +71,34 @@ import { Icon } from '@/components/icon/Icon';
 Use `IconName` for icon values stored in arrays, objects, state, or config. `Icon` has no `size` prop. Run `bun run icons:generate` when introducing a sprite name, and never edit `sprite.ts` manually. Load `references/icons.md` for the complete workflow.
 ## Animation Contract
 Animate only `transform` and `opacity`. The compositor drives those; every other
 property recalculates style on each frame for as long as the animation runs, and
 geometry properties add layout on top. Measured on this repository's fixture,
 identical at any element count from 1 to 32:
 | Animated property | Style recalculations/sec | Layouts/sec |
 |---|---|---|
 | `transform`, `opacity`, `filter` | 0 | 0 |
 | `rotate` (the individual property) | 60 | 0 |
 | `background-position`, `border-color`, `box-shadow` | 60 | 0 |
 | `width` and other geometry | 60 | 60 |
 - `rotate: 360deg` is not a cheap synonym for `transform: rotate(360deg)`.
  Prefer the `transform` form.
 - Cost applies for the entire time an animation runs, so an indicator tied to a
  long-running operation pays it continuously. An indicator that is not
  conveying anything should not be animating.
 - `will-change`, wrapper elements, `contain`, and `steps()` timing do not make a
  non-composited property cheap. Only changing the property does.
 - Verify with `bun run profile:animation` rather than reasoning about it; add a
  variant to `scripts/perf/animation-fixture.html` for a technique not covered.
  See `scripts/perf/DOCUMENTATION.md`.
 ## Verification
 - Animations are limited to `transform` and `opacity`, or their cost was measured and accepted.
 - No hardcoded/palette colors were introduced.
 - Buttons use shared variants and sizes.
 - Icons use `Icon`/`IconName`, and generated sprite changes are intentional.
@@ -65,6 +65,7 @@ High-value anchors:
 - Sync: `packages/ui/src/sync/DOCUMENTATION.md`
 - Stores: `packages/ui/src/stores/DOCUMENTATION.md`
 - CLI: `packages/web/bin/lib/DOCUMENTATION.md`
 - Performance measurement tooling: `scripts/perf/DOCUMENTATION.md`
 - VS Code runtime: `packages/vscode/src/DOCUMENTATION.md`
 - Electron: `packages/electron/README.md`
 - Mobile: `packages/mobile/README.md`
@@ -83,7 +83,8 @@
    "release:test:intel": "./scripts/test-release-build.sh x86_64",
    "release:test:arm": "./scripts/test-release-build.sh aarch64",
    "profile:idle": "node scripts/profile-idle.mjs",
-    "profile:session": "node scripts/profile-session.mjs"
+    "profile:session": "node scripts/profile-session.mjs",
    "profile:animation": "node scripts/profile-animation.mjs"
  },
  "dependencies": {
    "@base-ui/react": "^1.4.0",
@@ -0,0 +1,170 @@
 # Performance Measurement Tooling
 Owns the unattended performance capture commands and their shared Chrome
 DevTools Protocol plumbing. Read this before measuring OpenChamber performance
 or extending these scripts. The methodology rules they enforce come from
 `.agents/skills/performance-engineering/SKILL.md`.
 ## Commands
 | Command | Answers |
 |---|---|
 | `bun run profile:idle` | What the app does while nobody interacts with it. |
 | `bun run profile:session` | What receiving and rendering a live assistant response costs. |
 | `bun run profile:animation` | What a CSS animation costs, isolated from the app. |
 | `bun run profile:browser` | A manually driven capture, for interactions that cannot be scripted. |
 All of them measure a real browser over CDP. Pass `--help` to any of them for
 the full option list.
 ## Before Measuring Anything
 **Measure a production build.** A development build's render and bundle
 behaviour does not represent what users run.
 ```bash
 bun run build:ui && bun run build:web
 cd <a project directory> && node <repo>/packages/web/bin/cli.js serve --port 4599 --foreground
 ```
 `profile:idle` and `profile:session` need a running server; `profile:animation`
 serves its own fixture and needs nothing.
 ## profile:idle
 Loads the app, lets it settle, then records a window during which no input is
 delivered. Everything it reports is therefore work the app performs while the
 user is doing nothing — the class of regression users notice as fan noise,
 battery drain, and a permanently busy tab.
 Reports per second of idle time: main-thread busy time, script, style
 recalculation and layout time and counts, DOM node / document / frame /
 listener growth, heap trajectory including a least-squares growth rate, a CPU
 sampling profile with self time per function, and attribution of timer,
 animation-frame and observer work to the call site that scheduled it.
 ```bash
 # Baseline, then compare a change against it and fail on a budget.
 bun run profile:idle -- --url http://127.0.0.1:4599 --output artifacts/before
 bun run profile:idle -- --url http://127.0.0.1:4599 --baseline artifacts/before --budget-cpu 5
 ```
 Scenario options reach a specific mounted state, because idle cost depends on
 what is mounted: `--session`, `--tab`, `--panel <mode>`, `--expand-projects`,
 `--expand-sessions`, and `--then-tab` (navigate away after settling, to measure
 what a surface keeps doing once the user has left it).
 ## profile:session
 Creates a session, opens it in a browser, dispatches a prompt through the
 supported `openchamber session` CLI, and records until the session reports
 itself idle. No input is synthesised; the prompt is the only stimulus.
 Streaming is judged by responsiveness, not totals, so the report leads with the
 long-task distribution, a timeline-trace breakdown naming where time went,
 running animations, the application's own stream counters, and output-normalised
 metrics.
 ```bash
 bun run profile:session -- --url http://127.0.0.1:4599 --dir <project directory>
 # What an idle session costs while a different session is active elsewhere:
 bun run profile:session -- --view-session <idle session id> --expand-projects --expand-sessions
 ```
 This command calls a real model. Use a cheap one; `--model` overrides the
 configured selection.
 ## profile:animation
 Serves an isolated fixture and measures each animation variant directly, so a
 comparison takes seconds instead of an application rebuild plus a streamed
 response.
 ```bash
 bun run profile:animation
 bun run profile:animation -- --variant border-color --count 8
 ```
 Measured on this repository's fixture, at any element count from 1 to 32:
 | Animated property | Style recalculations/sec | Layouts/sec |
 |---|---|---|
 | none | 0 | 0 |
 | `transform` (rotate, translate, scale) | 0 | 0 |
 | `opacity`, `filter` | 0 | 0 |
 | `rotate` (the individual property) | 60 | 0 |
 | `background-position` | 60 | 0 |
 | `border-color` | 60 | 0 |
 | `box-shadow` | 60 | 0 |
 | `width` | 60 | 60 |
 Animate `transform` and `opacity`. Anything else recalculates style on every
 frame for as long as the animation runs, and geometry properties add layout on
 top. Note that `rotate: 360deg` is *not* equivalent to
 `transform: rotate(360deg)` in cost.
 Add a variant to `animation-fixture.html` to measure a property or technique
 that is not listed.
 ## Reading The Results
 Every run writes a JSON summary next to any raw capture, so results can be
 compared later without re-running:
 - `profile:idle` → `idle-summary.json`, `cpu-profile.cpuprofile`
 - `profile:session` → `session-summary.json`, `cpu-profile.cpuprofile`
 `--baseline <directory>` prints a per-metric delta table against a previous run
 of the same command. `--budget-*` options make the command exit non-zero, so the
 same invocation works as an investigation tool and as a regression gate.
 Artifacts can reveal project paths and endpoint names. They are gitignored; do
 not publish them without review.
 ## Validity Guarantees
 These commands fail loudly rather than reporting a clean result, because each
 of these failure modes once produced a confident, wrong "everything is fast":
 - **Throttled renderer.** Chrome stops producing frames and throttles timers for
  windows it considers backgrounded or occluded. Launch flags disable that, and
  every run measures frame liveness and warns when the renderer was not
  producing frames.
 - **Missing trace data.** `RunTask` is only emitted under the
  disabled-by-default timeline category. A capture without it would report zero
  long tasks; the missing-task case is reported instead.
 - **A scenario that never ran.** A session belonging to a directory the browser
  is not viewing renders nothing and produces a perfectly quiet profile.
  `profile:session` verifies both new message elements in the DOM and
  message-list render counters before believing a quiet result.
 Preserve this property when extending these scripts. A metric reading zero must
 be a measurement, never a disabled instrument.
 ## Methodology Rules
 - **Never report an "after" without a "before" on the identical scenario and
  build.** Rebuild the unchanged version and re-run it, however inconvenient.
  Expect plausible fixes to change nothing.
 - **A sampling profiler cannot explain native work.** Self time in `(program)`
  only means the time was not in interpreted JavaScript. Use the trace
  breakdown, which names parsing, style, layout, layerization, paint and raster.
 - **Normalise when the workload varies.** Assistant responses differ in length
  between runs, so per-second totals are not comparable; `profile:session`
  reports output-normalised metrics for this reason.
 - **Revert what you cannot measure.** A change that does not move its target
  metric is unvalidated complexity, not a small win.
 - **Reproduction may need production scale you do not have.** A threshold effect
  is invisible below its threshold. Compare the reporter's scale against yours
  on the dimension the code keys on before concluding a bug is absent.
 ## Module Layout
 | File | Responsibility |
 |---|---|
 | `cdp.mjs` | Chrome launch, target discovery, minimal CDP client. Owns the anti-throttling launch flags. |
 | `metrics.mjs` | Metric derivations shared by the profilers: growth rates, percentiles, long-task and trace-event summaries. |
 | `cpu-profile.mjs` | Aggregates `Profiler.stop()` output into self time per function. |
 | `idle-probe.mjs` | Page-side instrumentation installed before application code runs; attributes scheduled work to the call site that scheduled it. Must never change observable behaviour. |
 | `scenario.mjs` | Shared scenario setup, currently sidebar expansion. Setup always runs before the measured window. |
 | `animation-fixture.html` | Isolated animation variants for `profile:animation`. |
@@ -0,0 +1,65 @@
 <!doctype html>
 <html>
 <head>
 <meta charset="utf-8">
 <title>OpenChamber animation cost fixture</title>
 <style>
  body { margin: 0; background: #111; color: #eee; font: 12px ui-monospace, monospace; }
  .grid { display: flex; flex-wrap: wrap; gap: 8px; padding: 8px; }
  .cell { width: 24px; height: 24px; }
  svg { width: 24px; height: 24px; display: block; }
  .wrap { display: block; width: 24px; height: 24px; }
  /*
    One variant per animated property, so a run answers "what does animating
    this cost" rather than "is this particular component slow". Compositor-
    driven properties should cost a small constant; anything the compositor
    cannot handle recalculates style every frame.
  */
  @keyframes oc-transform-rotate { to { transform: rotate(360deg); } }
  @keyframes oc-rotate-property { to { rotate: 360deg; } }
  @keyframes oc-opacity { 0%, 100% { opacity: 0.2; } 50% { opacity: 1; } }
  @keyframes oc-translate { 0%, 100% { transform: translateY(0); } 50% { transform: translateY(6px); } }
  @keyframes oc-scale { 0%, 100% { transform: scale(0.7); } 50% { transform: scale(1); } }
  @keyframes oc-background-position { to { background-position: 200% 0; } }
  @keyframes oc-border-color { 0%, 100% { border-color: #345; } 50% { border-color: #8ab; } }
  @keyframes oc-box-shadow { 0%, 100% { box-shadow: 0 0 0 0 #8ab; } 50% { box-shadow: 0 0 8px 2px #8ab; } }
  @keyframes oc-filter { 0%, 100% { filter: brightness(0.6); } 50% { filter: brightness(1.4); } }
  @keyframes oc-width { 0%, 100% { width: 12px; } 50% { width: 24px; } }
  .v-none svg { animation: none; }
  .v-transform-rotate svg { animation: oc-transform-rotate 1s linear infinite; }
  .v-transform-rotate-willchange svg { animation: oc-transform-rotate 1s linear infinite; will-change: transform; }
  .v-transform-rotate-steps svg { animation: oc-transform-rotate 1s steps(12) infinite; }
  .v-transform-rotate-wrapper .wrap { animation: oc-transform-rotate 1s linear infinite; }
  .v-rotate-property svg { animation: oc-rotate-property 1s linear infinite; }
  .v-opacity svg { animation: oc-opacity 1.2s ease-in-out infinite; }
  .v-translate svg { animation: oc-translate 1.2s ease-in-out infinite; }
  .v-scale svg { animation: oc-scale 1.2s ease-in-out infinite; }
  .v-background-position .cell { background: linear-gradient(90deg, #223, #8ab, #223); background-size: 200% 100%; animation: oc-background-position 1.5s linear infinite; }
  .v-border-color .cell { border: 2px solid #345; animation: oc-border-color 1.5s linear infinite; }
  .v-box-shadow .cell { animation: oc-box-shadow 1.5s linear infinite; }
  .v-filter svg { animation: oc-filter 1.5s linear infinite; }
  .v-width svg { animation: oc-width 1.5s linear infinite; }
 </style>
 </head>
 <body>
 <div class="grid" id="grid"></div>
 <script>
  const SPINNER = '<svg viewBox="0 0 24 24" fill="none" stroke="#8ab" stroke-width="2">'
    + '<circle cx="12" cy="12" r="9" stroke-opacity=".25"/><path d="M21 12a9 9 0 0 0-9-9"/></svg>';
  const params = new URLSearchParams(location.search);
  const variant = params.get('variant') || 'none';
  const count = Math.max(1, Number(params.get('count') || 2));
  const grid = document.getElementById('grid');
  grid.className = 'grid v-' + variant;
  const usesWrapper = variant.endsWith('-wrapper');
  for (let index = 0; index < count; index += 1) {
    const cell = document.createElement('div');
    cell.className = 'cell';
    cell.innerHTML = usesWrapper ? '<span class="wrap">' + SPINNER + '</span>' : SPINNER;
    grid.appendChild(cell);
  }
 </script>
 </body>
 </html>
@@ -0,0 +1,207 @@
 #!/usr/bin/env node
 /**
 * Measures what a CSS animation costs, in isolation.
 *
 * A continuously running animation is one of the few things an idle interface
 * keeps paying for, and the price depends entirely on whether the compositor
 * can drive the animated property. Compositor-driven properties cost a small
 * constant; everything else recalculates style on every frame, roughly an order
 * of magnitude more.
 *
 * Rather than rebuilding the application and streaming a response to answer
 * that question, this command serves a fixture page and measures each variant
 * directly, so a whole comparison takes seconds. Add a variant to
 * `perf/animation-fixture.html` to measure a property or technique that is not
 * listed yet.
 *
 * The output answers two questions the fixture is designed for:
 * - which properties are cheap to animate;
 * - whether cost scales with the number of animated elements (`--count`).
 */
 import { createReadStream } from "node:fs"
 import { createServer } from "node:http"
 import { homedir } from "node:os"
 import { dirname, join, resolve } from "node:path"
 import { fileURLToPath } from "node:url"
 import process from "node:process"
 import { CdpClient, createPageTarget, launchChrome, reservePort, resolveChrome, wait } from "./perf/cdp.mjs"
 import { metricMap, round } from "./perf/metrics.mjs"
 const scriptDirectory = dirname(fileURLToPath(import.meta.url))
 const fixturePath = join(scriptDirectory, "perf", "animation-fixture.html")
 const DEFAULT_VARIANTS = [
  "none",
  "transform-rotate",
  "transform-rotate-willchange",
  "transform-rotate-steps",
  "transform-rotate-wrapper",
  "rotate-property",
  "opacity",
  "translate",
  "scale",
  "background-position",
  "border-color",
  "box-shadow",
  "filter",
  "width",
 ]
 const HELP = `Usage: bun run profile:animation -- [options]
 Measures the idle cost of CSS animations using an isolated fixture page.
 Options:
  --variant <name>     Measure only this variant (repeatable)
  --count <n>          Animated elements per variant (default: 2)
  --duration <seconds> Measurement window per variant (default: 10)
  --settle <seconds>   Wait before measuring each variant (default: 3)
  --chrome <path>      Chrome/Chromium executable
  --profile-dir <path> Reusable isolated Chrome profile
  --headed             Show the browser (default: headless)
  --json               Print results as JSON
  --help               Show this help
 Variants live in scripts/perf/animation-fixture.html. Add one there to measure
 a property or technique that is not covered.`
 const parseArgs = (argv) => {
  const options = {
    variants: [],
    count: 2,
    duration: 10,
    settle: 3,
    chrome: null,
    profileDir: join(homedir(), ".openchamber", "browser-profile-google-chrome"),
    headless: true,
    json: false,
  }
  for (let index = 0; index < argv.length; index += 1) {
    const value = argv[index]
    if (value === "--help") return { ...options, help: true }
    else if (value === "--headed") options.headless = false
    else if (value === "--json") options.json = true
    else if (value === "--variant") options.variants.push(argv[++index])
    else if (value === "--count") options.count = Number(argv[++index])
    else if (value === "--duration") options.duration = Number(argv[++index])
    else if (value === "--settle") options.settle = Number(argv[++index])
    else if (value === "--chrome") options.chrome = argv[++index]
    else if (value === "--profile-dir") options.profileDir = argv[++index]
    else throw new Error(`Unknown option: ${value}`)
  }
  if (!Number.isFinite(options.duration) || options.duration <= 0) throw new Error("--duration must be a positive number")
  if (!Number.isFinite(options.count) || options.count < 1) throw new Error("--count must be at least 1")
  if (options.variants.length === 0) options.variants = DEFAULT_VARIANTS
  return options
 }
 /** Serves only the fixture, on an ephemeral loopback port. */
 const startFixtureServer = () => new Promise((resolvePromise, reject) => {
  const server = createServer((_request, response) => {
    response.writeHead(200, { "content-type": "text/html; charset=utf-8" })
    createReadStream(fixturePath).pipe(response)
  })
  server.on("error", reject)
  server.listen(0, "127.0.0.1", () => {
    const address = server.address()
    if (!address || typeof address === "string") {
      server.close()
      reject(new Error("Could not bind the fixture server"))
      return
    }
    resolvePromise({ server, port: address.port })
  })
 })
 const measureVariant = async (client, url, options) => {
  const loaded = client.once("Page.loadEventFired", 30_000)
  await client.send("Page.navigate", { url })
  await loaded
  await wait(options.settle * 1000)
  const before = metricMap((await client.send("Performance.getMetrics")).metrics)
  const startedAt = Date.now()
  await wait(options.duration * 1000)
  const elapsedSeconds = (Date.now() - startedAt) / 1000
  const after = metricMap((await client.send("Performance.getMetrics")).metrics)
  const delta = (name) => Number(after[name] ?? 0) - Number(before[name] ?? 0)
  return {
    recalcStylePerSecond: round(delta("RecalcStyleCount") / elapsedSeconds),
    layoutsPerSecond: round(delta("LayoutCount") / elapsedSeconds),
    mainThreadBusyPercent: round((delta("TaskDuration") / elapsedSeconds) * 100),
    recalcStyleMsPerSecond: round((delta("RecalcStyleDuration") / elapsedSeconds) * 1000),
  }
 }
 const main = async () => {
  const options = parseArgs(process.argv.slice(2))
  if (options.help) {
    console.log(HELP)
    return
  }
  const chrome = resolveChrome(options.chrome)
  const { server, port: fixturePort } = await startFixtureServer()
  const debuggingPort = await reservePort()
  const chromeProcess = launchChrome({
    chrome,
    profileDir: resolve(options.profileDir),
    port: debuggingPort,
    headless: options.headless,
  })
  let client
  const results = []
  try {
    const target = await createPageTarget(debuggingPort)
    client = new CdpClient(target.webSocketDebuggerUrl)
    await client.connect()
    await Promise.all([
      client.send("Page.enable"),
      client.send("Runtime.enable"),
      client.send("Performance.enable"),
    ])
    await client.send("Emulation.setDeviceMetricsOverride", {
      width: 1200, height: 800, deviceScaleFactor: 1, mobile: false,
    })
    console.log(`Measuring ${options.variants.length} variants, ${options.count} element(s) each, ${options.duration}s per variant.\n`)
    for (const variant of options.variants) {
      const url = `http://127.0.0.1:${fixturePort}/?variant=${encodeURIComponent(variant)}&count=${options.count}`
      const measured = await measureVariant(client, url, options)
      results.push({ variant, ...measured })
      if (!options.json) {
        console.log(
          `${variant.padEnd(30)} recalc/s ${String(measured.recalcStylePerSecond).padStart(8)}`
          + `   layout/s ${String(measured.layoutsPerSecond).padStart(6)}`
          + `   busy% ${String(measured.mainThreadBusyPercent).padStart(6)}`,
        )
      }
    }
    if (options.json) {
      console.log(JSON.stringify({ count: options.count, durationSeconds: options.duration, results }, null, 2))
      return
    }
    const baseline = results.find((entry) => entry.variant === "none")
    if (baseline) {
      console.log(
        `\nA still page costs ${baseline.recalcStylePerSecond} style recalculations per second.`
        + " Compositor-driven properties add a small constant; anything far above that recalculates every frame.",
      )
    }
  } finally {
    client?.close()
    if (!chromeProcess.killed) chromeProcess.kill("SIGTERM")
    server.close()
  }
 }
 main().catch((error) => {
  console.error(`Animation profiling failed: ${error.message}`)
  process.exitCode = 1
 })