Skip to content

Trustworthy Node timings: full-resolution timestamps, warm-up before large tests, V100 baseline - #80

Merged
jowens merged 3 commits into
mainfrom
timestamp-resolution
Oct 24, 2026
Merged

jowens merged 3 commits into
mainfrom
timestamp-resolution

Conversation

@jowens

@jowens jowens commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Two changes so the Node regression runner's timings can be trusted.

1. Full-resolution timestamps

Dawn rounds GPU timestamps down to multiples of 65.536 µs (2^16 ns) by default, on every GPU, to limit timing side channels. All 115 timings in a V100 baseline run were multiples of 0.065536 ms, and so is every time in the existing M4 baseline. Most single executes in the suite take less than that.

  • examples/regression_node.mjs: creates its GPU with disable-dawn-features=timestamp_quantization.
  • docs/timing-strategy.md: a new "Timestamp resolution" section covering:
    • Node: the create() flag;
    • Chromium browsers: --disable-dawn-features=timestamp_quantization;
    • otherwise: use trials.
  • docs/node_webgpu_regression_plan.md: it blamed the 65.536 µs grain on Apple Silicon. It's Dawn, on any GPU; corrected.

2. Warm-up before timing large tests

GPUs lower their clocks when they look idle, and single executes separated by CPU work look idle. On daisy's V100 (GPU 0, the one driving the display), nvidia-smi showed the SM clock falling from 1,297 to 180 MHz during a sort timing run. The only throttle reason was "GPU Idle" (0x1). Every kernel slowed by up to 10×, even the one-workgroup onesweep_scan.

That made sort look broken: 2^24 keys took 33.9 ms (0.5 G keys/s). At full clock, the same sort takes 2.9 ms (5.7 G keys/s).

  • Runner: before the timed trials of each test of 1M elements or more, it now runs the primitive back to back for 200 ms. It doubles trials until one execute keeps the GPU busy for 20 ms.
  • Docs: the regression-plan doc describes the warm-up and how to lock clocks (nvidia-smi --lock-gpu-clocks) for runs that must be comparable.

Checked:

  • Lavapipe: every large scan, HierarchicalScan, MultisplitPerm and histogram test stays correct with the warm-up. (Sort fails there at 8 lanes regardless.)
  • V100 (daisy): all 127 tests pass, and the GPU ran at its 1,530 MHz boost clock (median) while busy.

At 2^24 elements on the V100, before (idle clocks) and now:

primitive before now
sort 33.9 ms 2.50 ms (6.7 G keys/s)
DLDFScan 0.54 ms 0.227 ms
HierarchicalScan 0.96 ms 0.379 ms
MultisplitPerm 7.6 ms 1.92 ms
histogram 0.48 ms 0.111 ms

3. V100 baseline

perf_baseline.json gains the V100 run. One entry is left out: the 2M i32 key-value sort took 2.91 ms against 0.54–0.63 ms for its siblings, apparently during a clock dip, so it will be recorded fresh next time.

After this merges, the M4 baseline should be re-recorded (UPDATE_BASELINE=true). It was recorded with rounded timestamps and no warm-up.

https://claude.ai/code/session_015kfNC6id65zEHzuAAH4jtJ


Generated by Claude Code

…t Dawn's rounding

Dawn rounds GPU timestamps down to multiples of 65.536 us by default, on
every GPU (a privacy measure). Every time in the M4 baseline, and all 115 in
a new V100 run, is a multiple of 0.065536 ms. The regression runner now
creates its GPU with disable-dawn-features=timestamp_quantization.
timing-strategy.md gets a "Timestamp resolution" section (Node: the create()
flag; Chromium: --disable-dawn-features=timestamp_quantization; otherwise
use trials). The regression-plan doc blamed the 65.536 us grain on Apple
Silicon; corrected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015kfNC6id65zEHzuAAH4jtJ
GPUs lower their clocks when they look idle, and single executes separated
by CPU work look idle. On daisy's V100 the SM clock fell from 1,297 to
180 MHz ("GPU Idle" the only reason) during a sort timing run, and every
kernel slowed by up to 10x, the same factor for all of them. Before the timed
trials of each test of 1M elements or more, the runner now runs the
primitive back to back for 200 ms, doubling trials until one execute keeps
the GPU busy for 20 ms. Documented in node_webgpu_regression_plan.md, with
clock locking for runs that must be comparable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015kfNC6id65zEHzuAAH4jtJ
@jowens jowens changed the title Full-resolution GPU timestamps in the Node regression runner; document Dawn's rounding Trustworthy Node timings: full-resolution timestamps, and warm up before timing large tests Oct 24, 2026
…ion timestamps and warm-up

Recorded at 1,530 MHz while busy (nvidia-smi). The 2M i32 key-value sort
(2.91 ms against 0.54-0.63 ms for its siblings, during a clock dip) is left
out so that it is recorded fresh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015kfNC6id65zEHzuAAH4jtJ
@jowens jowens changed the title Trustworthy Node timings: full-resolution timestamps, and warm up before timing large tests Trustworthy Node timings: full-resolution timestamps, warm-up before large tests, V100 baseline Oct 24, 2026
@jowens
jowens merged commit 7277435 into main Oct 24, 2026
4 checks passed
@jowens
jowens deleted the timestamp-resolution branch October 24, 2026 05:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants