perf: large threads open from a viewport snapshot, and typing and streaming render less by maria-rcks · Pull Request #18220 · pingdotgg/t3code

maria-rcks · 2026-10-11T13:08:56Z

stacked on #18219, which is stacked on #17609. this diff shows only this layer.

after the first two layers, the largest thread (4,495 items) still loaded a 4.6 MB snapshot and decoded it in one long task. every streamed batch re-rendered the whole chat view. every keystroke re-rendered the composer frame. the desktop composer waited for a boot transition that other startup work kept restarting. this layer removes that work:

threads:
- a thread opens from a 75-row viewport snapshot and pages older turns in when you expand a fold, scroll up or recall prompts with ArrowUp.
- completed markdown reuses its parsed tree.
- rows measure with ResizeObserver.
- a streamed reply repaints only its own row, once per frame.
client state:
- shell, vcs and pull request updates keep unchanged objects, so unrelated chat and sidebar rows skip rendering.
- the shell cache writes only changed rows.
- thread caches drop duplicated items.
composer:
- typing updates the editor locally instead of re-rendering the frame.
- the draft saves after a short idle and flushes on unload.
boot:
- the cached chat view renders in one task on desktop, and on web when the session answers late.
- an over-budget boot snapshot stops encoding at the budget.
sidebar, palette, settings:
- only visible sidebar rows render.
- palette search reuses normalized text.
- long settings lists mount in batches without long frames.
server:
- activity and visit events patch single fields in sqlite instead of re-encoding the whole thread.
- committed events are encoded once for every subscriber.
- the vcs status cache is bounded. it keeps subscribed workspaces, and its retained heap is cut in half.
- turn checkpoints reuse a git index cache.
- backend and spawn-host processes close inherited file descriptors.
desktop:
- the main process bundle is split.
- main-process requests use node's http agents without env proxies.

metric, median of 5 interleaved runs [min-max]	#18219 `0db57d8`	this PR `1634ef4`	change
largest thread cold open, paint	927 [870-1,088] ms	371 [343-426] ms	−60%
largest thread cold open, settled	1,246 [1,186-1,367] ms	394 [364-446] ms	−68%
largest thread cold open, long-task total	783 [774-801] ms	393 [315-402] ms	−50%
largest thread snapshot	4,595 KB	280 KB	−94%
scroll dropped frames / worst frame	9.4% / 83 ms	6.8% / 50 ms	−27% / −40%
streamed reply, main-thread ms per batch	20.7 [20.2-21.9]	6.08 [5.53-6.14]	−71%
typing, empty composer, script ms/key	6.28 [5.97-6.47]	3.11 [3.03-3.23]	−50%
typing, 50 KB draft, script ms/key	10.7 [10.1-10.7]	3.91 [3.69-4.10]	−63%
palette Enter → largest thread	738 [420-744] ms	540 [515-566] ms	−27%
diff panel first use	838 [810-891] ms	686 [646-749] ms	−18%
desktop composer editable, default shell	2,376 [2,306-2,670] ms	1,511 [1,448-1,924] ms	−36%
desktop composer editable, 0.8 s login shell	2,171 [2,153-2,504] ms	1,414 [1,254-1,804] ms	−35%
server streaming cpu per event, 4 subscribers	4.77 [4.67-5.00] ms	3.29 [3.23-3.42] ms	−31%

the whole stack against main 18656a3:

metric	main	this PR	change
cold web load → editable composer	1,422 ms	1,293 ms	−9%
warm web reload → editable composer	795 ms	592 ms	−25%
largest thread cold open, settled	1,404 ms	394 ms	−72%
typing, empty composer, script ms/key	6.34	3.11	−51%
desktop composer editable, default shell	5,893 ms	1,511 ms	−74%
server cold boot → first rpc, 1,350 threads	9,700 ms	3,458 ms	−64%
server rss at ready, 1,350 threads	740 MiB	282 MiB	−62%

measurement: built artifacts on one 16-core linux host. web runs in chromium 1223 and chrome-headless-shell 154. desktop is electron 44.4.5 under xvfb (software rendering). the server uses the 75-thread seed and a 1,350-thread copy. all four sides ran interleaved in the same batches, at loadavg 0.9-6.5. the numbers come from 1634ef4. the final-review fixes after it address correctness. on measured paths they grow the viewport snapshot 0.5% (it now covers the last 10 user turns) and drop a boot read in new tabs that saved 6 ms.

what got slower:

this layer: nothing separates outside noise. desktop default-shell first paint went 819 → 920 ms and window 692 → 737 ms, with overlapping ranges. with a 0.8 s login shell both got faster.
tradeoff, palette Enter: the first frame after Enter comes ~40 ms later. content paint still moved 211 ms sooner.
tradeoff, keybindings: the keybindings list finishes mounting 13-59 ms later in exchange for a smaller largest frame.
tradeoff, vcs cache: forced gc in a vcs stress test rose 266 → 471 ms, while retained heap fell 425 → 211 MiB and populate time 1,027 → 521 ms.
whole stack, first use of lazy surfaces: these got slower with perf: trim application startup and deferred runtime work #17609's lazy loading, and perf: trim application startup and deferred runtime work #17609 shows the same numbers. palette open went 192 → 393 ms, settings 255 → 357 ms and the terminal's first prompt 212 → 489 ms.

compatibility:

older and newer clients and servers work together in both directions, checked at runtime. the new checkpoint field deferredFileCount is optional.
thread cache entries are written in an envelope that older builds skip, so they fetch live data. plain entries written earlier still load.
cache manifests written by newer builds are kept.
no new migrations.

verification:

published head 0f89a24 includes main 5fe7568 through the two lower layers. two independent gpt-6.1-sol reviewers cleared all three exact heads for source correctness. latest-head code checks pass; the cancelled native fingerprint job is being rerun.
blacksmith at 0f89a24: seven package typechecks, 374 focused tests, 290 nearby tests, 15 terminal tests, scoped lint (15 warnings, zero errors), and web/server/desktop production builds pass. the terminal test uses the web package wasm configuration.
an existing accepted message.dispatch replay test caught different defaults in the first response and durable retry. 625f214 normalizes encoded events with the replay decoder before projections and publication. the original failing test and nearby event-store/stream/runtime tests pass; no tests were weakened. a scoped final-head repeat of the same server benchmark delivered all 4,000 events per run and kept pr/shell invariants: cpu/event median 3.366910 ms [3.351548–3.478533] at historical 1634ef4 versus 3.446426 ms [3.396738–3.571974] at 0f89a24 (+2.36%, overlapping ranges), event-loop p99 5.132287 versus 5.283839 ms, and 11 sql calls/event on both. this is five interleaved pairs after a discarded pair, 1,000 assistant updates at 20 ms to four subscribers, fresh copies of the same scaled database, linux/node 24.21.0/16 cpus, load 0.65–1.34. later main merges confound attribution to normalization. the startup tables above remain evidence for their historical commits.
real codex gpt-6.1-sol through the web client at 3ea8e81 read an uploaded file and returned its exact unique marker. the screenshot below shows the completed reply. this proves that earlier runtime head. the final merged client pass stopped when the preview host disconnected, leaving input-error persistence, approval draft preservation, and merged timeline/settings/panel interactions unverified. native usable electron composition on the merged head is also unverified.
a reviewer filed 21 findings, two of them p1. all are fixed, and each fix was re-reviewed.
blacksmith: all seven typechecks pass. the full web suite passes (6,745 tests), along with the server, client-runtime and mobile tests for touched files.
runtime checks in headless chromium and built electron:
- history paging, fold expansion and prompt recall
- draft persistence across reload and unload
- cache compatibility both ways
- one real provider prompt in a scratch clone
- updater first use
a final review round found no new p0 or p1 in this layer. it fixed:
- prompt recall from a window with no prompt, paging at most one page per key press
- a stale file save clearing the unsaved marker
- diff panel turn coverage
- review previews staying stale after outside git changes
- turn start when baseline capture crashes
- a desktop trace export that could hang
- drafts lost on desktop quit
- p3s in the cache, settings and sidebar
the send-after-background-warm-up bug fixed in perf: desktop and server start without shell and fork waits, and thread views stay mounted #18219 was re-checked on this layer's rewritten composer: Enter sends after 2 s and 3.5 s waits.
two more review rounds fixed two p2s and several p3s:
- on linux, every snap shot capture re-ran the 2 MB desktop main bundle. a lazy chunk re-required the entry bundle after a dependency deleted it from node's module cache. a bundle test now fails if any chunk requires the entry.
- sidebar rows opened into view by a shelf or "show more" lost their entering animation.
- p3s:
  - a viewport turn whose checkpoint sat in the surplus rows was dropped
  - agent-only threads listed extra checkpoints
  - a startup crash hid its cause
  - a deleted thread's cache entry could be written back
  - older builds could read a viewport cache entry as complete
  - stale read rows weren't repaired on visits
  - card/slim rows slid instead of crossfading
  - confirm dialogs lost their fade-in
  - a streaming row could paint stale text for a frame
  - the branch toolbar label could overflow for a frame

known minor gaps, left for follow-ups:

when the loaded rows start partway through a run, "Worked for" shows a shorter time until the fold is expanded.
on mobile, expanding a fold for a partly loaded turn shows only the loaded rows, with no gap marker. "load earlier activity" above it still loads them.
the desktop main process skips definition-site stack frames for modules evaluated at startup, as the web app does.

unverified: macos and windows desktop, real gpu compositing, physical IME, native mobile, relay and tunnel at runtime.

model: claude-opus-5-5 coordinating gpt-6.1-sol and claude subagents, harness: claude code in t3 code. latest integration: gpt-6.1-sol through the codex harness in t3 code.

… failure or added context ArrowUp from an empty window waits for an older page. With no recall position, typing, a send or a failed page left that wait in place, so a later page filled the composer with an old or just-sent prompt. An image, file, terminal context, annotation, review comment, approval or question added while the page loaded was ignored too. Each of these now ends the wait.

…ipped When a concurrent render brought an older prompt than the editor held, the editor kept its text but its latest-value ref still pointed at the older prompt, so selection updates and focusAt waited for a rewrite that never came.

Coverage started at the tenth latest user turn and capped raw turns at 150, two turns short of a turn-aligned window. It now starts at the twelfth latest user turn with the same raw cap, and lists every run when there are fewer.

The 30 s idle TTL on diff previews made a panel reopened after a short idle show loading. Previews are keyed by scope again with the default TTL, and a new cache key refreshes the preview in place, so a superseded preview is never kept beside the current one. The preview request no longer carries the cache key: the server keeps no finished previews, so the key only joined the same atom's own in-flight request.

Mobile caches the viewport snapshot, whose deferred checkpoints have empty file lists. After an OTA rollback an older bundle showed those turns with no changes. Such entries are now wrapped in an envelope that older bundles fail to decode, so they drop the entry and load live, as web does.

Wave 3 added an optional cacheKey to diff preview requests and a server Cache keyed by it. Since previews stopped keeping finished reads and the web panel stopped sending the key, nothing uses it, and older clients never sent it. The contract field, the cache and its tests go back to the plain read.

The test still expected every other version's cache directory to go on start. Since 4514cbc only directories unused for a week go, so it now checks that a stale one goes and a recently used one stays.

LinuxSnapShot imported NiriSnapShot lazily, but the entry chunk already holds it, so the lazy chunk required application.cjs. conf evicts that file from the require cache, so every capture support check or capture re-ran the whole entry and added listeners. Import it statically, and check that no real chunk requires the entry.

The wrapper error's stack omitted its cause, so a packaged crash showed only "Desktop module initialization failed" at main.cjs. Let the original error propagate once the stack limit is restored.

The store reads two rows beyond the 75-row window and returns full checkpoints for every run in them. The bounded snapshot dropped a checkpoint whose run only reached those rows, so the Diff panel and Review skipped that turn until the user paged up. Keep it listed with its files deferred.

A thread with no user turn among its latest 152 turn starts listed deferred checkpoints back to the 152nd turn, and the list jumped between 151 and 152 turns. Base listed only the window's runs; match it.

Web's cache waits for an idle callback after the deleted check. A deletion in that wait removed the entry, and the delayed write then recreated it. Remove the entry again when the thread was deleted by the time the write lands.

…lete The compact envelope was written only when something was compacted or deferred, so a viewport window could be cached plain and an older build would resume from it as the whole thread. Write every thread entry in the envelope, which older builds skip in favor of live data. Plain entries still load.

Field patches only updated a read row that was current, so a row an older build marked stale stayed stale through visits and turns, and shell, search and sweep readers kept decoding the full payload. Rebuild a stale or missing row from the thread once, as these paths did before, and keep the cheap JSON patch for current rows.

Rows past the first-paint budget mounted as placeholders, so the motion pass animated the placeholder and the observer swapped in the real row at rest a frame later. Rows that join an already painted list now read their real box before paint, and the list motion moves an in-flight animation onto the element that replaces a slot. A row that leaves fades out as the element that was on screen rather than its stale slot.

A row that changed variant reused its old box by thread key, so a card settling into the Settled shelf slid its slim replacement across the list. Positions now carry over only when the variant matches: the card fades out where it was and the slim row fades in at its slot.

maria-rcks added 30 commits October 11, 2026 05:36

perf(server): load pending migrations and cache current schemas

af7ec2c

perf(server): parallelize boot reads and streamline desktop startup

8dadb4b

perf(server): avoid repeated thread and event encoding

f50e02b

fix(server): close inherited electron descriptors before startup

91f17f9

fix(desktop): release closed preview tab generations

d3ae6b7

fix(server): preserve electron-owned descriptors during cleanup

fef15ac

perf(server): bound vcs status retention without json fingerprints

e9bb84c

fix(server): retain subscribed vcs statuses until release

d9ead43

fix(server): capture baselines before provider startup

e3a148b

perf(server): reuse validated indexes for warm checkpoints

3045c12

fix(server): pin cached indexes to resolved heads

5921afd

perf(server): keep disk copies for cold checkpoint indexes

ea74e42

perf(client): preserve state references and write smaller caches

162493b

perf(threads): hydrate older history outside the initial viewport

79cfb20

perf(web): reuse markdown parses and observer row measurements

52f269b

perf(web): render streamed tokens in the assistant row

9ee41d6

perf(web): isolate sidebar row updates and viewport content

5204ffb

perf(web): skip zero displacement at drag release

cde71b2

perf(web): scale virtual rows with interface font preferences

a13eb0c

perf(web): defer boot codecs and measure layouts with refs

b0b4ee5

perf(bundle): remove duplicate themes and redundant dependencies

5a0c1e4

perf(desktop): use native fetch in the main process

bfd3540

chore(cursor): remove a missing keychain token entry point

5904198

perf(desktop): split main evaluation and defer the updater sdk

9dd6545

perf(desktop): render the cached composer without the session wait

ecd490f

perf(review): reuse previews until the workspace changes

d291118

perf(web): warm file viewer assets while picking files

35d437e

perf(web): reuse palette search work and defer focus

38bc8e2

perf(web): defer settings surfaces and batch row mounts

3bdabb8

perf(web): avoid panel layout reads and stream terminal wasm

b9f404b

maria-rcks added 18 commits October 11, 2026 15:09

fix(server): migration 063's loaders reference exports knip can see

15a1b5d

fix(web): settings code font previews attach their diff before paint

98f2568

chore: merge t3/perf-wave2 round-3 fixes

dd49453

test(server): compile cache test follows the week-unused cleanup rule

d7dbcce

The test still expected every other version's cache directory to go on start. Since 4514cbc only directories unused for a week go, so it now checks that a stale one goes and a recently used one stays.

chore: merge t3/perf-wave2 with #17609 37f8c26

bf4007b

fix(desktop): a startup crash shows its cause

44d73d2

The wrapper error's stack omitted its cause, so a packaged crash showed only "Desktop module initialization failed" at main.cjs. Let the original error propagate once the stack limit is restored.

fix(server): agent-only threads list only their viewport's turns

3d2da8f

A thread with no user turn among its latest 152 turn starts listed deferred checkpoints back to the 152nd turn, and the list jumped between 151 and 152 turns. Base listed only the window's runs; match it.

juliusmarminge added this pull request to stack #18313 October 11, 2026 19:04

maria-rcks added 9 commits October 11, 2026 19:10

fix(web): confirm dialogs animate in again

a6822cb

fix(web): a reply row that mounts mid-stream paints its live text

2a5fa31

fix(web): branch toolbar labels compact before new label text paints

ed4fef7

chore: merge t3/perf-wave2 round-4 fixes

51dff76

chore(perf): merge updated wave2 into wave3

4182b1f

Merge branch 'perf/wave2' into perf/wave3

3ea8e81

fix(server): normalize appended events like durable replay

625f214

bompus mentioned this pull request Oct 12, 2026

[Bug]: Message sends and queued starts decode every node of the thread first: 83 to 119 ms on 12,000 to 17,000-node threads #18438

Closed

chore(perf): merge updated wave2 into wave3

0f89a24

Navigation Menu

Repository navigation

perf: large threads open from a viewport snapshot, and typing and streaming render less - #18220

perf: large threads open from a viewport snapshot, and typing and streaming render less#18220
maria-rcks wants to merge 118 commits into
t3/perf-wave2from
t3/perf-wave3

maria-rcks commented Oct 11, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

maria-rcks commented Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

maria-rcks commented Oct 11, 2026 •

edited

Loading