Skip to content

perf: fetch the largest bundle files first - #1064

Merged
LynithDev merged 2 commits into
Polyfrost:mainfrom
tunaseckin:perf/fetch-largest-bundle-files-first
Oct 25, 2026
Merged

LynithDev merged 2 commits into
Polyfrost:mainfrom
tunaseckin:perf/fetch-largest-bundle-files-first

Conversation

@tunaseckin

Copy link
Copy Markdown
Contributor

Bundle files are prefetched six at a time and the phase waits for the last one to finish. They were fetched in catalog order, so a large file near the end of the list started after most of the small ones were done, and everything then waited on it alone.

This came out of going through my own launcher logs. Across the four slowest update batches, every stall was a large file still downloading after the rest had finished: PolyPlus (52 MB) twice and SmartSearch (60 MB) twice, each started near the end of its batch. In one 26.2 batch of 38 updates the small files were all started within 5s, then the phase sat on SmartSearch for another 14.5s.

Sorting to_fetch by size puts the large files in the first slots, where they overlap the small ones. From that batch's timings the phase should drop from about 19.6s to about 14.5s, though I have only estimated that from the logs, not measured it with the change.

It also helps with the deadline: past() only stops new fetches from starting, so a large file started first always runs, where today it can be the one that starts too late and gets skipped.

cargo test -p oneclient_content --lib has one failure, local_manifest::tests::fabric_dependencies_list_depends_keys, but it fails the same way on main without this change.

Bundle files are prefetched six at a time and the phase waits for the
last one to finish. They were fetched in catalog order, so a large file
near the end of the list started after most of the small ones had
already finished, and everything then waited on it alone.

Sorting by size puts the large files in the first slots, where they
overlap the small ones. It also starts them before the deadline can stop
new fetches, so the file that most needed prefetching is no longer the
one most likely to be skipped.
@LynithDev
LynithDev merged commit 581d787 into Polyfrost:main Oct 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants