Repository navigation
perf: fetch the largest bundle files first - #1064
Merged
LynithDev merged 2 commits intoOct 25, 2026
Merged
Conversation
Bundle files are prefetched six at a time and the phase waits for the last one to finish. They were fetched in catalog order, so a large file near the end of the list started after most of the small ones had already finished, and everything then waited on it alone. Sorting by size puts the large files in the first slots, where they overlap the small ones. It also starts them before the deadline can stop new fetches, so the file that most needed prefetching is no longer the one most likely to be skipped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bundle files are prefetched six at a time and the phase waits for the last one to finish. They were fetched in catalog order, so a large file near the end of the list started after most of the small ones were done, and everything then waited on it alone.
This came out of going through my own launcher logs. Across the four slowest update batches, every stall was a large file still downloading after the rest had finished: PolyPlus (52 MB) twice and SmartSearch (60 MB) twice, each started near the end of its batch. In one 26.2 batch of 38 updates the small files were all started within 5s, then the phase sat on SmartSearch for another 14.5s.
Sorting
to_fetchby size puts the large files in the first slots, where they overlap the small ones. From that batch's timings the phase should drop from about 19.6s to about 14.5s, though I have only estimated that from the logs, not measured it with the change.It also helps with the deadline:
past()only stops new fetches from starting, so a large file started first always runs, where today it can be the one that starts too late and gets skipped.cargo test -p oneclient_content --libhas one failure,local_manifest::tests::fabric_dependencies_list_depends_keys, but it fails the same way onmainwithout this change.