Skip to content

Add retries to the pipeline - #1072

Open
kean wants to merge 11 commits into
mainfrom
feat/retries
Open

kean wants to merge 11 commits into
mainfrom
feat/retries

Conversation

@kean

@kean kean commented Oct 31, 2026

Copy link
Copy Markdown
Owner

Adds automatic retries, off by default, with one line to turn them on:

ImagePipeline.shared = ImagePipeline { $0.retryPolicy = .default }

The default policy retries twice, about 0.5 s and 1 s apart, on transient URLErrors and on 408, 429, 500, 502, 503, and 504. It honors Retry-After, never repeats a request that isn't idempotent, and leaves Low Data Mode failures alone. A load that fails because the device is offline waits for it to get back online, for up to 30 s, instead of spending its attempts.

API

  • ImagePipeline.RetryPolicy, set on Configuration.retryPolicy or ImageRequest.retryPolicy. .disabled opts a request out.
  • Delegate.retryDecision(for:pipeline:) for what a policy can't express. It's async, and every attempt goes through willLoadData again, so it can refresh a token.
  • ImageTask.Status.retryCount, plus ImageTask.Metrics.retryCount, per-attempt errors, a retryDelay stage, and a retry share of the time. summary and snapshot() report retries too, so a task that is waiting doesn't look stuck.
  • reload() on FetchImage, LazyImageState, and LazyImageView, which replaces the .id(UUID()) recipe.

Why it's in the fetch and not in ImageTask

The retries run in the task that fetches the data (AsyncFetchTask, a new layer under the two fetch tasks). Coalesced tasks share one sequence, so twenty cells with the same avatar make one set of requests rather than twenty, and the delegate is asked once per failed attempt.

A task that is waiting gives its dataLoadingQueue slot back, so a dead host doesn't hold up the others. Each attempt goes through the rate limiter and the queue again, at the current priority, and resumes what the failed one downloaded.

The policy stays out of the task keys. On a failure, the job asks the requests waiting on it and uses the one that allows the most retries, so a prefetch and a request with a different policy still coalesce, and this costs nothing until something fails. ImagePrefetcher tasks skip the policy of the pipeline, because prefetching is speculative, but a request that joins one brings its own.

There's no new ImageTask.Event case, because Event is @frozen.

Proof

  • .scripts/ci.sh: all 22 jobs pass on Xcode 27.1 RC, with Nuke at 1899 tests on iOS, 1896 on tvOS, and 1852 on macOS.
  • The NukeTests scheme under TSan, the 32 tests in NukeThreadSafetyTests, and NukeVideo pass.
  • 116 new tests in ImagePipelineRetryTests, RetryPolicyTests, and ConnectivityMonitorTests ran 40 times without a failure at a load average near 90, half of them under TSan.
  • Breaking each of 11 safeguards on purpose failed a test: stale callbacks, the reset between attempts, the decoder reset, the prefetch exemption, the shared policy, cancelling the wait, and the rest.

Not checked: it hasn't been compiled on Xcode 26.x, which CI uses for the oldest macOS job, because the 26.6 install here has no swiftc.

Performance

Measured as instructions retired, with no retry policy, in a release harness over the public API, interleaved against main. The machine was at a load average of 60–100, so there are no wall-clock numbers.

Change
ImageRequest creation +0.25% Real: 9 of 9 rounds, spread 0.04%, about 21 instructions for one more pointer
32-chunk download −0.03% Flat
Cancelled task +0.21% Flat
Memory cache hit −0.99% Flat
Load +0.85% 21 rounds. An A/A control read +0.34%
Data load +0.96% 21 rounds. An A/A control read −0.09%

The last two are small but aren't clearly noise, and nothing in the success path accounts for them. These numbers predate the rebase onto #1031. NukePerformanceTests needs a run on a quiet machine before this merges.

Before merging

  • Stage.Kind.retryDelay and Metrics.Category.retry are new enum cases, so this is additive only if it lands before 14.0. After that, they would break exhaustive switches.
  • Fix resumable downloads retrying a rejected range, progress on a resumed download without Content-Length, and empty local resources #1026 is still open. Without it, a custom policy that retries 416 repeats the rejected range. The default policy doesn't retry 416.
  • The diagnostics schema goes to 3 for the new stage kind and Stage.error.
  • A custom delegate with no policy adds one actor hop to every failure, because the delegate has to be asked. The default delegate stays synchronous.
  • Not included: fallback URLs and the Low Data Mode fallback, which are separate items in the plan, and retrying decode failures, because the same bytes would fail again.
  • No CHANGELOG entry yet.

@kean
kean force-pushed the feat/retries branch 4 times, most recently from 82cfa5f to cd5476a Compare November 1, 2026 14:56
kean added 4 commits November 2, 2026 19:26
A failed load can be retried with `ImagePipeline.RetryPolicy`, set on
`Configuration/retryPolicy` or `ImageRequest/retryPolicy`, or with the new
`Delegate/retryDecision(for:pipeline:)`, which is async so that it can
refresh a token. Retries are off by default.

The retries run in the task that fetches the data rather than in `ImageTask`,
so tasks that coalesce share one sequence of them. A task that is waiting
gives its slot in `dataLoadingQueue` back, and each attempt goes through the
rate limiter, the queue, and `willLoadData` again. The policy stays out of the
task keys: on a failure the task asks its subscribers for theirs, so a prefetch
and a request with a different policy still coalesce.

The policy honors `Retry-After`, read off the `URLSessionTask` since
`DataLoader` rejects these responses before the pipeline sees them, and waits
for the device to get back online instead of spending attempts while offline.

Every attempt is a `download` stage in the diagnostics, which now record the
error of each one, and the wait between them is a `retryDelay` stage. The
readers that took the first download stage take the last one, `summary` ends
with the number of retries, `snapshot()` says which tasks are waiting to retry
so that they don't look stuck, and the schema version is bumped.
`ImageLoader.reload()` was only reachable from the loader, which left
`.id(UUID())` as the way to retry from a `LazyImage`. The requirement has a
default implementation, so other conformances of `LazyImageState` keep working.
Add "Retrying Failed Requests" and link it from the articles that touch on the
subject. The "Auto Retry" section of the performance guide now leads with the
policy, because `waitsForConnectivity` keeps the slot in the queue while it
waits. `troubleshooting.md` explains the new parts of the log and the snapshot.
The samples are compiled in `RetrySnippets.swift`.
Replaces the two walks that used a predicate returning false to reach
every task. Also explains why attemptDidFail replaces onCancelled.
kean added 7 commits November 5, 2026 19:36
A chunk that arrives while a preview is being decoded is skipped, and the
mock server sends every chunk at once. When the failed attempt's decode
was still running, the retry downloaded the whole image without decoding a
preview, so it never created a second decoder, and the test failed on the
Xcode 27 iOS job.
The branch used when diagnostics are off didn't pass the attempt, so a
completion URLSession delivers for an attempt the pipeline already ended,
such as one over the size limit, could resume the retry with the
cancellation, and the retry's own task was never cancelled.

The attempt is now a required argument, so the compiler catches a call
that leaves it out.
A shared load took the policy that allows the most retries, even when it
didn't retry the failure and another request's policy did, so a 503 failed
at once for the request that would have retried it. The policy is now the
one that retries the failure, and of those, the one that allows the most
retries.

Without a delegate, the policy decides on the spot, so a failure that
isn't retried, such as a 404, fails without a Task and an actor hop, and a
cancellation can't turn it into `.cancelled` in between.

The delegate isn't asked about a load that was cancelled before the retry
task ran. The wait for a retry starts with the failure, so a snapshot says
the load is waiting while the delegate decides, and the time it takes,
such as for a token refresh, is in the `retryDelay` stage.
With a delegate, the download stage of the failed attempt stayed current
until `willLoadData` returned on the retry. If it threw, the stage was
ended a second time, with 0 bytes and the delegate's error. And with no
download for that attempt, `Metrics.retryCount` didn't count the retry
that `Status.retryCount` did. An attempt with a delegate now counts by its
`willLoadData` stage.

An empty response ended its download stage before failing with
`dataIsEmpty`, so the stage had no error. It now has it, from a data
loader and from a closure.
When a retry couldn't resume, the decoder was dropped, but a preview of
the old bytes that was still being decoded went out after the restart, and
it can be of a different image. It's now dropped.

The limits on evaluating the preview policy are per download, and they
weren't reset, so a first attempt that used them up left the retry
without previews.
`NWPathMonitor` reports the path a moment after it starts, and a short
timeout ran out first, so `.retryWhenOnline(after: 0, timeout: 0)` always
failed, even online. A wait whose time runs out before the first report
now ends with that report, whatever it says.

A late call from an observer that was already stopped is ignored, since
it would otherwise count as the first report of the next one.
`RetryDecision` and `RetryPolicy.Backoff` are the kind of enum that gets
a case later, such as a retry with a changed request, and the rest of the
public enums say whether they will.

`retryAfter` created a `DateFormatter` for every HTTP date it parsed, and
when a host goes down, every image on the screen fails at once.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant