Repository navigation
providers: retry a transient model-call failure instead of discarding the run - #62
Merged
Merged
Conversation
… the run A 12-pair audit lost all 12 calls to a single `Request timed out`, twice in a row. The failure propagated out of the per-pair list comprehension in audit_repo and nothing was written, so every completed pair was thrown away because one was unlucky. The fix distinguishes two failure modes that were previously treated alike. A missing CLI, an unknown model or a reached call ceiling fails identically on every attempt -- retrying wastes time and buries the cause. A timeout, a 429 or a 5xx is a property of the moment. Only the second kind is retried, three attempts with a linear backoff, and the decision is made on the message rather than the exception type because the bridge raises RuntimeError for everything and the type carries no information. Two details that matter more than the retry itself. EVERY RETRY IS COUNTED, in USAGE and in the usage line the user actually reads. A run that silently retried is indistinguishable from a clean one, and a flaky bridge would then look like a healthy one. `retries` is declared in USAGE rather than materialised on first use, so no consumer has to guard against a key that appears only when something went wrong. THE CALL CEILING COUNTS LOGICAL CALLS, NOT ATTEMPTS. `_guard()` runs once before the first attempt, so --max-calls limits the questions asked rather than the packets sent; otherwise a flaky connection would silently halve a stated ceiling. A test pins that. Six contract tests cover both directions: a timeout is retried and counted, a missing CLI is not retried and its error reaches the caller unchanged, attempts are bounded with the real error surfacing rather than a generic one, and the ceiling is never treated as transient -- which would turn a deliberate limit into a delay. The whole thing was then exercised against the actual failure shape: a 12-call run with a timeout on the fifth now completes, reporting 12 calls and 1 retry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A 12-pair audit lost all 12 calls to a single
Request timed out— twice in a row. The failure propagated out of the per-pair list comprehension inaudit_repoand nothing was written, so every completed pair was thrown away because one was unlucky.The distinction the code was missing
The decision is made on the message, not the exception type, because the bridge raises
RuntimeErrorfor every failure mode and the type carries no information.Two details that matter more than the retry
Every retry is counted — in
USAGEand in the usage line the user actually reads. A run that silently retried is indistinguishable from a clean one, and a flaky bridge would then look like a healthy one.retriesis declared inUSAGErather than materialised on first use, so no consumer has to guard against a key that only appears when something went wrong.The call ceiling counts logical calls, not attempts.
_guard()runs once before the first attempt, so--max-callslimits the questions asked rather than the packets sent — otherwise a flaky connection would silently halve a stated ceiling. There's a test for that specifically.Tests
Six contract tests, covering both directions:
Then exercised against the actual failure shape: a 12-call run with a timeout on the fifth call now completes, reporting
12 calls · 1 retry.ruff check .passes ·pytest342 passed (336 + 6)🤖 Generated with Claude Code