Repository navigation
Hold the paused reply once in ToolCallLimitExceededException history - #7213
Open
Ridzzz0Alam wants to merge 1 commit into
Open
Ridzzz0Alam wants to merge 1 commit into
Ridzzz0Alam wants to merge 1 commit into
Conversation
When an application resumes a paused tool call batch, the prompt may already end with the paused reply. `DefaultToolCallingManager` drops that trailing reply before it builds the conversation history, so the reply appears once. The branch that throws `ToolCallLimitExceededException` still built its partial history from the raw prompt instructions, so the reply appeared twice, followed by the partial tool results. An application that sends that history back to the model gets an error, because the first copy of the reply has tool calls without results. Compute the de-duplicated previous messages before the calls run and use them for both the regular and the partial history. Signed-off-by: Ridzzz0Alam <ridwan0gts@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When an application resumes a paused tool call batch with
ToolCallingManager#executeToolCalls(Prompt, ChatResponse, ToolCallDecisions)the prompt may already end with the paused reply. This happens for example when the application passes its whole conversation.DefaultToolCallingManagerdrops that trailing reply before it builds the conversation history so the reply appears only once.The branch that throws
ToolCallLimitExceededExceptiondid not do the same. It built the partial history from the rawprompt.getInstructions()so the paused reply appeared twice:An application that catches the exception and sends
getPartialToolExecutionResult().conversationHistory()back to the model gets an error from the provider. The first copy of the reply carries tool calls without results.Reproducer
Fix
Compute the de-duplicated previous messages before any call runs and pass them to
executeToolCall. The regular history and the partial history of a limit breach are then built from the same messages. The methodexecuteToolCallis private so there is no API change.How tested
Added
promptEndingWithTheReplyHoldsItOnceInThePartialHistoryOfLimitBreachtoDefaultToolCallingManagerDecisionsTests. The test fails without the fix (expected size 3 but was 4) and passes with it.The
spring-ai-modelmodule on JDK 17.0.19 ran 1037 tests with 0 failures.The async tool calling proposal in #4756 builds its own history on the limit path. It may need the same handling if it is picked up again.