Conversation
…refusal The agents SDK raises ModelRefusalError directly for some providers (observed with Gemini through LiteLLM) instead of returning a structured refusal content item. Only the latter was recognised, so the former fell through to the generic crash path: a full ERROR-level traceback instead of the clean one-line warning a refusal gets, and the SDK's redundant "Model refused to produce output: ..." wrapper text stored as the error instead of the plain refusal. Both are now recognised as the same non-retryable outcome, with the same warning-level log and the same plain refusal text.
Contributor
|
LiteLTM -> LiteLLM
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1295
The bug
_run_cyclealready recognises one shape of provider refusal: an OpenAI-style structuredrefusalcontent item, wrapped as this module's ownProviderRefusalErrorand given a clean, single-linelogger.warning, a "failed" status, and no retry.The agents SDK has a second shape: it raises
agents.exceptions.ModelRefusalErrordirectly when a provider reports a refusal without that structured content. #1295 reports this happening with Gemini 3.7 Flash through LiteLLM:ModelRefusalError: Model refused to produce output: Response withheld by the provider's content filter.That shape wasn't recognised, so it fell through to the generic
except Exceptionpath:logger.exception, instead of the one-line warning the equivalent structured refusal gets;Functionally the agent still ended up "failed" and not silently dropped, so scan coverage wasn't reduced further than a refusal already reduces it, but the two identical situations were surfaced very differently.
Change
_refusal_text(exc)returns the refusal text for eitherProviderRefusalErrororModelRefusalError, and the one existing check is written against it instead of justProviderRefusalError. Both are also excluded from_is_transient_model_error, matchingProviderRefusalError's existing behaviour (a refusal is a decision, not a transient failure, so retrying is not attempted).Tests
Added
test_sdk_model_refusal_fails_cleanly_without_a_traceback, mirroring the existingtest_structured_provider_refusal_fails_interactive_agentbut with a stream whoserun_loop_exceptionis aModelRefusalErrorinstead of a structured refusal item. It asserts the status, the plain refusal text, and that nothing at ERROR level with a traceback was logged.Proved against
main'sexecution.py: the new test fails there (wrong error text, andlogger.exceptionfires) and passes with the change.tests/test_execution.py: 62 passed. Full suite: same 32 failures as onmain(Windows-specific test assumptions unrelated to this file, already tracked by other issues), and none new.ruff0.15.20 check and format are clean;mypy --platformlinux/darwin/win32 report nothing new inexecution.py(their reported errors are pre-existing, in unrelated files pulled in transitively).