Skip to content

fix(execution): surface a ModelRefusalError the same as a structured refusal - #1319

Open
itzzdev09 wants to merge 2 commits into
usestrix:mainfrom
itzzdev09:fix/model-refusal-clear-surfacing
Open

itzzdev09 wants to merge 2 commits into
usestrix:mainfrom
itzzdev09:fix/model-refusal-clear-surfacing

Conversation

@itzzdev09

Copy link
Copy Markdown
Contributor

Closes #1295

The bug

_run_cycle already recognises one shape of provider refusal: an OpenAI-style structured refusal content item, wrapped as this module's own ProviderRefusalError and given a clean, single-line logger.warning, a "failed" status, and no retry.

The agents SDK has a second shape: it raises agents.exceptions.ModelRefusalError directly when a provider reports a refusal without that structured content. #1295 reports this happening with Gemini 3.7 Flash through LiteLLM: ModelRefusalError: Model refused to produce output: Response withheld by the provider's content filter.

That shape wasn't recognised, so it fell through to the generic except Exception path:

  • a full ERROR-level traceback via logger.exception, instead of the one-line warning the equivalent structured refusal gets;
  • the SDK's own "Model refused to produce output: ..." wrapper text stored as the agent's error, instead of the plain refusal text.

Functionally the agent still ended up "failed" and not silently dropped, so scan coverage wasn't reduced further than a refusal already reduces it, but the two identical situations were surfaced very differently.

Change

_refusal_text(exc) returns the refusal text for either ProviderRefusalError or ModelRefusalError, and the one existing check is written against it instead of just ProviderRefusalError. Both are also excluded from _is_transient_model_error, matching ProviderRefusalError's existing behaviour (a refusal is a decision, not a transient failure, so retrying is not attempted).

Tests

Added test_sdk_model_refusal_fails_cleanly_without_a_traceback, mirroring the existing test_structured_provider_refusal_fails_interactive_agent but with a stream whose run_loop_exception is a ModelRefusalError instead of a structured refusal item. It asserts the status, the plain refusal text, and that nothing at ERROR level with a traceback was logged.

Proved against main's execution.py: the new test fails there (wrong error text, and logger.exception fires) and passes with the change.

tests/test_execution.py: 62 passed. Full suite: same 32 failures as on main (Windows-specific test assumptions unrelated to this file, already tracked by other issues), and none new. ruff 0.15.20 check and format are clean; mypy --platform linux/darwin/win32 report nothing new in execution.py (their reported errors are pre-existing, in unrelated files pulled in transitively).

…refusal

The agents SDK raises ModelRefusalError directly for some providers (observed with Gemini through LiteLLM) instead of returning a structured refusal content item. Only the latter was recognised, so the former fell through to the generic crash path: a full ERROR-level traceback instead of the clean one-line warning a refusal gets, and the SDK's redundant "Model refused to produce output: ..." wrapper text stored as the error instead of the plain refusal.

Both are now recognised as the same non-retryable outcome, with the same warning-level log and the same plain refusal text.
@greptile-apps

greptile-apps Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

The PR appears safe to merge after correcting the minor provider-name typo in the new test documentation.

Findings

  1. P2 LiteLLM Name Is Misspelled
Fix with agent prompt
### Issue 1
tests/test_execution.py:35
The new fixture docstring spells the provider integration as `LiteLTM` instead of `LiteLLM`. This makes the provider-specific context harder to search and may confuse future maintainers.

```suggestion
    Some providers (observed with Gemini through LiteLLM) surface a content-
```

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

This PR normalizes directly raised SDK model refusals with existing structured provider refusals.

  • Extracts plain refusal text from either refusal exception shape.
  • Prevents model refusals from entering transient retry handling.
  • Records a clean failed status, emits a warning without a traceback, and preserves terminal parent notification.
  • Adds regression coverage for an SDK-raised ModelRefusalError.

Reviews (1) · Last reviewed commit: "fix(execution): surface a ModelRefusalEr..."

Comment thread tests/test_execution.py Outdated

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gemini 3.7 Flash provider content filtering causes Strix agents to fail during autonomous security scans

1 participant