feat(models): forward MCP OAuth scopes in claude_code and codex_cli generators

docs/claude_code_agent_testing.md

@@ -284,7 +284,7 @@ EvalBench accepts the **same MCP server config schema as Gemini CLI** for HTTP s
 |---|---|
 | `httpUrl` | → `url` + auto-adds `type: "http"` |
 | `authProviderType: google_credentials` | → injects `Authorization: Bearer <ADC token>` (from `gcloud auth application-default print-access-token`, falling back to `gcloud auth print-access-token`) **and** sets a `headersHelper` so Claude Code re-mints a fresh ADC token on every connection (avoids ~1h expiry). Google API MCP endpoints reject the plain user token on tool calls — see [Troubleshooting](#mcp-tool-call-fails-with-incompatible-auth-server-does-not-support-dynamic-client-registration). |
-| `oauth.scopes` | (dropped — Claude Code doesn't use Gemini's OAuth delegation) |
+| `oauth.scopes` | → forwarded as `--scopes` to `gcloud auth application-default print-access-token` in both `headersHelper` and the initial static `Authorization` header. If the scoped token request fails, the generator falls back to an unscoped ADC token (see [Troubleshooting](#user-adc-and-non-default-oauth-scopes-403-forbidden-on-tool-call)). |
 | `headers` | → passed through as-is |
 | `command` / `args` (stdio) | → passed through as-is |
 
@@ -490,6 +490,17 @@ Usually a token problem (see the DCR entry above). Checklist:
 - Set the quota project header: `headers: { X-Goog-User-Project: <project> }`.
 - Verify directly: `curl -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" -H "X-Goog-User-Project: <project>" -H "Content-Type: application/json" -H "Accept: application/json, text/event-stream" -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"list_instances","arguments":{"project":"<project>"}}}' https://sqladmin.googleapis.com/mcp`
 
+### User ADC and Non-Default OAuth Scopes (`403 Forbidden` on tool call)
+
+When using user credentials (via `gcloud auth application-default login`), `gcloud` only requests the default `cloud-platform` scopes by default. If an MCP server specifies a non-default OAuth scope (e.g. DFA Reporting or Google Ads), `gcloud auth application-default print-access-token --scopes=...` will fail unless that scope was explicitly granted at login time.
+
+When the scoped request fails, the generator quietly falls back to an **unscoped ADC token**. The MCP server connects normally, but actual tool calls will later fail with a confusing `403 Forbidden`.
+
+To resolve this with user ADC, re-authenticate and explicitly specify the required scopes:
+```bash
+gcloud auth application-default login --scopes="https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/<scope>"
+```
+
 ### `npm exec` is slow on first run
 
 `npm exec --yes <package>@<version>` downloads the package on first use (~30 sec). Subsequent runs use the cache.

docs/codex_cli_agent_testing.md

@@ -305,7 +305,7 @@ EvalBench accepts the **same MCP server config schema as Gemini CLI and Claude C
 | `httpUrl` | → `url` (TOML, streamable HTTP server) |
 | `headers` | → `http_headers` (TOML inline table) |
 | `authProviderType: google_credentials` | → mints an **ADC** token (`gcloud auth application-default print-access-token`, falling back to `gcloud auth print-access-token`) and passes it to Codex via `bearer_token_env_var` (env var `EVALBENCH_GCLOUD_MCP_TOKEN`). The generator re-mints a fresh token before **every turn** (each `codex exec` re-reads the env var), so it doesn't expire mid-suite. `X-Goog-User-Project` stays a static `http_headers` entry. Google API MCP endpoints reject the plain user token on tool calls — see [Troubleshooting](#mcp-server-fails-with-401-unauthorized-real-cloud-sql-endpoint). |
-| `oauth.scopes` | (dropped — Codex doesn't use Gemini's OAuth delegation) |
+| `oauth.scopes` | → forwarded as `--scopes` to `gcloud auth application-default print-access-token` when minting `bearer_token_env_var` (`EVALBENCH_GCLOUD_MCP_TOKEN`). If the scoped token request fails, the generator falls back to an unscoped ADC token (see [Troubleshooting](#user-adc-and-non-default-oauth-scopes-403-forbidden-on-tool-call)). |
 | `command` / `args` / `env` / `cwd` (stdio) | → passed through as-is into a `[mcp_servers.NAME]` stdio block |
 
 ### HTTP MCP server (Cloud SQL Managed)
@@ -575,6 +575,17 @@ The injected token is missing, expired, the **wrong kind**, or your principal la
 - `X-Goog-User-Project` header points at a project that has the Cloud SQL Admin API enabled.
 - Token expiry is handled automatically: the generator mints a fresh ADC token into `EVALBENCH_GCLOUD_MCP_TOKEN` before every turn and Codex reads it via `bearer_token_env_var`, so long suites don't hit stale-token 401s. (Requires a Codex build that supports `bearer_token_env_var`; if yours doesn't, the token won't be applied — fall back to a static `http_headers` `Authorization`.)
 
+### User ADC and Non-Default OAuth Scopes (`403 Forbidden` on tool call)
+
+When using user credentials (via `gcloud auth application-default login`), `gcloud` only requests default scopes by default. If an MCP server specifies a non-default OAuth scope (e.g. DFA Reporting or Google Ads), `gcloud auth application-default print-access-token --scopes=...` will fail unless that scope was explicitly granted at login time.
+
+When the scoped request fails, the generator quietly falls back to an **unscoped ADC token**. The MCP server connects normally, but actual tool calls will subsequently fail with a confusing `403 Forbidden`.
+
+To resolve this with user ADC, re-authenticate and explicitly specify the required scopes:
+```bash
+gcloud auth application-default login --scopes="https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/<scope>"
+```
+
 ### `Invalid TOML` when Codex starts
 
 Either a manual edit to `~/.codex/config.toml` clobbered the generated file, or a hand-written `setup.config` value contains an unsupported type. Fix:

evalbench/generators/models/claude_code.py

@@ -1,4 +1,5 @@
 from .agent_cli import AgentCliGenerator
+from . import mcp_client
 from .tool_naming import canonicalize_claude_tool_name
 import subprocess
 import os
@@ -290,7 +291,11 @@ def _translate_mcp_config(self, server_name: str, config: dict) -> dict:
 
         # Translate `authProviderType: google_credentials` into Bearer header
         auth_provider = config.pop("authProviderType", None)
-        # Gemini-style `oauth.scopes` is ignored by Claude Code; drop it
+
+        # Gemini-style `oauth.scopes` is forwarded to gcloud, but the `oauth`
+        # block itself must be removed so Claude Code doesn't treat it as an
+        # interactive MCP OAuth 2.0 dynamic registration config.
+        scopes = mcp_client.extract_mcp_oauth_scopes(config)
         config.pop("oauth", None)
 
         if auth_provider == "google_credentials":
@@ -304,12 +309,12 @@ def _translate_mcp_config(self, server_name: str, config: dict) -> dict:
             # Code's env), it prints nothing and Claude Code falls back to the
             # static header below.
             config.setdefault(
-                "headersHelper", self._google_credentials_headers_helper())
+                "headersHelper", self._google_credentials_headers_helper(scopes=scopes))
 
             # Baked static token: the initial value and the fallback for when
             # headersHelper can't run. headersHelper output takes precedence.
             if "Authorization" not in headers:
-                token = self._fetch_gcloud_access_token()
+                token = self._fetch_gcloud_access_token(scopes=scopes)
                 if token:
                     headers["Authorization"] = f"Bearer {token}"
                 else:
@@ -322,7 +327,9 @@ def _translate_mcp_config(self, server_name: str, config: dict) -> dict:
         return config
 
     @staticmethod
-    def _google_credentials_headers_helper() -> str:
+    def _google_credentials_headers_helper(
+        scopes: Optional[list[str]] = None,
+    ) -> str:
         """Shell command Claude Code executes on each MCP connection to mint a
         fresh Google bearer token.
 
@@ -333,13 +340,25 @@ def _google_credentials_headers_helper() -> str:
         prints nothing (non-zero exit) when neither is available so Claude Code
         keeps using the baked static header.
         """
+        scoped_cmd = ""
+        if scopes:
+            for s in scopes:
+                assert re.fullmatch(r"[A-Za-z0-9_.:/-]+", s), (
+                    f"Invalid or unsafe OAuth scope: {s!r}"
+                )
+            scopes_str = ",".join(scopes)
+            scoped_cmd = (
+                f'gcloud auth application-default print-access-token --scopes="{scopes_str}" 2>/dev/null || '
+            )
         return (
-            'tok="$(gcloud auth application-default print-access-token '
+            f'tok="$({scoped_cmd}gcloud auth application-default print-access-token '
             '2>/dev/null || gcloud auth print-access-token 2>/dev/null)"; '
             '[ -n "$tok" ] && printf \'{"Authorization":"Bearer %s"}\' "$tok"'
         )
 
-    def _fetch_gcloud_access_token(self) -> str:
+    def _fetch_gcloud_access_token(
+        self, scopes: Optional[list[str]] = None
+    ) -> str:
         """Fetches a Google Cloud access token for MCP `google_credentials` auth.
 
         Prefers Application Default Credentials (``gcloud auth
@@ -372,10 +391,26 @@ def _fetch_gcloud_access_token(self) -> str:
             token_env["GOOGLE_APPLICATION_CREDENTIALS"] = adc
         # ADC first (works for these endpoints); user token as a fallback for
         # any MCP server that happens to accept it.
-        commands = [
+        commands = []
+        if scopes:
+            for s in scopes:
+                assert re.fullmatch(r"[A-Za-z0-9_.:/-]+", s), (
+                    f"Invalid or unsafe OAuth scope: {s!r}"
+                )
+            scopes_str = ",".join(scopes)
+            commands.append(
+                [
+                    "gcloud",
+                    "auth",
+                    "application-default",
+                    "print-access-token",
+                    f"--scopes={scopes_str}",
+                ]
+            )
+        commands.extend([
             ["gcloud", "auth", "application-default", "print-access-token"],
             ["gcloud", "auth", "print-access-token"],
-        ]
+        ])
         for cmd in commands:
             try:
                 result = subprocess.run(

evalbench/generators/models/codex_cli.py

@@ -1,3 +1,4 @@
+from . import mcp_client
 from .agent_cli import AgentCliGenerator
 from .tool_naming import canonical_tool_name
 import subprocess
@@ -123,6 +124,7 @@ def __init__(self, querygenerator_config):
         self.config_path = os.path.join(self.codex_config_dir, "config.toml")
         self.inline_mcp_servers = {}
         self.enabled_plugins = {}
+        self._gcloud_mcp_scopes = []
         self._setup()
 
     @staticmethod
@@ -607,6 +609,9 @@ def _translate_mcp_config(self, server_name: str, config: dict) -> dict:
         out: dict = {"url": url}
         headers = dict(config.get("headers") or {})
 
+        scopes = mcp_client.extract_mcp_oauth_scopes(config)
+        config.pop("oauth", None)
+
         auth_provider = config.get("authProviderType")
         if auth_provider == "google_credentials" and "Authorization" not in headers:
             # Supply the bearer token via `bearer_token_env_var` rather than a
@@ -618,7 +623,12 @@ def _translate_mcp_config(self, server_name: str, config: dict) -> dict:
             # stays a static header.
             out["bearer_token_env_var"] = self._GCLOUD_MCP_TOKEN_ENV
             self._needs_gcloud_mcp_token = True
-            token = self._fetch_gcloud_access_token()
+            if not hasattr(self, "_gcloud_mcp_scopes"):
+                self._gcloud_mcp_scopes = []
+            for s in scopes:
+                if s and s not in self._gcloud_mcp_scopes:
+                    self._gcloud_mcp_scopes.append(s)
+            token = self._fetch_gcloud_access_token(scopes=scopes)
             if token:
                 self.env[self._GCLOUD_MCP_TOKEN_ENV] = token  # initial value
             else:
@@ -630,7 +640,9 @@ def _translate_mcp_config(self, server_name: str, config: dict) -> dict:
             out["http_headers"] = headers
         return out
 
-    def _fetch_gcloud_access_token(self) -> str:
+    def _fetch_gcloud_access_token(
+        self, scopes: Optional[list[str]] = None
+    ) -> str:
         """Fetches a Google Cloud access token for MCP `google_credentials` auth.
 
         Prefers Application Default Credentials (``gcloud auth
@@ -652,10 +664,30 @@ def _fetch_gcloud_access_token(self) -> str:
         adc = self.env.get("GOOGLE_APPLICATION_CREDENTIALS")
         if adc and os.path.exists(adc):
             token_env["GOOGLE_APPLICATION_CREDENTIALS"] = adc
-        commands = [
+
+        if scopes is None:
+            scopes = getattr(self, "_gcloud_mcp_scopes", None)
+
+        commands = []
+        if scopes:
+            for s in scopes:
+                assert re.fullmatch(r"[A-Za-z0-9_.:/-]+", s), (
+                    f"Invalid or unsafe OAuth scope: {s!r}"
+                )
+            scopes_str = ",".join(scopes)
+            commands.append(
+                [
+                    "gcloud",
+                    "auth",
+                    "application-default",
+                    "print-access-token",
+                    f"--scopes={scopes_str}",
+                ]
+            )
+        commands.extend([
             ["gcloud", "auth", "application-default", "print-access-token"],
             ["gcloud", "auth", "print-access-token"],
-        ]
+        ])
         for cmd in commands:
             try:
                 result = subprocess.run(
@@ -843,7 +875,9 @@ def _run_codex_cli(self, cli_cmd: CLICommand, timeout_seconds=None):
         # is a fresh process that re-reads `bearer_token_env_var`, so minting a
         # new ADC token here keeps it from expiring across a long suite.
         if getattr(self, "_needs_gcloud_mcp_token", False):
-            token = self._fetch_gcloud_access_token()
+            token = self._fetch_gcloud_access_token(
+                scopes=getattr(self, "_gcloud_mcp_scopes", None)
+            )
             if token:
                 env[self._GCLOUD_MCP_TOKEN_ENV] = token
 

evalbench/generators/models/mcp_client.py

@@ -27,6 +27,23 @@ def _format_error(e: BaseException) -> str:
     return f"{type(e).__name__}: {e}" if str(e) else type(e).__name__
 
 
+def extract_mcp_oauth_scopes(server_config: dict) -> list[str]:
+    """Extracts OAuth scopes from an MCP server config.
+
+    Accepts the canonical Gemini CLI schema:
+      oauth:
+        scopes:
+          - https://www.googleapis.com/auth/...
+    """
+    oauth = server_config.get("oauth")
+    if not isinstance(oauth, dict):
+        return []
+    scopes = oauth.get("scopes")
+    if not isinstance(scopes, list):
+        return []
+    return [str(s).strip() for s in scopes if str(s).strip()]
+
+
 def auth_headers(server_config: dict) -> dict | None:
     """Build request headers for an MCP server (configured headers + auth).
 
@@ -39,7 +56,7 @@ def auth_headers(server_config: dict) -> dict | None:
         import google.auth
         import google.auth.transport.requests
 
-        scopes = (server_config.get("oauth") or {}).get("scopes")
+        scopes = extract_mcp_oauth_scopes(server_config)
         if not scopes:
             raise McpToolsError(
                 "google_credentials auth requires oauth.scopes on the MCP "

evalbench/test/claude_code_test.py

@@ -3,6 +3,8 @@
 import sys
 from unittest.mock import MagicMock, patch, ANY
 
+import pytest
+
 # Add parent directory to path so we can import generators
 sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
@@ -240,3 +242,78 @@ def test_extract_skills_strips_plugin_namespace():
 
     with patch.object(ClaudeCodeGenerator, '_get_installed_skills', return_value=set()):
         assert generator.extract_skills(stdout) == ["cloud-sql-postgres-admin"]
+
+
+def test_translate_mcp_config_forwards_oauth_scopes():
+    generator = object.__new__(ClaudeCodeGenerator)
+    generator.env = {}
+
+    mcp_config = {
+        "httpUrl": "https://test-dfareporting.sandbox.googleapis.com/mcp",
+        "authProviderType": "google_credentials",
+        "oauth": {
+            "scopes": [
+                "https://www.googleapis.com/auth/cloud-platform",
+                "https://www.googleapis.com/auth/dfareporting",
+            ]
+        },
+        "headers": {
+            "X-Goog-User-Project": "my-project"
+        }
+    }
+
+    with patch.object(generator, '_fetch_gcloud_access_token', return_value="fake_token") as mock_fetch:
+        translated = generator._translate_mcp_config("dfareporting", mcp_config)
+
+        assert "oauth" not in translated
+        assert "authProviderType" not in translated
+        assert translated["type"] == "http"
+        assert translated["url"] == "https://test-dfareporting.sandbox.googleapis.com/mcp"
+        assert translated["headers"]["Authorization"] == "Bearer fake_token"
+        assert translated["headers"]["X-Goog-User-Project"] == "my-project"
+        assert '--scopes="https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/dfareporting"' in translated["headersHelper"]
+
+        mock_fetch.assert_called_once_with(scopes=[
+            "https://www.googleapis.com/auth/cloud-platform",
+            "https://www.googleapis.com/auth/dfareporting",
+        ])
+
+
+def test_fetch_gcloud_access_token_passes_scopes():
+    generator = object.__new__(ClaudeCodeGenerator)
+    generator.env = {}
+
+    scopes = [
+        "https://www.googleapis.com/auth/cloud-platform",
+        "https://www.googleapis.com/auth/dfareporting",
+    ]
+
+    with patch('generators.models.claude_code.subprocess.run') as mock_run:
+        mock_proc = MagicMock()
+        mock_proc.stdout = "scoped_token_xyz\n"
+        mock_run.return_value = mock_proc
+
+        token = generator._fetch_gcloud_access_token(scopes=scopes)
+
+        assert token == "scoped_token_xyz"
+        first_call_cmd = mock_run.call_args_list[0][0][0]
+        assert first_call_cmd == [
+            "gcloud", "auth", "application-default", "print-access-token",
+            "--scopes=https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/dfareporting",
+        ]
+
+
+def test_headers_helper_rejects_unsafe_scopes():
+    with pytest.raises(AssertionError, match="Invalid or unsafe OAuth scope"):
+        ClaudeCodeGenerator._google_credentials_headers_helper(
+            scopes=['https://example.com"; rm -rf /; "']
+        )
+
+
+def test_fetch_gcloud_access_token_rejects_unsafe_scopes():
+    generator = object.__new__(ClaudeCodeGenerator)
+    generator.env = {}
+    with pytest.raises(AssertionError, match="Invalid or unsafe OAuth scope"):
+        generator._fetch_gcloud_access_token(
+            scopes=['$(whoami)']
+        )