Don't record an overlay status when the job was cancelled - #4122
Don't record an overlay status when the job was cancelled#4122henrymercer wants to merge 2 commits into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Warning
- Copilot's review of this pull request may be incomplete because some of the changed files are excluded by your Copilot content exclusion settings. See Excluding content from Copilot for details.
Copilot review overview
🟢 Approval recommended
The focused implementation is consistent with existing input handling and covers the new behavior with unit tests.
Review tier: Balanced
Findings: None
What changed in this PR
Prevents cancelled jobs from incorrectly disabling overlay analysis through cached failure markers.
Changes:
- Passes the runtime job status into post-processing.
- Skips overlay failure recording for cancellations unless CodeQL already reported an error.
- Adds unit coverage for cancellation scenarios.
| File | Description |
|---|---|
src/init-action-post.ts |
Captures post-step job status before restoring inputs. |
src/init-action-post-helper.ts |
Adds cancellation-aware overlay status handling. |
src/init-action-post-helper.test.ts |
Tests cancellation and prior-error behavior. |
init/action.yml |
Defines the internal job-status input. |
lib/entry-points.js |
Generated output; excluded from review by policy. |
Files excluded by content exclusion policy (1)
- lib/entry-points.js
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
mbg
left a comment
There was a problem hiding this comment.
Broadly looks good and makes sense, thanks! Just a few considerations that came to my mind while reviewing this.
| [Internal] The ID of the check run, as provided by the Actions runtime environment. Do not set this value manually. | ||
| default: ${{ job.check_run_id }} | ||
| required: false | ||
| job-status: |
There was a problem hiding this comment.
Never nice to have to feed this in through an extra action input, but I don't see a better approach for getting hold of the job status. An alternative option might be to set CODEQL_ACTION_STEP_(init|analyze|...) environment variables / state that we set to e.g. starting when the respective action starts and then to success or failure depending on the outcome. That should then allow us to identify which step started, succeeded, or failed (gracefully or not). For the overlay status, we could then check that all available environment variables with a CODEQL_ACTION_STEP_ prefix are success and none are starting or failure. The downside is that it wouldn't catch if the failure isn't related to what happens in CodeQL Action steps, or we fail to even set the starting value.
| // a failure would disable overlay analysis needlessly. Note that we still record a failure if one | ||
| // of our own Actions reported an error before the run was cancelled. | ||
| if ( | ||
| jobStatus?.trim().toLowerCase() === "cancelled" && |
There was a problem hiding this comment.
Rather than checking for the case where we don't want to record the overlay status, would it make sense to make this a check for the cases where we do want to record the overlay status?
That would be more defensive and guard against the hypothetical case of a new status value that we don't recognise.
I would also expect that we'd have a jobStatus value here unless something broke badly?
| * it can gather telemetry, does not get to report anything. | ||
| */ | ||
| function didCodeQlReportError(): boolean { | ||
| const jobStatus = process.env[EnvVar.JOB_STATUS]; |
There was a problem hiding this comment.
Add a ReadOnlyEnv parameter and use getOptional.
| return ( | ||
| jobStatus === JobStatus.FailureStatus || | ||
| jobStatus === JobStatus.ConfigErrorStatus | ||
| ); |
There was a problem hiding this comment.
Could we end up with one of these if the workflow job was cancelled? E.g. because it caused a thread abort style exception to be thrown at an inconvenient moment?
The overlay status mechanism records a marker in the Actions cache when building an overlay-base database fails, so that later runs skip overlay analysis instead of failing the same way, but it records one whenever the analyze step did not report success, which also covers runs that were cancelled, since the
initpost step still runs in that case. A cancellation tells us nothing about whether the analysis would have succeeded, and a marker disables overlay analysis for the whole repository until the next CodeQL release, so this adds an internaljob-statusinput defaulting to${{ job.status }}and skips recording when the job was cancelled.Risk assessment
For internal use only. Please select the risk level of this change:
Which use cases does this change impact?
Workflow types:
dynamicworkflows (Default Setup, Code Quality, ...).Products:
analysis-kinds: code-scanning.Environments:
github.comand/or GitHub Enterprise Cloud with Data Residency.How did/will you validate this change?
cancelledto the post step..test.tsfiles).If something goes wrong after this change is released, what are the mitigation and rollback strategies?
overlay_analysis_status_saveis enabled, so disabling that flag restores the previous behaviour. The new input is inert on its own.How will you know if something goes wrong after this change is released?
codeql-action/overlay-disableddiagnostic reportsskipped-due-to-cached-status, so the rate of that reason shows whether we are recording fewer markers as intended, and overlay adoption shows whether we have stopped recording ones we needed.Are there any special considerations for merging or releasing this change?
Merge / deployment checklist