Skip to content

[runner] Terminate the job's whole session, not just its shell - #4283

Merged
un-def merged 1 commit into
masterfrom
pr_runner_terminate_job_session
Sep 11, 2026
Merged

un-def merged 1 commit into
masterfrom
pr_runner_terminate_job_session

Conversation

@un-def

@un-def un-def commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Previously, a job that did not stop on the interrupt was left to WaitDelay, which SIGKILLs cmd.Process -- the wrapper shell -- and nothing else. Everything the job had started went on running, reparented to PID 1, and was cleaned up only when the container was destroyed.

The interrupt cannot cover that gap on its own. The line discipline delivers it to the terminal's foreground process group, and job control puts a cmd & job in a process group of its own, where neither the interrupt nor the SIGHUP the kernel sends to the foreground group on the session leader's exit will reach it.

Now the job's session is signalled directly. A session is the right scope because a child inherits its parent's session id, and only setsid() changes it.

When the job is asked to stop, termination is staged: SIGHUP once hupDelay has passed, SIGKILL once killDelay has. SIGHUP is what a terminal hangup delivers and what an idle shell exits on, which also clears the shell dash leaves behind -- interrupted, it returns to reading the terminal rather than exiting. SIGCONT goes with it, since a stopped process would not act on the SIGHUP until something resumes it.

The stages run from a goroutine so they proceed while cmd.Wait is still blocked, and execJob joins it afterwards, because the processes a job leaves behind can outlive the shell that started them. Each stage is skipped when the session is already empty, so a job that stops on the interrupt is not delayed. Zombies are not occupants: they have exited and only await reaping.

When the job instead reaches its own end, its leftovers -- a sidecar started with &, say -- are sent the same hangup, but nothing is waited for. They are about to lose the terminal and then the container without being told either way, so they are told; the job succeeded, though, so there is nothing left to enforce and no reason to delay reporting it.

Exceeding the log quota now cancels the command's context instead of killing the shell outright, so it escalates the same way rather than through a second, blunter path.

Previously, a job that did not stop on the interrupt was left to
`WaitDelay`, which SIGKILLs `cmd.Process` -- the wrapper shell -- and
nothing else. Everything the job had started went on running, reparented
to PID 1, and was cleaned up only when the container was destroyed.

The interrupt cannot cover that gap on its own. The line discipline
delivers it to the terminal's foreground process group, and job control
puts a `cmd &` job in a process group of its own, where neither the
interrupt nor the SIGHUP the kernel sends to the foreground group on the
session leader's exit will reach it.

Now the job's session is signalled directly. A session is the right
scope because a child inherits its parent's session id, and only
`setsid()` changes it.

When the job is asked to stop, termination is staged: SIGHUP once
`hupDelay` has passed, SIGKILL once `killDelay` has. SIGHUP is what a
terminal hangup delivers and what an idle shell exits on, which also
clears the shell dash leaves behind -- interrupted, it returns to
reading the terminal rather than exiting. SIGCONT goes with it, since a
stopped process would not act on the SIGHUP until something resumes it.

The stages run from a goroutine so they proceed while `cmd.Wait` is
still blocked, and `execJob` joins it afterwards, because the processes
a job leaves behind can outlive the shell that started them. Each stage
is skipped when the session is already empty, so a job that stops on the
interrupt is not delayed. Zombies are not occupants: they have exited
and only await reaping.

When the job instead reaches its own end, its leftovers -- a sidecar
started with `&`, say -- are sent the same hangup, but nothing is waited
for. They are about to lose the terminal and then the container without
being told either way, so they are told; the job succeeded, though, so
there is nothing left to enforce and no reason to delay reporting it.

Exceeding the log quota now cancels the command's context instead of
killing the shell outright, so it escalates the same way rather than
through a second, blunter path.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@un-def
un-def merged commit a15c156 into master Sep 11, 2026
26 checks passed
@un-def
un-def deleted the pr_runner_terminate_job_session branch September 11, 2026 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant