Skip to content

ci: up26.03 upgrade jobs fail in the 26.03 baseline install until 26.03 leaves the matrix #2029

Description

@pau-hedgehog

https://github.com/githedgehog/fabricator/actions/runs/33433263541/job/99625370329

v-up26.03-mesh-gw-iso-l2vni, step "(26.03) Run and test VLAB". control-1's Flatcar install fails at the mount step, the node never reboots, and the job exits code 2 after the post-process SSH timeout.

Aug 31 20:04:32 control-1 hhfab-flatcar-install[1509]: Aug 31 20:04:32.004 DBG Running command cmd="partprobe /dev/vda"
Aug 31 20:04:32 control-1 hhfab-flatcar-install[1509]: Aug 31 20:04:32.056 DBG Running command cmd="mount -t auto /dev/vda9 /mnt/rootdir"
Aug 31 20:04:32 control-1 hhfab-flatcar-install[1509]: Aug 31 20:04:32.068 DBG mount: mount: /mnt/rootdir: special device /dev/vda9 does not exist.
Aug 31 20:04:32 control-1 hhfab-flatcar-install[1509]: Aug 31 20:04:32.072 ERR mounting root: running "mount": exit status 32

Same udev race as #1822, at the same call site its 2026-08-11 comment described. Fixed on master by d45dd2d, merged 2026-08-27, tagged in v0.50.0 only. v0.49.3 was tagged 2026-08-14 and still carries the unguarded partprobe then mount sequence at pkg/fab/recipe/flatcar/os_install.go:230-238.

run-vlab.yaml:199 installs the released hhfab for the upgrade baseline and :237 runs old/hhfab vlab up, so step "(26.03) Run and test VLAB" runs v0.49.3 regardless of what master carries. That step keeps hitting this race until 26.03 is dropped from the CI upgrade matrix or the 26.03 channel resolves to a build containing the fix.

Filed for failure tracking.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions