Move runtime test build into product jobs

[jkoritzinsky]

Instead of having a separate runtime test build job and test run job for the tests under src/tests, move the test build and run into the product build jobs.

We already do this for NativeAOT and Mono legs, and we're seeing pretty insane queue times for build machines for some platforms (macOS in particular). This should be faster as we cut the queue time for new build machines out and do all the work on one machine, even though we are now doing additional work (building src/tests more times).

I only updated the runtime.yml pipeline to start as that's the Pri0 test build, not Pri1. Eventually we want to do this for all of the test builds to allow us to delete a bunch of infrastructure, but we aren't there yet.

[dotnet-policy-service]

Tagging subscribers to this area: @akoeplinger, @matouskozak, @simonrozsival
See info in area-owners.md if you want to be subscribed.

[Copilot]

Copilot review overview

🟡 Changes recommended

The new CoreCLR call sites for build-runtime-tests-and-send-to-helix.yml don’t override its runtimeFlavor/runtimeVariant defaults (mono/monointerpreter), which can misconfigure the Helix publish/test execution path.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Lite
Findings: 1

New issues introduced by this change (1)

Severity	Finding
	eng/​pipelines/​runtime.yml — build-runtime-tests-and-send-to-helix.yml defaults to runtimeFlavor: 'mono' and…

What changed in this PR

This PR restructures the runtime.yml Azure Pipelines definition to eliminate the standalone CoreCLR src/tests build/run jobs (for the Pri0 runtime pipeline) and instead build + submit runtime tests to Helix from within the existing product build jobs, aiming to reduce queue time and improve end-to-end latency on constrained platforms (e.g., macOS).

Changes:

• Move CoreCLR runtime test build + Helix submission into multiple existing product build jobs (and remove the separate CoreCLR runtime test build/run job blocks).
• Add/adjust Helix queue setup wiring in affected job matrices and increase several job timeouts to accommodate the additional test build work.
• Fix/standardize conditional behavior in Helix submission templates so boolean condition: false is respected and conditions can be threaded through the build + send templates.

File	Description
eng/​pipelines/​runtime.yml	Embeds runtime test build + Helix submission into product build jobs; removes standalone runtime test build/run job blocks; updates queue setup and timeouts.
eng/​pipelines/​common/​templates/​runtimes/​send-to-helix-step.yml	Adjusts how the step-level condition is applied when delegating to the inner send step.
eng/​pipelines/​common/​templates/​runtimes/​send-to-helix-inner-step.yml	Makes the condition parameter a proper boolean default and applies it cleanly alongside succeeded() and non-empty send parameters.
eng/​pipelines/​common/​templates/​runtimes/​build-runtime-tests.yml	Threads a condition parameter to the “Build Tests” steps so callers can gate test builds.
eng/​pipelines/​common/​templates/​runtimes/​build-runtime-tests-and-send-to-helix.yml	Passes condition through to build steps and ensures additional AOT steps and the send step honor the caller’s condition.

Suppressed comments (4)

eng/pipelines/runtime.yml:475

• This CoreCLR runtime-tests Helix submission doesn't override build-runtime-tests-and-send-to-helix.yml defaults (runtimeFlavor: mono, runtimeVariant: monointerpreter). That means the send step will set RuntimeFlavor/_RuntimeVariant to Mono interpreter values unless you pass the intended CoreCLR settings here.

                parameters:
                  creator: dotnet-bot
                  testRunNamePrefixSuffix: CoreCLR_$(_BuildConfig)
                  useHelixMonitor: ${{ variables.enableHelixJobMonitor }}

eng/pipelines/runtime.yml:514

• Same as other new CoreCLR legs: without overriding runtimeFlavor/runtimeVariant, the Helix publish template will use its Mono interpreter defaults. Please set CoreCLR explicitly to avoid sending the wrong payload metadata / running the wrong execution path.

                parameters:
                  creator: dotnet-bot
                  testRunNamePrefixSuffix: CoreCLR_$(_BuildConfig)
                  useHelixMonitor: ${{ variables.enableHelixJobMonitor }}

eng/pipelines/runtime.yml:587

• This CoreCLR test submission step relies on build-runtime-tests-and-send-to-helix.yml, whose defaults are Mono/monointerpreter. Set runtimeFlavor to CoreCLR (and clear runtimeVariant) so helixpublishwitharcade.proj doesn't treat this as Mono.

                parameters:
                  creator: dotnet-bot
                  testRunNamePrefixSuffix: CoreCLR_$(_BuildConfig)
                  useHelixMonitor: ${{ variables.enableHelixJobMonitor }}

eng/pipelines/runtime.yml:628

• This CoreCLR runtime-tests submission doesn't override build-runtime-tests-and-send-to-helix.yml defaults (runtimeFlavor: mono, runtimeVariant: monointerpreter). Pass runtimeFlavor: coreclr and clear runtimeVariant to ensure Helix publish uses the CoreCLR execution path.

                parameters:
                  creator: dotnet-bot
                  testRunNamePrefixSuffix: CoreCLR_$(_BuildConfig)
                  useHelixMonitor: ${{ variables.enableHelixJobMonitor }}

[Copilot]

Copilot review overview

🔵 Needs a closer look

It significantly changes the runtime CI job topology and Helix submission flow across multiple legs, so it warrants careful human validation of pipeline behavior and coverage.

Review tier: Lite
Findings: None

Issues resolved since last review (1)

Severity	Finding
	eng/​pipelines/​runtime.yml — build-runtime-tests-and-send-to-helix.yml defaults to runtimeFlavor: 'mono' and… View resolved comment

[agocke]

Is this going to a problem if the build fails due to a compiler error?