Skip to content

ci: say what the Worker size gate's reading is repeatable to, not only what it is accurate to - #279

Merged
os-bill merged 3 commits into
mainfrom
claude/issue-277-size-gate-repeatability
Sep 8, 2026
Merged

ci: say what the Worker size gate's reading is repeatable to, not only what it is accurate to#279
os-bill merged 3 commits into
mainfrom
claude/issue-277-size-gate-repeatability

Conversation

@os-bill

@os-bill os-bill commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Fixes #277

ci.yml's size-gate comment sets the reading up as a precise instrument: it quotes a 1.96 KiB agreement with the figure Cloudflare accepted, calls that 0.0033 %, and concludes "the reading tracks the enforced figure". That is the right claim for the gate's job — accuracy against the limit — and it is silent on repeatability, which is what a diff-attribution question needs. This adds the missing half, with numbers that were measured rather than assumed.

Comment only. The budget, the limit and how the measurement is taken are unchanged — the gate works and this does not touch it. The diff is 61 added lines in one comment block, no deletions.

The experiment: one commit, run twice, zero tree difference

66cb0a4 is an empty commit. Its tree hash is 4190e2242e65b7ce470f15bca943886e8159d258 — byte-identical to main's tree at 50aacb9. Run 34252385192 was then re-run, so attempts 1 and 2 built the same SHA on two different ubuntu-latest runners.

That is what the card's own control could not quite reach. The card's strongest pair (0e26657f vs 87880ba) still differed by one file, .github/workflows/ci.yml, and rested on the argument that a workflow file cannot enter the Worker bundle. That argument is correct, but between attempts 1 and 2 there is no file to argue about at all.

# tree where Total Upload: gzip
1 66cb0a4 attempt 1, job 102149701548, runner 1000697309 58549.04 KiB 8800.26 KiB
2 66cb0a4 attempt 2, job 102152795435, runner 1000697521 58553.46 KiB 8800.65 KiB
3 1bf8b70 run 34277657373, job 102234545211, runner 1000701662 58549.06 KiB 8797.64 KiB

Readings 1 and 2 are tree-identical. Reading 3 is this PR's own second commit — a different tree, since it carries the comment, but the same bundle inputs, because a workflow file cannot enter the Worker. It is the weaker of the two kinds of control and is labelled that way in the comment.

Largest gap: 4.42 KiB, between two runs of the same commit. All three runs were green and all three printed within budget.

What follows, and what does not:

  • 4.42 KiB is larger than the 1.96 KiB calibration agreement the comment quotes. So that agreement cannot be read as evidence of sub-KiB precision either — it is a single comparison taken inside this much noise. It remains fine as what it claims to be.
  • The shape is a tight cluster and one outlier, not a reading that wanders across a 4.42 KiB band: readings 1 and 3 agree to 0.02 KiB, reading 2 sits ~4.4 KiB above both. Three points cannot carry a distribution, so none is modelled.
  • The gzip column moves too, which shows the artifact's bytes genuinely differ rather than the same number being displayed differently. It does not track the raw figure — reading 3 has the smallest compressed size and a middling raw one — so it confirms that much and nothing more.
  • Holding the machine fixed shrinks the spread without closing it. Four cold rebuilds of 66cb0a4's tree in one container gave 58548.93 / 58548.93 / 58548.94 / 58549.55 KiB — 0.62 KiB apart. Each was a full cold build (.next, .open-next and the turbo cache removed, turbo run build --force), otherwise turbo replays one cached .next and the "repeat" never builds twice.

What this can and cannot conclude

Can: the reading is not reproducible over identical bundle inputs, in the environment class the gate actually runs in — and for readings 1 and 2, over a byte-identical tree. The card's premise holds. A per-PR delta of single-digit KiB is inside the measured spread and has not been shown to be a change in the bundle; a real growth of a few KiB is equally invisible.

Cannot: anything about why. Attempts on GitHub-hosted runners are different runner instances — same environment class, not the same machine — so this does not separate build non-determinism from environment-dependent content and does not test the runner-temp-path hypothesis the card lists. n is small (3 CI runs, 4 local builds), so 4.42 KiB is the largest same-tree gap measured, never a proven bound. No cause is claimed.

The population is frozen on purpose

Every push produces another reading, so "fold in the latest" has no end. The three readings above are a snapshot at named commits, and the comment says so in its own words: it is not maintained, nobody is obliged to update it, and a later reading that differs is the point being made rather than a defect in the block. The CI run for the final commit is deliberately not folded in.

On the empty commit

It is a measurement instrument, not a CI kick. This repo's standing rule against empty commits to re-trigger CI is about commits that exist only to make a pipeline run again; this one exists because the deliverable is a reading over a tree with no diff. The squash merge folds it away — nothing empty lands on main, and the merged commit is the comment change alone.

Verification

  • Gate Worker bundle fits the size budgetgreen on all three runs (### Worker bundle size — within budget; | measured | 58549.04 | 89.34 % |, | measured | 58553.46 | 89.35 % |, | measured | 58549.06 | 89.34 % |).
  • Gates Node floor, Generated zh-Hant is current, Locale surface, turbo run type-check, turbo run test, Half-state sweeper self-test, The docs site renders — green on all three. Deploy docs and Upload the Worker bundle skipped by their own push + refs/heads/main guard, so no pull-request run here can publish.
  • .github/workflows/ci.yml parses as YAML after the edit; the gate step's env (61440 / 65536) and its run body are byte-unchanged; control-byte scan clean; diff is additions only.

Reported from session session_01ChPQM8jamxLUfUAxwFpJ8S.

🤖 Generated with Claude Code

Empty on purpose. The card asks for the gate to be run twice on ONE
unchanged commit; this commit exists so a pull-request run and a re-run
of that same run produce two readings over one identical SHA, with no
tree diff to argue about. The measured figures land in the next commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ChPQM8jamxLUfUAxwFpJ8S
…ate to

The size-gate comment sets the reading up as a precise instrument — it
quotes a 1.96 KiB / 0.0033% agreement with the figure Cloudflare
accepted and concludes "the reading tracks the enforced figure". That is
the right claim for the gate's job, accuracy against the limit, and it
is silent on repeatability, which is what a diff-attribution question
needs.

Measured, not assumed. One commit run twice with no tree difference at
all (66cb0a4, an empty commit whose tree hash is byte-identical to
main's at 50aacb9), run 34252385192 attempts 1 and 2, two different
ubuntu-latest runners: 58549.04 KiB and 58553.46 KiB, 4.42 KiB apart.
Four cold rebuilds of that tree in one container spanned 0.62 KiB.

Comment only. The budget, the limit and how the measurement is taken are
unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ChPQM8jamxLUfUAxwFpJ8S
The push of the previous commit produced a reading of its own (1bf8b70,
run 34277657373, job 102234545211): 58549.06 KiB. That tree is not
byte-identical to 66cb0a4's — it carries the comment — but the bundle
inputs are, since a workflow file cannot enter the Worker, so it is
recorded as the weaker of the two kinds of control and labelled that way.

It changes the claim, not just the count. Two of the three readings agree
to 0.02 KiB while the third sits ~4.4 KiB above both, so the shape is a
tight cluster with one outlier rather than a reading that wanders across
a 4.42 KiB band. The gzip column moves too (8800.26 / 8800.65 / 8797.64),
which shows the artifact's bytes genuinely differ and nothing more — it
does not track the raw figure. n corrected to 3 CI runs and 4 local
builds.

Every run produces another reading, so the block now says in its own
words that it is a snapshot at named commits and is not maintained.
Freezing the population is what stops "fold in the latest" from running
forever.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ChPQM8jamxLUfUAxwFpJ8S
@os-bill
os-bill marked this pull request as ready for review September 8, 2026 21:06
@os-bill
os-bill merged commit d725081 into main Sep 8, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The Worker size gate's reading moves by a few KiB between builds of bundle-identical trees, so small per-PR deltas cannot be attributed

2 participants