Per-device schedule rows are not folding into URL rows (variants/job stuck at 1.06)

[harper-joseph]

Summary

Since v0.66.0 the schedule is one row per URL, and a claim should hand the renderer one job carrying every device in deviceTypes.default. Old per-device rows convert on their first render, so a corpus should fold within roughly one render cycle.

On a large production corpus it is not folding. Measured over 622,694 renders across 60 workers:

completed (variants) / jobs = 1.06        <- should approach the device count (2)
variantContextsShared       = 5.3% of renders

94% of jobs still render a single device, two cycles after the deploy.

It is not a measurement artifact

Both counters were read from source: stats.jobs++ fires once per posted result (Worker.ts, result-post path), stats.completed++ once per variant render attempt. variantsSkipped was 0 for the whole window. The two independent signals agree — variantContextsShared at 5.3% implies the same ~6% multi-variant share as completed/jobs = 1.06.

It is not slow progress

Hourly over a 15-hour window the ratio does not trend:

hour	variants/job
0-3	1.06 - 1.08
4-8	1.13 -> 1.35 -> 1.27 -> 1.33
9-14	1.11 -> 1.04 -> 1.05

It starts at 1.06 and ends at 1.05. The bulge in hours 4-8 lines up with that corpus's overnight demand peak, i.e. the multi-variant share rises when scheduled renders dominate and falls back afterwards. Conversion would be monotonic; this is not.

Already ruled out

Every schedule writer files a URL-keyed row for devices in deviceTypes.default:

• util/changeProbe.js — writeSchedule(row.url, ...)
• resources/Target.js — writeSchedule(url, ...)
• http_handlers/bot_request.js — scheduleKey = config.deviceTypes.default.includes(deviceType) ? cacheUrl : cacheKey, so device-keyed only for a device outside the default set
• util/invalidationReenqueue.js — reads rows with getScheduleRow first and only reschedules ones that already exist

So nothing is manufacturing new device-keyed rows, and fold is computed correctly at RenderQueue.js:161 (!perDevice || config.deviceTypes.default.includes(device)) — both default devices qualify.

/prerender_admin/schedule cannot be used to check row shape directly: it normalises the key, so URL-keyed and device-keyed lookups return the same row.

Why it matters

A folded job renders each device back-to-back sharing the browser context and replaying the document — the second variant needs roughly a fifth of the same-origin fetches. Unfolded, that saving is never realised, and on a CPU-bound fleet it translates directly into pod count: the difference between a second variant costing 50% and 100% of the first is ~12 pods vs ~16 for the same corpus.

Suggested next steps

1. Determine whether URL-keyed rows are actually being created (a direct table read, or a local Harper reproduction — /prerender_admin/schedule will not answer it).
2. If they are, find what still claims the device rows.
3. Consider surfacing row shape in the admin API, since there is currently no way to observe the fold's progress from outside.

🤖 Generated with Claude Code