Skip to content

renderer: reduce image streaming overhead - #1467

Merged
simonklee merged 23 commits into
mainfrom
ot-magick
Aug 30, 2026
Merged

renderer: reduce image streaming overhead#1467
simonklee merged 23 commits into
mainfrom
ot-magick

Conversation

@simonklee

@simonklee simonklee commented Aug 29, 2026

Copy link
Copy Markdown
Member

Reduce frame preparation and transport costs so local terminals can
consume image updates with less CPU work. Bound pending frames when
output stalls and fix per-frame GPU resource growth so sustained
rendering does not accumulate work or retain native handles.

Keep raw transport as the default and make file transport opt-in for
terminals that acknowledge image uploads.

#1328 partially superseded, but it has NativeImage.adoptRgbaFile(path, width, height) that adopts an existing producer-created RGBA file lazily. Eligible renders send that same path without reading, copying, or rewriting its pixels.

Keep raw and retained PNG behavior by default. Add bounded level-1 zlib preparation with raw fallback, and opt-in local file leases gated by medium and explicit-ID ACK probes. Route replies through renderer input, retain files until ACK, and cancel bounded leases on failures, timeout, suspend, and destroy.
Share pixel import options and native SIMD conversion between fresh imports and exclusive pool updates. Keep fresh publication handles, pinned snapshots, and publishRgba intact.

Cover strided BGRA and alpha modes, allocation reuse, invalid input, PNG invalidation, and failed native wrappers. Extend the CPU benchmark to separate native conversion from storage reuse.
Read the renderer's stored terminal environment instead of libc getenv so Bun and Node honor runtime TMPDIR updates before construction. Skip file expiry, clock reads, and directory lookup for raw/zlib uploads. Add a probe-directory cleanup regression and exercise compression allocation failures with compressible input.
Use the existing feed-idle retry to coalesce ordinary frames before animation callbacks, composition, and native encoding. Keep control and shutdown output independent, and preserve new one-shot requests after pause or stop.

Count queued and consumer-pinned span references for native feed high-water admission. Draining spans alone must not return credit.

The deterministic delayed-Writable regression reduces 33 callbacks and 8,986,380 queued bytes to one callback and one 272,332-byte frame over 32 blocked ticks. It also checks partial completion, latest-state resume, cancellation, diff consistency, and shutdown ordering.

Verified native -j2 build and tests (2112 pass), full core Bun suite (5492 pass), Node suite (4749 pass), root build, packed distribution smoke tests, formatting, and lint.
Balance requestAnimationFrame live ownership only when cancellation removes a pending callback. Let cancelled admission waits settle renderer idle without releasing transport-owned chunks, and reject stale wakeups with a scheduling generation.

Mark split-footer transition batches as control output through the existing packed flags. These batches bypass feed high water while retaining atomic publication, hard allocation limits, ordering, and the ordinary frame admission gate. Keep the existing native batched API as a normal-output wrapper.

Add delayed-Writable regressions for natural RAF idle, cancellation before write completion, stale continuations, and captured stdout at 4096 pinned spans through destroy, suspend, and passthrough transitions, including terminal setup. Add native coverage for ordinary rejection, control admission, and atomic failure at max_bytes.

Verified full Bun suite (5497 pass), Node suite (4754 pass), native -j2 tests (2113 pass), native and root builds, packed distribution smoke tests, formatting, and lint.
Fix the regression where dropping the last automatic live owner cancelled coalesced tree updates or captured stdout. Preserve pending demand, including a retry whose credit has already returned, without keeping continuous mode active.

Close residual scheduling gaps: do not register feed-idle retries while suspended, update saved auto demand on actual live ownership changes during suspension, and cancel queued one-shot work on stop even when continuous mode is inactive. Preserve explicit start and ownerless auto() intent, as well as new requests made after stop.

Add focused delayed-Writable regressions before fixing each path. Output admission, encoded byte ordering, and chunk leases remain unchanged; no native API changes.

Verified 25 admission tests, full Bun suite (5508 pass), full Node suite (4765 pass), native -j2 suite (2113 pass), library build, packed distribution smoke tests, formatting, and lint.
Capture a generation for each accepted one-shot request and invalidate it when start, suspend, or stop cancels queued work. Reject stale activations before rendering and prevent stale finally blocks from clearing a newer request.

Reproduce both ABA failures without asynchronous frame callbacks: same-turn request/stop/request rendered twice, and a request queued after a completed frame was cleared by its older finalizer. Exercise the palette repaint test through the scheduler instead of invoking the now-tokenized private activation method.

Verified 27 admission tests, 161 relevant renderer tests, 46 palette tests, the full Bun suite (5510 pass), the full Node suite (4767 pass), library build, packed distribution smoke tests, formatting, and lint. No native or unrelated lifecycle changes.
Make the arena available from the example selector while preserving
standalone use. Reuse the GPU across visits and wait for pending
readbacks during teardown so returning to the menu does not accumulate
provider resources or leave the renderer running continuously.
Let applications switch Kitty transports without restarting the
renderer or replacing image sources
@simonklee
simonklee marked this pull request as ready for review August 30, 2026 20:08
@simonklee
simonklee merged commit c1a52d2 into main Aug 30, 2026
29 checks passed
@simonklee
simonklee deleted the ot-magick branch August 30, 2026 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant