Priority P1; F9. Current raw-scan save materializes all images and Q arrays, while image/metadata/CA caches are independently accumulated and snapshotted.
Introduce one atomic frame-record collection and a bounded writer queue; append frames/batches to chunked extendible datasets. Account for scan lengths beyond RAM cache capacity, missing metadata, rejected frames and writer failures. Preserve scientific completeness explicitly; retained cache tails are not complete scans.
All actual h5py writes must live in utils/hdf5_writer.py. Current volume_io.py also writes HDF5, contrary to AGENTS.md: move that implementation into the Qt-free writer while retaining compatible delegating APIs and the no-Qt import contract. HDF5Handler remains a delegator. Coordinate schema consolidation with existing #140; do not close it for incremental I/O alone.
Acceptance: scan larger than the cache retains every accepted scientific frame on disk with matched IDs/metadata; aligned missing-value representation; byte-bounded queue/RSS; deterministic stop/flush/error behavior and concurrent-save coverage; reopened raw/volume files preserve orientation, dtype, calibration and coverage. Depends on #150/#151 and coordinates #109/#140.
Work already implemented
The merged live-grid session already has headless volume saving and memory/concurrency guards. This task targets legacy raw scan recording and consolidation of the HDF5 write owner.
The local performance implementation passed 14 focused tests and Ruff during the 8 September review; this is not proof of complete issue acceptance. No code from those uncommitted changes has been published as part of this planning batch.
Completion policy
Use Refs for partial PRs. Close only when all acceptance criteria have linked merged code and verification; keep hardware-dependent criteria open until exact-SHA results exist. Retain the closed issue history.
Priority P1; F9. Current raw-scan save materializes all images and Q arrays, while image/metadata/CA caches are independently accumulated and snapshotted.
Introduce one atomic frame-record collection and a bounded writer queue; append frames/batches to chunked extendible datasets. Account for scan lengths beyond RAM cache capacity, missing metadata, rejected frames and writer failures. Preserve scientific completeness explicitly; retained cache tails are not complete scans.
All actual h5py writes must live in utils/hdf5_writer.py. Current volume_io.py also writes HDF5, contrary to AGENTS.md: move that implementation into the Qt-free writer while retaining compatible delegating APIs and the no-Qt import contract. HDF5Handler remains a delegator. Coordinate schema consolidation with existing #140; do not close it for incremental I/O alone.
Acceptance: scan larger than the cache retains every accepted scientific frame on disk with matched IDs/metadata; aligned missing-value representation; byte-bounded queue/RSS; deterministic stop/flush/error behavior and concurrent-save coverage; reopened raw/volume files preserve orientation, dtype, calibration and coverage. Depends on #150/#151 and coordinates #109/#140.
Work already implemented
The merged live-grid session already has headless volume saving and memory/concurrency guards. This task targets legacy raw scan recording and consolidation of the HDF5 write owner.
The local performance implementation passed 14 focused tests and Ruff during the 8 September review; this is not proof of complete issue acceptance. No code from those uncommitted changes has been published as part of this planning batch.
Completion policy
Use Refs for partial PRs. Close only when all acceptance criteria have linked merged code and verification; keep hardware-dependent criteria open until exact-SHA results exist. Retain the closed issue history.