CUDA: model manager reports "available 0.00 MB device" on discrete GPUs — klein / z-image (LLM conditioner) fail at weight prep
Build: master cc515a0, Linux x64, CUDA 13.0 toolkit, GGML_NATIVE=OFF, archs 75-real;80-real;86-real;87-real;89-real;90-real;100-real;120-real, FA on.
GPUs (identical failure on both):
- RTX 4070 Ti SUPER 16 GB with 12.5 GB free (
nvidia-smi verified)
- RTX 2060 6 GB (sm_75) with 5.7 GB free
Recipe (from docs/flux2.md, official klein recipe + offload for 6 GB):
sd-cli --diffusion-model flux-2-klein-4b-Q8_0.gguf \
--vae klein-vae.safetensors \
--llm Qwen3-4B-Instruct-2507-Q4_K_M.gguf \
--backend llm=cpu \
-p "un pato en un estanque, fotorrealista" --cfg-scale 1.0 \
-W 512 -H 512 --steps 2 --seed 7 -v --offload-to-cpu --diffusion-fa \
-o out.png
Log: placement succeeds (total params 6637MB: diffusion 4101MB VRAM, text_encoders 2375MB RAM, vae 160MB VRAM), then:
[WARN] model_manager.cpp:1762 - model manager cannot make enough memory available on CUDA0: need 616.76 MB device / 104.76 MB budget, available 0.00 MB device / unlimited budget
[ERROR] ggml_runner.cpp:876 - flux segment 1/27 (flux.prelude) failed during weight preparation
conditioner.hpp:2220: GGML_ASSERT(!hidden_states.empty()) failed
Same signature for z_image_turbo-Q4_K.gguf + same --llm (z_image segment 1/36 (z_image.prelude)).
Tried, none rescue it:
Ruled out:
- Real exhaustion: 12.5 GB free on the 4070 Ti, needs 616 MB, reports exactly
0.00 on both cards → accounting, not capacity.
- UMA override:
prop.integrated=0 (probed via cudaGetDeviceProperties on both GPUs), GGML_CUDA_ENABLE_UNIFIED_MEMORY unset on both machines.
- Weights/hardware: the same files generate fine on the same RTX 2060 under KoboldCpp 1.121's bundled sd.cpp (~1.4 s/step, clean images) → master-only regression, not weights.
Suspected area: ModelManager::check_capacity / ggml_backend_dev_memory CUDA path returning free=0 (or free > total → 0) on these setups; byte-identical refusal numbers across runs. Possibly related to #1976 (same WARN shape on Vulkan video paths), but here it hits image + LLM-conditioner paths (klein flux2, z-image) on CUDA.
Happy to run any experiment (other flag combos, debug builds, bisect window if pointed at one).
CUDA: model manager reports "available 0.00 MB device" on discrete GPUs — klein / z-image (LLM conditioner) fail at weight prep
Build: master
cc515a0, Linux x64, CUDA 13.0 toolkit,GGML_NATIVE=OFF, archs75-real;80-real;86-real;87-real;89-real;90-real;100-real;120-real, FA on.GPUs (identical failure on both):
nvidia-smiverified)Recipe (from
docs/flux2.md, official klein recipe + offload for 6 GB):Log: placement succeeds (
total params 6637MB: diffusion 4101MB VRAM, text_encoders 2375MB RAM, vae 160MB VRAM), then:Same signature for
z_image_turbo-Q4_K.gguf+ same--llm(z_image segment 1/36 (z_image.prelude)).Tried, none rescue it:
--auto-fit off(unlike the Wan case in Video regressions on 16 GB Vulkan: LTX-2.5 + MiniMax-H3 broken since master-864, Wan 2.2 broken by master-866 (graph segmentation / weight-budget behavior) — --auto-fit off only rescues Wan #1976 — here it does NOT help)--max-vram, with / without--offload-to-cpu--backend llm=cpu(then it dies earlier atqwen3 segment 1/29 (llm.text.prelude)with the same 0.00 available)Ruled out:
0.00on both cards → accounting, not capacity.prop.integrated=0(probed viacudaGetDevicePropertieson both GPUs),GGML_CUDA_ENABLE_UNIFIED_MEMORYunset on both machines.Suspected area:
ModelManager::check_capacity/ggml_backend_dev_memoryCUDA path returningfree=0(orfree > total → 0) on these setups; byte-identical refusal numbers across runs. Possibly related to #1976 (same WARN shape on Vulkan video paths), but here it hits image + LLM-conditioner paths (klein flux2, z-image) on CUDA.Happy to run any experiment (other flag combos, debug builds, bisect window if pointed at one).