All issues

Search results

[Bug] AutoEP FP32 gradient clipping averages norms across ranks with different expert ownership

bugSomething isn't workingSomething isn't working

training

Status: Open.

#8475 In deepspeedai/DeepSpeed;

· gss10282023 opened on Sep 10, 2026

[RFC] Enable Correct and High-Performance Muon Optimizer under ZeRO CPU Offload (Stage 1/2/3)

Status: Open.

#8463 In deepspeedai/DeepSpeed;

· jinyouzhi opened on Sep 8, 2026

[FEATURE] Align decode fronts and trim dead zones in continuous batching

enhancementNew feature or requestNew feature or request

Status: Open.

#8462 In deepspeedai/DeepSpeed;

· nathon-lee opened on Sep 8, 2026

Muon cannot run under BF16_Optimizer: flat partitions with no Newton-Schulz applied anywhere

Status: Open.

#8461 In deepspeedai/DeepSpeed;

· alanhuangyoo opened on Sep 8, 2026

[FEATURE] Add profiling support for continuous batching

enhancementNew feature or requestNew feature or request

Status: Open.

#8459 In deepspeedai/DeepSpeed;

· nathon-lee opened on Sep 8, 2026

[BUG] Do not silently ignore use_shared_prefill with continuous batching

bugSomething isn't workingSomething isn't working

training

Status: Open.

#8458 In deepspeedai/DeepSpeed;

· nathon-lee opened on Sep 8, 2026

AutoTP loads module weights by hardcoded class name, so 219 of 233 transformers norm classes are silently left on meta

Status: Open.

#8447 In deepspeedai/DeepSpeed;

· alanhuangyoo opened on Sep 6, 2026

ZeRO-3 applies Muon's Newton-Schulz once per micro-batch, so gradient accumulation changes the optimizer

Status: Open.

#8443 In deepspeedai/DeepSpeed;

· alanhuangyoo opened on Sep 6, 2026

Default gradient_clipping divides every Muon update by its own norm, shrinking the step by a model-sized factor

Status: Open.

#8439 In deepspeedai/DeepSpeed;

· alanhuangyoo opened on Sep 6, 2026

Muon under AutoTP orthogonalizes each shard, so the update depends on the TP degree

Status: Open.

#8437 In deepspeedai/DeepSpeed;

· alanhuangyoo opened on Sep 6, 2026

[BUG] Muon + fp16 does not train under ZeRO 1/2: the first loss-scale overflow permanently poisons the momentum buffer

Status: Open.

#8432 In deepspeedai/DeepSpeed;

· alanhuangyoo opened on Sep 6, 2026

[Per-Head Muon] Head geometry that lives outside num_attention_heads/head_dim: Kimi-K3 KDA layers and GLM-5.2's DSA indexer

Status: Open.

#8420 In deepspeedai/DeepSpeed;

· alanhuangyoo opened on Sep 5, 2026