What problem are you facing?
spec.huggingFace.sizeGiB is required, and the docs say "Size it to the model, since a value below the model's size leaves no room to stage the weights." That instruction is wrong, and there is no instruction that would be right.
The hydration Job stages the repository, not the model. hf download {repo} fetches every file in the repo, and a HuggingFace repository routinely publishes the same weights more than once: a consolidated checkpoint beside the sharded safetensors (original/consolidated.00.pth in Llama, consolidated.safetensors in Mistral), one copy per framework (bert ships safetensors, PyTorch, Flax, TensorFlow, ONNX and CoreML), or a second runtime's format (openai/gpt-oss-20b ships a 13.8 GB metal/model.bin).
| Repository |
Published |
Weights an engine reads |
|
openai/whisper-large-v3 |
24.7 GB |
3.1 GB |
8.0x |
google-bert/bert-base-uncased |
3.5 GB |
0.4 GB |
7.8x |
meta-llama/Llama-3.1-405B-Instruct |
2443.8 GB |
811.7 GB |
3.0x |
mistralai/Mistral-7B-Instruct-v0.3 |
29.0 GB |
14.5 GB |
2.0x |
Qwen/Qwen2.5-14B-Instruct |
29.6 GB |
29.6 GB |
1.0x |
deepseek-ai/DeepSeek-V3 |
688.6 GB |
688.6 GB |
1.0x |
The multiplier is a property of how a publisher packaged the repo, it ranges from 1x to 8x across models we ship in examples/, and nothing on a model's HuggingFace page shows it. Sizing to the model underprovisions and the Job fills the volume; the only alternative is to overprovision by an unknown factor, or to find the number by listing the repo's files by hand.
How could Modelplane help solve your problem?
Make sizeGiB optional and derive it, defaulting to the size of what the Job is going to stage plus headroom. The HuggingFace API lists every file in a repo with its size, so this is exact rather than estimated, and it is the repository's size because the repository is what gets staged.
apiVersion: modelplane.ai/v1alpha1
kind: ModelCache
metadata:
name: llama-31-405b
namespace: ml-team
spec:
source: HuggingFace
huggingFace:
repo: meta-llama/Llama-3.1-405B-InstructThe field stays settable as an override. The docs line needs correcting either way, since it describes a rule that does not produce a working value.
Narrowing what gets staged is worth adding alongside it, as optional include and exclude file patterns defaulting to the whole repository, with the derived size following whatever they select:
spec:
source: HuggingFace
huggingFace:
repo: meta-llama/Llama-3.1-405B-Instruct
exclude: ["original/*"] # 2.4 TB -> 812 GBNarrowing has to be declared rather than inferred, because a ModelCache does not know which engine will read it. One cache serves many ModelDeployments, and the file a deployment needs depends on its loader: staging the safetensors shards openai/gpt-oss-20b indexes would silently drop the 13.8 GB metal/model.bin a llama.cpp-style engine reads. That is the same boundary storage.requires draws, and it is why this is a pattern list rather than a rule Modelplane applies. Passing the patterns through to what hf download already accepts also means no Modelplane vocabulary has to track new formats.
The patterns are safe here because resolution precedes hydration. The function matches them against the resolved file list and stages an explicit set, so the set that sizes the PVC and the set that gets fetched cannot disagree, and a pattern that matches nothing or drops config.json shows up in status before a byte moves. include applies first when set, then exclude removes, which is what huggingface_hub already does.
Between the two, the defaults stay on the safe side: an oversized stage costs storage, a mis-narrowed one costs a serving outage.
What problem are you facing?
spec.huggingFace.sizeGiBis required, and the docs say "Size it to the model, since a value below the model's size leaves no room to stage the weights." That instruction is wrong, and there is no instruction that would be right.The hydration Job stages the repository, not the model.
hf download {repo}fetches every file in the repo, and a HuggingFace repository routinely publishes the same weights more than once: a consolidated checkpoint beside the sharded safetensors (original/consolidated.00.pthin Llama,consolidated.safetensorsin Mistral), one copy per framework (bert ships safetensors, PyTorch, Flax, TensorFlow, ONNX and CoreML), or a second runtime's format (openai/gpt-oss-20bships a 13.8 GBmetal/model.bin).openai/whisper-large-v3google-bert/bert-base-uncasedmeta-llama/Llama-3.1-405B-Instructmistralai/Mistral-7B-Instruct-v0.3Qwen/Qwen2.5-14B-Instructdeepseek-ai/DeepSeek-V3The multiplier is a property of how a publisher packaged the repo, it ranges from 1x to 8x across models we ship in
examples/, and nothing on a model's HuggingFace page shows it. Sizing to the model underprovisions and the Job fills the volume; the only alternative is to overprovision by an unknown factor, or to find the number by listing the repo's files by hand.How could Modelplane help solve your problem?
Make
sizeGiBoptional and derive it, defaulting to the size of what the Job is going to stage plus headroom. The HuggingFace API lists every file in a repo with its size, so this is exact rather than estimated, and it is the repository's size because the repository is what gets staged.The field stays settable as an override. The docs line needs correcting either way, since it describes a rule that does not produce a working value.
Narrowing what gets staged is worth adding alongside it, as optional
includeandexcludefile patterns defaulting to the whole repository, with the derived size following whatever they select:Narrowing has to be declared rather than inferred, because a
ModelCachedoes not know which engine will read it. One cache serves manyModelDeployments, and the file a deployment needs depends on its loader: staging the safetensors shardsopenai/gpt-oss-20bindexes would silently drop the 13.8 GBmetal/model.bina llama.cpp-style engine reads. That is the same boundarystorage.requiresdraws, and it is why this is a pattern list rather than a rule Modelplane applies. Passing the patterns through to whathf downloadalready accepts also means no Modelplane vocabulary has to track new formats.The patterns are safe here because resolution precedes hydration. The function matches them against the resolved file list and stages an explicit set, so the set that sizes the PVC and the set that gets fetched cannot disagree, and a pattern that matches nothing or drops
config.jsonshows up in status before a byte moves.includeapplies first when set, thenexcluderemoves, which is whathuggingface_hubalready does.Between the two, the defaults stay on the safe side: an oversized stage costs storage, a mis-narrowed one costs a serving outage.