code-gen P4.4: serve a HuggingFace model reference as a routing target

[gotoplanb]

Part of #22 · Tags: [Cognitive] [Agnostic] · Supersedes the cross-host slice of #32 · Deferred (see trigger below)

Let a routing rule point preferred_model / eval_shadow_models at a HuggingFace model repo; Conduct resolves it, stages it locally, and serves it through the same swap/dispatch path as a named local model.

- task_type: code_generation
  preferred_model: gemma4:e4b                                   # local tags still work
  eval_shadow_models:
    - { model: gemma4:12b, rate: 1.0 }
    - { model: "huggingface://gotoplanb/gemma-e4b-wander-dpo@<rev>", rate: 1.0 }

Framing — identity-agnostic (this is the whole point)

Conduct does not know or care whether a referenced model is "ours," a fine-tune, or a stranger's repo. huggingface://gotoplanb/gemma-e4b-dpo@v3 and huggingface://google/gemma-3-4b-it are the same kind of thing — a model source to resolve and serve. There is no special "register our checkpoint" handshake.

This dissolves the cross-host push problem the original version of this issue carried: with Conduct + Wander on separate hosts, there is nothing to push into Conduct. The handoff is one-directional and clean:

Wander: pull DPO data (client key, #41) → train → PUBLISH checkpoint to HF   (client's job)
Conduct: routing rule references huggingface://<repo>@<rev> → PULL + serve    (just a model source)

So this is a generic capability ("serve any HF model"), not a fine-tune-registration feature — same path a future user would use to point Conduct at any community model. Responsibilities: Conduct produces data + serves models; the client owns training + publishing. (Publishing to HF is never Conduct's job.)

Acceptance

• A rule referencing huggingface://<repo>@<rev> resolves and the worker dispatches to it.
• @<rev> is honored (pin to a commit for reproducible benches).
• model_used echoes the repo@rev string so downstream tools group by it.
• A failed pull (404 / auth / transient) surfaces in the job's error — never silently falls back to the primary.
• Private-repo pulls use Conduct's own HF_TOKEN (env), not client creds.
• One test: register a small public HF model, run one job through it, assert the result.

Open design questions (settle at build time)

• Revision pinning: SHA only (immutable) vs also tag refs (mutable on HF). Lean SHA for reproducible lift reports.
• Quantization: HF repos can ship multiple weights (bf16, q4_k_m…). Reference syntax to pick one — e.g. ...@<rev>#q4_k_m.
• Caching: lazy on first request vs pre-warm on routing-config load; keep a local mirror of pulled snapshots.

Why deferred

No checkpoint exists to serve yet (wander#7 hasn't run). Trigger: build this when wander#7 produces a checkpoint and wander#8 needs it benched — at that point the exact repo/rev, format, size, and quant are known, so the design questions above answer themselves.

Related

• code-gen P4.2: register a fine-tuned checkpoint as a selectable model #32 — same-host local-tag registration (shipped; this is the cross-host generalization)
• code-gen P4.3: publish composite-preference datasets to HuggingFace Hub #41 — client-key data export (shipped) — the inbound half of the same identity-agnostic, no-admin-creds handoff
• wander#7 (DPO run, produces the checkpoint) · wander#8 (lift report, benches it)

[gotoplanb]

Deferred (left open) — needs a model to move + a transfer-mechanism decision

This is a real future need but not actionable yet, and it carries a design decision that's Dave's to make — so I'm not building it reflexively.

Why not now

• There's no fine-tuned model to register yet. wander#7 (the DPO run) hasn't produced one. Until it does, there's nothing to pull.
• The cross-host question is real but undecided. Conduct and Wander run on separate hosts, so a fine-tuned model on Wander's box does need some transfer to Conduct's inference host. huggingface://repo@rev is one mechanism; others (a registry pull, an object-store URL, a one-time upload endpoint) are equally valid. Per Dave, the HF specifics are a suggestion. Picking the mechanism is worth doing when there's a concrete model + the lift-report (wander#8) requirements in hand, not speculatively.
• The local path already exists for same-host (code-gen P4.2: register a fine-tuned checkpoint as a selectable model #32: Ollama tag → routing target). The gap is purely cross-host transport.
• The author's own open questions (SHA-vs-tag pinning, quantization selection @v3#q4_k_m, caching strategy) confirm this needs design, not a reflexive build.

The auth principle is already satisfied

#42's framing was partly about avoiding SSH/admin creds into Conduct's host. Note the publish direction is now clean (#41): Wander pulls its data with a client key, no admin token. The model-pull direction can follow the same "no admin creds across the boundary" rule whenever we build it.

Suggested trigger

Revisit when wander#7 produces a checkpoint and wander#8 needs it benched on Conduct — at that point we'll know the exact model format/size and can pick the lightest transfer that honors the no-admin-creds rule. Leaving open as the tracking issue; supersedes the cross-host slice of #32 (same-host #32 already shipped).