NicoLab28

/

ClipProj-MiniMax-H3

like 139

You did nothing wrong — our example workflow was broken, and you loaded it. Sorry about that.

Look closely at your node: the values are shifted up by one slot. type is holding the projection filename, projection is holding cuda:0, device is holding resident. The node gained a type field after that workflow was published, and ComfyUI matches widgets by position rather than by name — so every value after it slid up, silently.

Two ways to fix it right now:

Either delete the node and add a fresh ClipProjLoader, then set:

• clip_name — Huihui-Qwen3-VL-4B-Instruct-abliterated-fp8_scaled.safetensors (yours is fine)
• type — auto
• projection — your mmh3-4b-*.safetensors
• device — cuda:0
• mode — resident

Or re-download the example workflow, which is fixed as of now:
https://github.com/nicolab28/ComfyUI-ClipProj/blob/main/example_workflows/minimax_h3_clipproj.json

One aside while you are at it: mmh3-4b-ClipProj-celeb-mlp is a couple of generations old. mmh3-4b-ClipProj-v3.1 is measurably better, especially on non-English speech, and it loads on the same node with no other change.

Thanks for reporting it — nobody else had, and it would have kept catching people.

Two things I should have added.

Set type to auto rather than minimax. It reads the safetensors header and works out the architecture on its own, so there is nothing to get wrong — and it keeps working if you swap the encoder for a different variant later.

And you probably want a simpler node than the one you are using. ClipProjLoader is the all-in-one version, built for multi-GPU setups — that is what the device and mode fields are for. On a single card it just gives you more ways to misconfigure things.

The plainer route is two nodes:

• ComfyUI's own CLIPLoader for the encoder
• ClipProjApply right after it, which takes the CLIP and applies the matrix

Same result, two widgets instead of five, and no device juggling to think about.

can't something similar be done for LTX models?

Short answer: I looked into it, and the effort/benefit ratio does not work out. Here is what I found.

LTX 2.5's conditioning interface is nothing like H3's. H3 taps a single layer of Qwen3-VL at 5120 dims, which is what makes a linear projection from a smaller encoder tractable. LTX 2.5 concatenates 49 layers of its fine-tuned Gemma 4 12B — 188,160 dims — and feeds that into a DualLinearProjection producing 4096 for video plus 2048 for audio. Learning a matrix into that is a much bigger problem, and the calibration corpus would have to be rebuilt from scratch.

The target itself is surprisingly compressible: I measured an effective rank of about 440 directions out of 6144, against 3255 out of 5120 for H3. So a projection would probably work. That is not the issue.

The issue is that there is almost nothing to win. An official Q4 of the 12B already exists at 7.38 GB, and dynamic VRAM handles it. A projected encoder might land around 3-5 GB — an economy, not the unlock ClipProj is on H3, where it takes 15.7 GB down to 4.6.

And LTX's fine-tuned 12B cannot generate text any more, it outputs gibberish. So a full pipeline needs a second model to write the prompt anyway, which eats back part of whatever VRAM you saved.

Weeks of calibration and benchmarking for a couple of gigabytes. I would rather spend that time on H3, where the projection actually changes what hardware can run the model. If LTX ever exposes a single-layer tap, the picture changes and I will revisit it.