abenzerps

/

K2-Horizon-MoVA-36B-A4B-GGUF

like 8

Thanks for pointing this out. You are correct that standard upstream llama.cpp does not currently include K2-Horizon support. These GGUF files were built and tested with the publicly available MBZUAI-IFM llama.cpp fork, using its K2Horizon branch. The fork is separate from this model repository, which contains the GGUF artifacts and metadata only. As a result, an unmodified upstream llama.cpp build will not load these files at this time. I will update this note when upstream support becomes available.

and no fork in repo
ahem
https://github.com/MBZUAI-IFM/llama.cpp/tree/model/K2Horizon

I cloned that fork and branch (commit 35999d101cf2233fc54f09c3c8d599da7303ce02), but I get an error when trying to load the q4_k_m. Here is what I use to build:
cmake -B build -DGGML_CUDA=ON
cmake --build ./build --config Release -j --target llama-cli llama-mtmd-cli llama-server llama-gguf-split

Here is what I get for an error. Any ideas? Thanks!

0.01.628.951 I srv load_model: loading model 'D:\nn\models\llm\abenzerps\K2-Horizon-MoVA-36B-A4B-GGUF\K2-Horizon-MoVA-36B-A4B-Q4_K_M.gguf'
Failed to process regex: '(?:'[sS]|'[tT]|'[rR][eE]|'[vV][eE]|'[mM]|'[lL][lL]|'[dD])|[^\r\n\p{L}\p{N}]?(?:\p{L}|\p{M}|\u200C|\u200D)+|\p{N}{1,3}| ?[^\s\p{L}\p{N}]+[\r\n]|\s[\r\n]+|\s+(?!\S)|\s+'
Regex error: regex_error(error_escape): The expression contained an invalid escaped character, or a trailing escape.
0.02.541.994 E llama_model_load: error loading model: error loading model vocabulary: Failed to process regex
0.02.542.005 E llama_model_load_from_file_impl: failed to load model
0.02.542.056 E common_fit_params: encountered an error while trying to fit params to free device memory: failed to load model
Failed to process regex: '(?:'[sS]|'[tT]|'[rR][eE]|'[vV][eE]|'[mM]|'[lL][lL]|'[dD])|[^\r\n\p{L}\p{N}]?(?:\p{L}|\p{M}|\u200C|\u200D)+|\p{N}{1,3}| ?[^\s\p{L}\p{N}]+[\r\n]|\s[\r\n]+|\s+(?!\S)|\s+'
Regex error: regex_error(error_escape): The expression contained an invalid escaped character, or a trailing escape.
0.03.070.782 E llama_model_load: error loading model: error loading model vocabulary: Failed to process regex
0.03.070.794 E llama_model_load_from_file_impl: failed to load model
0.03.070.797 E cmn common_init_: failed to load model 'D:\nn\models\llm\abenzerps\K2-Horizon-MoVA-36B-A4B-GGUF\K2-Horizon-MoVA-36B-A4B-Q4_K_M.gguf'
0.03.070.799 E srv load_model: failed to load model, 'D:\nn\models\llm\abenzerps\K2-Horizon-MoVA-36B-A4B-GGUF\K2-Horizon-MoVA-36B-A4B-Q4_K_M.gguf'
0.03.070.814 I srv operator (): operator (): cleaning up before exit...
0.03.072.842 E

I cloned that fork and branch (commit 35999d101cf2233fc54f09c3c8d599da7303ce02), but I get an error when trying to load the q4_k_m. Here is what I use to build:
cmake -B build -DGGML_CUDA=ON
cmake --build ./build --config Release -j --target llama-cli llama-mtmd-cli llama-server llama-gguf-split

Here is what I get for an error. Any ideas? Thanks!

0.01.628.951 I srv load_model: loading model 'D:\nn\models\llm\abenzerps\K2-Horizon-MoVA-36B-A4B-GGUF\K2-Horizon-MoVA-36B-A4B-Q4_K_M.gguf'
Failed to process regex: '(?:'[sS]|'[tT]|'[rR][eE]|'[vV][eE]|'[mM]|'[lL][lL]|'[dD])|[^\r\n\p{L}\p{N}]?(?:\p{L}|\p{M}|\u200C|\u200D)+|\p{N}{1,3}| ?[^\s\p{L}\p{N}]+[\r\n]|\s[\r\n]+|\s+(?!\S)|\s+'
Regex error: regex_error(error_escape): The expression contained an invalid escaped character, or a trailing escape.
0.02.541.994 E llama_model_load: error loading model: error loading model vocabulary: Failed to process regex
0.02.542.005 E llama_model_load_from_file_impl: failed to load model
0.02.542.056 E common_fit_params: encountered an error while trying to fit params to free device memory: failed to load model
Failed to process regex: '(?:'[sS]|'[tT]|'[rR][eE]|'[vV][eE]|'[mM]|'[lL][lL]|'[dD])|[^\r\n\p{L}\p{N}]?(?:\p{L}|\p{M}|\u200C|\u200D)+|\p{N}{1,3}| ?[^\s\p{L}\p{N}]+[\r\n]|\s[\r\n]+|\s+(?!\S)|\s+'
Regex error: regex_error(error_escape): The expression contained an invalid escaped character, or a trailing escape.
0.03.070.782 E llama_model_load: error loading model: error loading model vocabulary: Failed to process regex
0.03.070.794 E llama_model_load_from_file_impl: failed to load model
0.03.070.797 E cmn common_init_: failed to load model 'D:\nn\models\llm\abenzerps\K2-Horizon-MoVA-36B-A4B-GGUF\K2-Horizon-MoVA-36B-A4B-Q4_K_M.gguf'
0.03.070.799 E srv load_model: failed to load model, 'D:\nn\models\llm\abenzerps\K2-Horizon-MoVA-36B-A4B-GGUF\K2-Horizon-MoVA-36B-A4B-Q4_K_M.gguf'
0.03.070.814 I srv operator (): operator (): cleaning up before exit...
0.03.072.842 E

Your build command is fine, and the GGUF file is not the problem.
The error happens while loading the tokenizer, before the model weights are loaded. On native Windows, this fork falls back to the system regex parser, which cannot process the model’s Unicode tokenizer pattern.
For now, the reliable workaround is to build and run the same fork under WSL2 with CUDA enabled:

git clone --branch model/K2Horizon https://github.com/MBZUAI-IFM/llama.cpp
cd llama.cpp
git checkout 35999d101cf2233fc54f09c3c8d599da7303ce02
cmake -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --target llama-cli llama-server

Then load the existing GGUF through its WSL path, such as /mnt/d/.... I tested this exact commit and Q4_K_M GGUF successfully on Linux/GCC.
A proper native Windows fix requires an update to the fork’s K2-Horizon tokenizer support.

and no fork in repo
ahem
https://github.com/MBZUAI-IFM/llama.cpp/tree/model/K2Horizon

official gglm repo I mean and I must admin that it is well hidden, I have then reviewed original model card and discussion and didn't find info about it...