prabanta-dev/Qwen3-Next-80B-A3B-Instruct-JNS
Text Generation • Updated
Qwen for Janas-LLM
37 tok/s from Qwen3-Next-80B on a 32 GB laptop; 24 without drafts vs 2.8 for llama.cpp. Qwen models converted for Janas-LLM.
Note 48 GB, more than the laptop's memory: writes 24 tok/s vs 2.8 for llama.cpp (37 with its draft block), reads 62 vs 16.
prabanta-dev/Qwen3-Coder-Next-JNS
Text Generation • UpdatedNote 48 GB, more than the laptop's memory: writes 22 tok/s vs 2.2 for llama.cpp, reads 66 vs 14. For code.
prabanta-dev/Qwen3.6-35B-A3B-JNS
Text Generation • Updated • 1Note Writes 19 tok/s vs 12 for llama.cpp, reads 104 vs 79.
prabanta-dev/Qwen3-30B-A3B-JNS
Text Generation • UpdatedNote Writes 32 tok/s vs 15 for llama.cpp, reads 135 vs 122. Also bartowski's Q4_K_M with an importance matrix, and unsloth's 3 bits (writes 36 vs 11, but reads a prompt slower: 100 vs 112).
prabanta-dev/Qwen3.5-9B-JNS
Text Generation • UpdatedNote Writes 14 tok/s vs 8.7 for llama.cpp, reads 111 vs 78.
prabanta-dev/Qwen3-4B-JNS
Text Generation • UpdatedNote Writes 27 tok/s vs 18 for llama.cpp, reads 193 vs 164. Also bartowski's Q4_K_M with an importance matrix (26 vs 18, 191 vs 160).
prabanta-dev/Qwen3.5-2B-JNS
Text Generation • UpdatedNote Writes 52 tok/s vs 28 for llama.cpp, reads 441 vs 323.
prabanta-dev/Qwen3-0.6B-JNS
Text Generation • UpdatedNote Writes 130 tok/s vs 45 for llama.cpp, reads 939 vs 670.
prabanta-dev/Qwen3.5-0.8B-JNS
Text Generation • Updated • 1Note Writes 109 tok/s vs 32 for llama.cpp, reads 799 vs 519. A test model.
prabanta-dev/Qwen3-Embedding-0.6B-JNS
Feature Extraction • UpdatedNote Embeddings, for janas-server --embedding-model.