Henry Kroll III
hellork
AI & ML interests
The best models that work for my setup are any of Qwen/Qwen3.6-35B-A3B GGUF with APEX quants.
Consumer-grade (8GB GPU) text generation, image manipulation recognition, 3D model generation, language preservation and translation, and business use cases.
Recent Activity
new activity about 7 hours ago
unsloth/Qwen3.6-27B-MTP-GGUF:IQ3XXS giving garbage output new activity about 10 hours ago
unsloth/Qwen3.6-27B-MTP-GGUF:Official llama.cpp now has '--spec-type draft-mtp' new activity 1 day ago
unsloth/Qwen3.6-27B-MTP-GGUF:Low VRAM<=8GB best result (40t/s WITHOUT MTP)Organizations
Models to query photos
-
xtuner/llava-phi-3-mini-gguf
Image-to-Text • 4B • Updated • 2.48k • 137 -
cjpais/llava-1.6-mistral-7b-gguf
Image-Text-to-Text • 7B • Updated • 25.6k • 114 -
xtuner/llava-llama-3-8b-v1_1-gguf
Image-to-Text • 8B • Updated • 3.01k • 225 -
second-state/Llava-v1.5-7B-GGUF
Text Generation • 7B • Updated • 3.23k • 18
Under 10B GGUFs Non-linear Quantized to try on 4GiB VRAM. Leaderboard https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard
-
hellork/Qwen2-7B-Instruct-IQ4_NL-GGUF
Text Generation • 8B • Updated • 20 -
SixOpen/Nous-Capybara-7B-V1.9-IQ4_NL-GGUF
7B • Updated • 19 -
SixOpen/Phi-3-mini-4k-instruct-IQ4_NL-imat.gguf
Text Generation • 4B • Updated • 40 -
n3shelter/Peach-9B-8k-Roleplay-IQ4_XS-GGUF
Text Generation • 9B • Updated • 43 • 1
Models to query photos
-
xtuner/llava-phi-3-mini-gguf
Image-to-Text • 4B • Updated • 2.48k • 137 -
cjpais/llava-1.6-mistral-7b-gguf
Image-Text-to-Text • 7B • Updated • 25.6k • 114 -
xtuner/llava-llama-3-8b-v1_1-gguf
Image-to-Text • 8B • Updated • 3.01k • 225 -
second-state/Llava-v1.5-7B-GGUF
Text Generation • 7B • Updated • 3.23k • 18
Under 10B GGUFs Non-linear Quantized to try on 4GiB VRAM. Leaderboard https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard
-
hellork/Qwen2-7B-Instruct-IQ4_NL-GGUF
Text Generation • 8B • Updated • 20 -
SixOpen/Nous-Capybara-7B-V1.9-IQ4_NL-GGUF
7B • Updated • 19 -
SixOpen/Phi-3-mini-4k-instruct-IQ4_NL-imat.gguf
Text Generation • 4B • Updated • 40 -
n3shelter/Peach-9B-8k-Roleplay-IQ4_XS-GGUF
Text Generation • 9B • Updated • 43 • 1