Speculative-decoding drafts - a JRamirez-UAB Collection

JRamirez-UAB 's Collections

Sysadmin agent models: our picks

27B Dense MTP Vision for 24 GB GPUs

Speculative-decoding drafts

27B Visual Reasoning for 16-24 GB GPUs

4B Dense for 8-16 GB GPUs

9B Dense MTP Vision for 12-16 GB GPUs

19B-A3B MoE MTP Vision for 16-24 GB GPUs

30B-A3B MoE MTP for 24 GB GPUs

35B-A3B MoE MTP Vision for 24 GB GPUs

40B-A2.5B MoE for 24-48 GB GPUs

120-180B MoE Vision for 48-96 GB GPUs

Helpers: decision and triage

Experimental — HybridAgent on forked vLLM

Tested and discarded

Speculative-decoding drafts

updated 2 days ago

Draft models that speed up our models with speculative decoding. The output is the same as the target model's.

JRamirez-UAB/Qwen3.8-27B-DFlash2-W4A16

Text Generation • 2B • Updated 19 days ago • 489 • 1

Note DFlash2 draft for Qwen3.8-27B (W4A16 main, W8A16 branch). Tested on one and on two RTX 3090 (the 27B and the draft on the same GPU or on separate ones).
JRamirez-UAB/Qwen3.8-27B-DSpine-vLLM-quants

Text Generation • 2B • Updated 2 days ago • 435 • 1

Note DSpine draft for Qwen3.8-27B: 7 quantizations + vLLM plugin (vLLM 0.31 and 0.33)