---
license: gemma
base_model: google/gemma-3-27b-it
tags:
- eagle3
- speculative-decoding
- sglang
- draft-model
- rtx-5090
language:
- en
---

# gemma-3-27b-eagle3-drafter

an EAGLE-3 speculative-decoding draft head for [google/gemma-3-27b-it](https://huggingface.co/google/gemma-3-27b-it), trained from scratch on a single RTX 5090 (32GB) over four unattended nights.

pair it with Gemma-3-27B-it in sglang and generation gets faster with no change in output: verified tokens are exactly what the target model would have produced. measured on the training rig, best config (tree-3-4-8), against plain decoding:

| workload | speedup | accept length |
|---|---|---|
| repetitive text | 1.52x | 1.97 |
| code | 1.44x | 1.84 |
| prose | 1.33x | 1.56 |
| chat | 1.23x | 1.57 |

(release bench at the selected checkpoint, step 21000; base decode 59.6 tok/s on the same card, code reaching 89.0 tok/s with the drafter by the final bench. zero failed requests across all bench runs.)

## usage (sglang)

```bash
python -m sglang.launch_server \
  --model-path <your-gemma-3-27b-it path or AWQ variant> \
  --speculative-algorithm EAGLE3 \
  --speculative-draft-model-path witcheer/gemma-3-27b-eagle3-drafter \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 4 \
  --speculative-num-draft-tokens 8
```

the three configs benched:

| config | num_steps | eagle_topk | num_draft_tokens | note |
|---|---|---|---|---|
| chain-3-1-4 | 3 | 1 | 4 | cheapest, lowest gain |
| **tree-3-4-8** | 3 | 4 | 8 | **best net speedup, recommended** |
| tree-5-8-16 | 5 | 8 | 16 | higher accept length, lower net speedup: the extra draft work costs more than the extra accepts pay |

verified with sglang 0.5.14, target served as AWQ, context 4096, cuda graphs on.

## what's inside

- 716M-parameter single-layer llama-architecture EAGLE-3 head (`LlamaForCausalLMEagle3`, bf16), hidden size 5376 to match the Gemma-3-27B residual stream
- draft vocab 32000 with `d2t`/`t2d` maps to Gemma's 262k vocabulary
- no Gemma weights are included; every tensor in this repo was trained from scratch

## training

- framework: [SpecForge](https://github.com/sgl-project/SpecForge)
- data: a 54k-sample chat and code dataset
- hardware: one RTX 5090 (32GB), four nights of otherwise idle time (23:00 to ~05:20 each), ~25h GPU total
- checkpoint selection: every checkpoint was release-benched live; the speedup curve peaked at step 21000 (39% of one epoch) and regressed at 28000 (prose 1.33x to 1.24x), so 21000 ships. training loss alone does not show this: bench your checkpoints.

curve at a glance (tree-3-4-8):

| step | prose | code | repetitive | chat |
|---|---|---|---|---|
| 7000 | 1.22x | 1.40x | 1.37x | 1.25x |
| 14000 | 1.29x | 1.43x | 1.44x | 1.22x |
| **21000** | **1.33x** | **1.44x** | **1.52x** | **1.23x** |
| 28000 | 1.24x | 1.49x | 1.50x | 1.25x |

## licence

this head was trained on hidden states produced by Gemma-3-27B-it, so it is distributed as a Gemma model derivative under the [Gemma Terms of Use](https://ai.google.dev/gemma/terms). the drafter never generates final output on its own; all emitted tokens are verified by the Gemma target model.