Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,75 @@
+---
+license: gemma
+base_model: google/gemma-3-27b-it
+tags:
+- eagle3
+- speculative-decoding
+- sglang
+- draft-model
+- rtx-5090
+language:
+- en
+---
+
+# gemma-3-27b-eagle3-drafter
+
+an EAGLE-3 speculative-decoding draft head for [google/gemma-3-27b-it](https://huggingface.co/google/gemma-3-27b-it), trained from scratch on a single RTX 5090 (32GB) over four unattended nights.
+
+pair it with Gemma-3-27B-it in sglang and generation gets faster with no change in output: verified tokens are exactly what the target model would have produced. measured on the training rig, best config (tree-3-4-8), against plain decoding:
+
+| workload | speedup | accept length |
+|---|---|---|
+| repetitive text | 1.52x | 1.97 |
+| code | 1.44x | 1.84 |
+| prose | 1.33x | 1.56 |
+| chat | 1.23x | 1.57 |
+
+(release bench at the selected checkpoint, step 21000; base decode 59.6 tok/s on the same card, code reaching 89.0 tok/s with the drafter by the final bench. zero failed requests across all bench runs.)
+
+## usage (sglang)
+
+```bash
+python -m sglang.launch_server \
+  --model-path <your-gemma-3-27b-it path or AWQ variant> \
+  --speculative-algorithm EAGLE3 \
+  --speculative-draft-model-path witcheer/gemma-3-27b-eagle3-drafter \
+  --speculative-num-steps 3 \
+  --speculative-eagle-topk 4 \
+  --speculative-num-draft-tokens 8
+```
+
+the three configs benched:
+
+| config | num_steps | eagle_topk | num_draft_tokens | note |
+|---|---|---|---|---|
+| chain-3-1-4 | 3 | 1 | 4 | cheapest, lowest gain |
+| **tree-3-4-8** | 3 | 4 | 8 | **best net speedup, recommended** |
+| tree-5-8-16 | 5 | 8 | 16 | higher accept length, lower net speedup: the extra draft work costs more than the extra accepts pay |
+
+verified with sglang 0.5.14, target served as AWQ, context 4096, cuda graphs on.
+
+## what's inside
+
+- 716M-parameter single-layer llama-architecture EAGLE-3 head (`LlamaForCausalLMEagle3`, bf16), hidden size 5376 to match the Gemma-3-27B residual stream
+- draft vocab 32000 with `d2t`/`t2d` maps to Gemma's 262k vocabulary
+- no Gemma weights are included; every tensor in this repo was trained from scratch
+
+## training
+
+- framework: [SpecForge](https://github.com/sgl-project/SpecForge)
+- data: a 54k-sample chat and code dataset
+- hardware: one RTX 5090 (32GB), four nights of otherwise idle time (23:00 to ~05:20 each), ~25h GPU total
+- checkpoint selection: every checkpoint was release-benched live; the speedup curve peaked at step 21000 (39% of one epoch) and regressed at 28000 (prose 1.33x to 1.24x), so 21000 ships. training loss alone does not show this: bench your checkpoints.
+
+curve at a glance (tree-3-4-8):
+
+| step | prose | code | repetitive | chat |
+|---|---|---|---|---|
+| 7000 | 1.22x | 1.40x | 1.37x | 1.25x |
+| 14000 | 1.29x | 1.43x | 1.44x | 1.22x |
+| **21000** | **1.33x** | **1.44x** | **1.52x** | **1.23x** |
+| 28000 | 1.24x | 1.49x | 1.50x | 1.25x |
+
+## licence
+
+this head was trained on hidden states produced by Gemma-3-27B-it, so it is distributed as a Gemma model derivative under the [Gemma Terms of Use](https://ai.google.dev/gemma/terms). the drafter never generates final output on its own; all emitted tokens are verified by the Gemma target model.