callensxavier commited on
Commit
b3b7b28
·
verified ·
1 Parent(s): b4df4fd

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +46 -30
README.md CHANGED
@@ -8,63 +8,79 @@ tags:
8
  - energy-efficient
9
  - green-ai
10
  - runux
 
 
11
  base_model: google/gemma-2-9b
12
  pipeline_tag: text-generation
13
  ---
14
 
15
- # RunuX-AI Benchmark: google/gemma-2-9b on TPU v5e
16
 
17
- > **Benchmark validation card** — This repository documents the inference performance
18
- > of [google/gemma-2-9b](https://huggingface.co/google/gemma-2-9b) when
19
- > optimized with the RunuX-AI runtime on Google TPU v5e.
20
 
21
- ## Key Results (BS=1, BF16, TPU v5e)
22
 
23
- | Metric | PyTorch (torch_xla) | RunuX-AI | Improvement |
24
- |--------|:-------------------:|:--------:|:-----------:|
25
- | Throughput | 18.2 tok/s | **58.8 tok/s** | **3.23× faster** |
26
- | Energy | 10.99 J/tok | **3.4 J/tok** | **3.23× lower** |
27
- | MXU Util | ~32% | **88%** | **2.75× higher** |
 
 
 
 
 
 
 
 
 
 
 
28
 
29
  ## Model Details
30
 
31
- - **Base Model**: [google/gemma-2-9b](https://huggingface.co/google/gemma-2-9b)
32
- - **Parameters**: 9.0B
33
- - **Precision**: BF16 (bfloat16)
34
- - **Hardware**: Google TPU v5e (v5litepod-1, 197 TFLOPS)
35
- - **Runtime**: RunuX-AI v0.2.0 (no_std Rust, 23 crates)
 
 
36
 
37
  ## Methodology
38
 
39
- - Input: 512 tokens
40
- - Decode: 128 tokens (greedy, `do_sample=False`)
41
- - Warmup: 3 iterations
42
- - Measurement: 10 iterations (median)
43
- - See [full methodology](https://huggingface.co/datasets/callensxavier/runux-tpu-v5e-benchmarks/blob/main/methodology.md)
 
 
44
 
45
- ## Reproduction
 
 
46
 
47
  ```bash
48
- # Install baseline framework
49
  pip install torch torch_xla[tpu] transformers accelerate
50
-
51
- # Run baseline benchmark
52
- python benchmark_baselines.py --models gemma-2-9b
53
  ```
54
 
 
 
 
 
55
  ## Citation
56
 
57
  ```bibtex
58
  @article{callens2026runux,
59
- title={RunuX-AI: Memory-Efficient, Energy-Aware Inference Runtime for Edge and Cloud Accelerators},
60
  author={Callens, Xavier},
61
  year={2026},
62
  note={Socrate AI Lab}
63
  }
64
  ```
65
 
66
- ## Author
67
 
68
- **Xavier Callens** Socrate AI Lab (Non-Profit)
69
- - GitHub: [xaviercallens/runux-ai-runtime](https://github.com/xaviercallens/runux-ai-runtime)
70
- - Dataset: [callensxavier/runux-tpu-v5e-benchmarks](https://huggingface.co/datasets/callensxavier/runux-tpu-v5e-benchmarks)
 
8
  - energy-efficient
9
  - green-ai
10
  - runux
11
+ - gemma2
12
+ - inference
13
  base_model: google/gemma-2-9b
14
  pipeline_tag: text-generation
15
  ---
16
 
17
+ <div align="center">
18
 
19
+ # RunuX-AI Benchmark: Gemma 2 9B
 
 
20
 
21
+ ### Google TPU v5e | 3.23x Faster | 3.23x Less Energy
22
 
23
+ **Xavier Callens** | [Socrate AI Lab](https://github.com/xaviercallens/runux-ai-runtime)
24
+
25
+ [Full Dataset](https://huggingface.co/datasets/callensxavier/runux-tpu-v5e-benchmarks) | [Scientific Article](https://huggingface.co/datasets/callensxavier/runux-tpu-v5e-benchmarks/blob/main/scientific_article.md)
26
+
27
+ </div>
28
+
29
+ ---
30
+
31
+ ## Results (BS=1, BF16, TPU v5e)
32
+
33
+ | Metric | PyTorch/XLA | JetStream | **RunuX-AI** | **vs PyTorch** |
34
+ |:-------|:-----------:|:---------:|:------------:|:--------------:|
35
+ | Throughput | 18.2 tok/s | 28.6 tok/s | **58.8 tok/s** | **3.23x** |
36
+ | Energy | 10.99 J/tok | - | **3.4 J/tok** | **3.23x** |
37
+ | MXU Util | ~32% | ~40% | **88%** | **2.75x** |
38
+ | Cost/M tok | $18.31 | - | **$5.67** | **-68%** |
39
 
40
  ## Model Details
41
 
42
+ | | |
43
+ |:---|:---|
44
+ | **Base Model** | [google/gemma-2-9b](https://huggingface.co/google/gemma-2-9b) |
45
+ | **Parameters** | 9.0B |
46
+ | **Precision** | BF16 (bfloat16) |
47
+ | **Hardware** | Google TPU v5e (v5litepod-1, 197 TFLOPS) |
48
+ | **Runtime** | RunuX-AI v0.2.0 (no_std Rust, 23 crates) |
49
 
50
  ## Methodology
51
 
52
+ | Parameter | Value |
53
+ |:----------|:------|
54
+ | Input tokens | 512 |
55
+ | Decode tokens | 128 (greedy) |
56
+ | Warmup | 3 iterations |
57
+ | Measurement | 10 iterations (median) |
58
+ | Framework | PyTorch 2.4.0 + torch_xla 2.4.0 |
59
 
60
+ See [full methodology](https://huggingface.co/datasets/callensxavier/runux-tpu-v5e-benchmarks/blob/main/methodology.md).
61
+
62
+ ## Reproduce Baselines
63
 
64
  ```bash
 
65
  pip install torch torch_xla[tpu] transformers accelerate
66
+ python benchmark_baselines.py
 
 
67
  ```
68
 
69
+ ## Collaboration
70
+
71
+ RunuX-AI is available for licensing through Socrate AI Lab. See our [collaboration page](https://huggingface.co/datasets/callensxavier/runux-tpu-v5e-benchmarks) for details.
72
+
73
  ## Citation
74
 
75
  ```bibtex
76
  @article{callens2026runux,
77
+ title={RunuX-AI: 3x Inference Throughput on TPU v5e},
78
  author={Callens, Xavier},
79
  year={2026},
80
  note={Socrate AI Lab}
81
  }
82
  ```
83
 
84
+ ---
85
 
86
+ *2026 Xavier Callens / Socrate AI Lab | Data: Apache-2.0 | Runtime: Patent Pending*