callensxavier commited on
Commit
9300a94
·
verified ·
1 Parent(s): b3b7b28

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +30 -46
README.md CHANGED
@@ -8,79 +8,63 @@ tags:
8
  - energy-efficient
9
  - green-ai
10
  - runux
11
- - gemma2
12
- - inference
13
  base_model: google/gemma-2-9b
14
  pipeline_tag: text-generation
15
  ---
16
 
17
- <div align="center">
18
 
19
- # RunuX-AI Benchmark: Gemma 2 9B
 
 
20
 
21
- ### Google TPU v5e | 3.23x Faster | 3.23x Less Energy
22
 
23
- **Xavier Callens** | [Socrate AI Lab](https://github.com/xaviercallens/runux-ai-runtime)
24
-
25
- [Full Dataset](https://huggingface.co/datasets/callensxavier/runux-tpu-v5e-benchmarks) | [Scientific Article](https://huggingface.co/datasets/callensxavier/runux-tpu-v5e-benchmarks/blob/main/scientific_article.md)
26
-
27
- </div>
28
-
29
- ---
30
-
31
- ## Results (BS=1, BF16, TPU v5e)
32
-
33
- | Metric | PyTorch/XLA | JetStream | **RunuX-AI** | **vs PyTorch** |
34
- |:-------|:-----------:|:---------:|:------------:|:--------------:|
35
- | Throughput | 18.2 tok/s | 28.6 tok/s | **58.8 tok/s** | **3.23x** |
36
- | Energy | 10.99 J/tok | - | **3.4 J/tok** | **3.23x** |
37
- | MXU Util | ~32% | ~40% | **88%** | **2.75x** |
38
- | Cost/M tok | $18.31 | - | **$5.67** | **-68%** |
39
 
40
  ## Model Details
41
 
42
- | | |
43
- |:---|:---|
44
- | **Base Model** | [google/gemma-2-9b](https://huggingface.co/google/gemma-2-9b) |
45
- | **Parameters** | 9.0B |
46
- | **Precision** | BF16 (bfloat16) |
47
- | **Hardware** | Google TPU v5e (v5litepod-1, 197 TFLOPS) |
48
- | **Runtime** | RunuX-AI v0.2.0 (no_std Rust, 23 crates) |
49
 
50
  ## Methodology
51
 
52
- | Parameter | Value |
53
- |:----------|:------|
54
- | Input tokens | 512 |
55
- | Decode tokens | 128 (greedy) |
56
- | Warmup | 3 iterations |
57
- | Measurement | 10 iterations (median) |
58
- | Framework | PyTorch 2.4.0 + torch_xla 2.4.0 |
59
 
60
- See [full methodology](https://huggingface.co/datasets/callensxavier/runux-tpu-v5e-benchmarks/blob/main/methodology.md).
61
-
62
- ## Reproduce Baselines
63
 
64
  ```bash
 
65
  pip install torch torch_xla[tpu] transformers accelerate
66
- python benchmark_baselines.py
67
- ```
68
-
69
- ## Collaboration
70
 
71
- RunuX-AI is available for licensing through Socrate AI Lab. See our [collaboration page](https://huggingface.co/datasets/callensxavier/runux-tpu-v5e-benchmarks) for details.
 
 
72
 
73
  ## Citation
74
 
75
  ```bibtex
76
  @article{callens2026runux,
77
- title={RunuX-AI: 3x Inference Throughput on TPU v5e},
78
  author={Callens, Xavier},
79
  year={2026},
80
  note={Socrate AI Lab}
81
  }
82
  ```
83
 
84
- ---
85
 
86
- *2026 Xavier Callens / Socrate AI Lab | Data: Apache-2.0 | Runtime: Patent Pending*
 
 
 
8
  - energy-efficient
9
  - green-ai
10
  - runux
 
 
11
  base_model: google/gemma-2-9b
12
  pipeline_tag: text-generation
13
  ---
14
 
15
+ # RunuX-AI Benchmark: google/gemma-2-9b on TPU v5e
16
 
17
+ > **Benchmark validation card** — This repository documents the inference performance
18
+ > of [google/gemma-2-9b](https://huggingface.co/google/gemma-2-9b) when
19
+ > optimized with the RunuX-AI runtime on Google TPU v5e.
20
 
21
+ ## Key Results (BS=1, BF16, TPU v5e)
22
 
23
+ | Metric | PyTorch (torch_xla) | RunuX-AI | Improvement |
24
+ |--------|:-------------------:|:--------:|:-----------:|
25
+ | Throughput | 18.2 tok/s | **58.8 tok/s** | **3.23× faster** |
26
+ | Energy | 10.99 J/tok | **3.4 J/tok** | **3.23× lower** |
27
+ | MXU Util | ~32% | **88%** | **2.75× higher** |
 
 
 
 
 
 
 
 
 
 
 
28
 
29
  ## Model Details
30
 
31
+ - **Base Model**: [google/gemma-2-9b](https://huggingface.co/google/gemma-2-9b)
32
+ - **Parameters**: 9.0B
33
+ - **Precision**: BF16 (bfloat16)
34
+ - **Hardware**: Google TPU v5e (v5litepod-1, 197 TFLOPS)
35
+ - **Runtime**: RunuX-AI v0.2.0 (no_std Rust, 23 crates)
 
 
36
 
37
  ## Methodology
38
 
39
+ - Input: 512 tokens
40
+ - Decode: 128 tokens (greedy, `do_sample=False`)
41
+ - Warmup: 3 iterations
42
+ - Measurement: 10 iterations (median)
43
+ - See [full methodology](https://huggingface.co/datasets/callensxavier/runux-tpu-v5e-benchmarks/blob/main/methodology.md)
 
 
44
 
45
+ ## Reproduction
 
 
46
 
47
  ```bash
48
+ # Install baseline framework
49
  pip install torch torch_xla[tpu] transformers accelerate
 
 
 
 
50
 
51
+ # Run baseline benchmark
52
+ python benchmark_baselines.py --models gemma-2-9b
53
+ ```
54
 
55
  ## Citation
56
 
57
  ```bibtex
58
  @article{callens2026runux,
59
+ title={RunuX-AI: Memory-Efficient, Energy-Aware Inference Runtime for Edge and Cloud Accelerators},
60
  author={Callens, Xavier},
61
  year={2026},
62
  note={Socrate AI Lab}
63
  }
64
  ```
65
 
66
+ ## Author
67
 
68
+ **Xavier Callens** Socrate AI Lab (Non-Profit)
69
+ - GitHub: [xaviercallens/runux-ai-runtime](https://github.com/xaviercallens/runux-ai-runtime)
70
+ - Dataset: [callensxavier/runux-tpu-v5e-benchmarks](https://huggingface.co/datasets/callensxavier/runux-tpu-v5e-benchmarks)