Update README.md
Browse files
README.md
CHANGED
|
@@ -32,7 +32,7 @@ Merged via element-wise max across all calibration runs.
|
|
| 32 |
|
| 33 |
Even in NVFP4, this is a huge model and it requires 440GB of VRAM just for the weights, and easily 100GB more for activations and the K/V cache.
|
| 34 |
|
| 35 |
-
I've tested it with 8x RTX 6000 Pro and
|
| 36 |
|
| 37 |
4x RTX 6000 Pro may be possible with CPU offloading, but I haven't tried it.
|
| 38 |
|
|
|
|
| 32 |
|
| 33 |
Even in NVFP4, this is a huge model and it requires 440GB of VRAM just for the weights, and easily 100GB more for activations and the K/V cache.
|
| 34 |
|
| 35 |
+
I've tested it with 8x RTX 6000 Pro and get ~50 tokens/sec.
|
| 36 |
|
| 37 |
4x RTX 6000 Pro may be possible with CPU offloading, but I haven't tried it.
|
| 38 |
|