---
base_model: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
license: other
license_name: nvidia-open-model-license
tags:
- nemotron-h
- mixture-of-experts
- moe
- pruning
- reap
- expert-pruning
pipeline_tag: text-generation
library_name: transformers
---

# Nemotron-3-Nano-30B-A3B — REAP-pruned to 64 experts

A **50% expert-pruned** version of [`nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16),
produced with **REAP** (Router-weighted Expert Activation Pruning). Half the
routed experts in each MoE layer were removed, shrinking the model from
**~59 GB to ~32 GB** while keeping the same routing width.

| | Base | This model |
|---|---|---|
| Routed experts / MoE layer | 128 | **64** |
| Experts activated per token | 6 | 6 (unchanged) |
| Total params | ~31.6 B | ~31.6 B nominal (fewer experts stored) |
| Size (BF16) | ~59 GB | **~32 GB** |
| Architecture | `nemotron_h` (hybrid Mamba-Transformer MoE) | same |

## What is REAP?

[REAP](https://github.com/CerebrasResearch/reap) is a one-shot, post-training
method (no retraining) that scores every expert by the mean of
`router_weight × activation_norm` over a calibration set, then removes the
least-salient experts. Because only a few experts fire per token, many are
redundant — pruning them reduces the model's memory footprint. Note that
**experts-per-token is unchanged (6)**, so the *active* compute per token is the
same; the win is **total size / memory**, not decode speed.

## How this was made

- **Method:** REAP, layer-wise (block-by-block) calibration.
- **Compression:** `compression_ratio 0.5` → 64 of 128 routed experts kept per MoE layer.
- **Calibration:** 256 samples from [`theblackcat102/evol-codealpaca-v1`](https://huggingface.co/datasets/theblackcat102/evol-codealpaca-v1) at 512-token context.
- **Hardware:** pruned on an NVIDIA DGX Spark (GB10).

## Caveats

- This is a **moderate-calibration** prune (256 samples). It generates coherent,
  on-task text in sanity checks, but has **not** been run through formal
  benchmarks — expect some quality regression vs the base model, especially
  outside the coding/reasoning calibration domain. A production prune would use
  a larger, more diverse calibration set.
- `nemotron_h` includes Mamba layers. With `trust_remote_code=False` (native
  transformers modeling), it runs on a **pure-torch fallback**; install
  [`mamba-ssm`](https://github.com/state-spaces/mamba) + `causal-conv1d` for the
  fast path.

## Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

repo = "sapidlabs/Nemotron-3-Nano-30B-A3B-REAP-64e"  # this repo
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
    repo, torch_dtype=torch.bfloat16, device_map="cuda"
).eval()

msgs = [{"role": "user", "content": "Write a Python function for the nth Fibonacci number."}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True).to("cuda")
out = model.generate(**inputs, max_new_tokens=256)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

## License

Inherits the base model's license (NVIDIA Open Model License). This is a derivative of `nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B`.