---
license: other
license_name: nvidia-open-model-agreement
license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/
base_model: nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16
tags:
  - nvidia
  - nemotron
  - multimodal
  - quantized
  - 4-bit
  - int4
language:
  - en
library_name: transformers
pipeline_tag: any-to-any
---

# Nemotron-3-Nano-Omni-30B-A3B — INT4

INT4 weight-only quantization of
[`nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16`](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16).
Sized to fit a single ≥ 24 GB consumer / workstation GPU.

| Property | Value |
|---|---|
| Base model | nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 |
| Active parameters / token | ~3B (of 31B total) |
| Modality | text + image + audio + video → text |
| Quantization | INT4 weight-only |
| Approx. on-disk size | ~22 GB |
| Context length | up to 256k tokens |
| Languages | English |

## Validation

Loaded and verified with vLLM ≥ 0.20.0 (native `nemotron_v3` /
NanoNemotronVL path). Round-trip correctness: bit-exact within INT4
quantization step (per-layer dequantize MAE ≈ 1e-5).

### Score on `needle-1M-bench-mvp` 50K

| Metric | Score |
|---|---|
| Overall recall | **90.0 %** |
| Paper-anchored recall | 80.0 % |
| Synthetic-codes recall | 100.0 % |
| Haystack tokens | 50,566 |
| Max output tokens | 2048 |
| Scorer | `strip_think_includes` (centralized) |

Single miss is the deepest needle (depth 49,496 / 50,566). All 9 other
depths score 100 %.

Leaderboard:
[`drawais/needle-1M-bench-mvp`](https://huggingface.co/datasets/drawais/needle-1M-bench-mvp).
Per-row YAML:
[`.eval_results/nemotron-omni-30b-a3b-w4a16-50k.yaml`](https://huggingface.co/datasets/drawais/needle-1M-bench-mvp/blob/main/.eval_results/nemotron-omni-30b-a3b-w4a16-50k.yaml).

## Load (vLLM, text)

```python
from vllm import LLM, SamplingParams

llm = LLM(
    model="drawais/Nemotron-3-Nano-Omni-30B-A3B-W4A16",
    trust_remote_code=True,
    max_model_len=65536,
)
params = SamplingParams(temperature=0.6, top_p=0.95, max_tokens=4096)
print(llm.generate(["Hello, world!"], params)[0].outputs[0].text)
```

## Serve (vLLM, OpenAI-compatible)

```bash
vllm serve drawais/Nemotron-3-Nano-Omni-30B-A3B-W4A16 \
  --trust-remote-code \
  --max-model-len 65536 \
  --gpu-memory-utilization 0.94
```

Then point any OpenAI client at the local endpoint:

```python
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="dummy")
print(client.chat.completions.create(
    model="drawais/Nemotron-3-Nano-Omni-30B-A3B-W4A16",
    messages=[{"role": "user", "content": "Hello"}],
    max_tokens=64,
).choices[0].message.content)
```

For multimodal usage (image / audio / video), reasoning controls,
recommended `--reasoning-parser nemotron_v3`, tool-calling flags, and
per-modality serving recommendations, follow the upstream
[Nemotron-3-Nano-Omni model card](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16).
If audio inputs are used: `pip install vllm[audio]`.

## Footprint

~22 GB on disk for the weights. Total VRAM should leave headroom for
KV cache and multimodal-encoder activations; recommended:

- ≥ 32 GB consumer for full context + multimodal
- ≥ 24 GB with reduced `--max-model-len` and text-only usage

`trust_remote_code=True` is required.

## Acknowledgements

Source model © NVIDIA Corporation, released under the
[NVIDIA Open Model Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/).
This artifact is a Derivative Work as defined in that agreement.
See [`LICENSE`](LICENSE) and [`NOTICE`](NOTICE) for full text and
required attribution.

## License

[NVIDIA Open Model Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/) (Release Date: April 2, 2026).

Commercially usable. You are free to create and distribute Derivative
Works. NVIDIA does not claim ownership of outputs.

The full agreement text is included in [`LICENSE`](LICENSE). The
attribution notice required by Section 3(c) is in [`NOTICE`](NOTICE):

> *Licensed by NVIDIA Corporation under the NVIDIA Open Model Agreement.*