drawais/Nemotron-3-Nano-Omni-30B-A3B-W4A16

Nemotron-3-Nano-Omni-30B-A3B — INT4

INT4 weight-only quantization of nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16. Sized to fit a single ≥ 24 GB consumer / workstation GPU.

Property	Value
Base model	nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16
Active parameters / token	~3B (of 31B total)
Modality	text + image + audio + video → text
Quantization	INT4 weight-only
Approx. on-disk size	~22 GB
Context length	up to 256k tokens
Languages	English

Validation

Loaded and verified with vLLM ≥ 0.20.0 (native nemotron_v3 / NanoNemotronVL path). Round-trip correctness: bit-exact within INT4 quantization step (per-layer dequantize MAE ≈ 1e-5).

Score on needle-1M-bench-mvp 50K

Metric	Score
Overall recall	90.0 %
Paper-anchored recall	80.0 %
Synthetic-codes recall	100.0 %
Haystack tokens	50,566
Max output tokens	2048
Scorer	strip_think_includes (centralized)

Single miss is the deepest needle (depth 49,496 / 50,566). All 9 other depths score 100 %.

Leaderboard: drawais/needle-1M-bench-mvp. Per-row YAML: .eval_results/nemotron-omni-30b-a3b-w4a16-50k.yaml.

Load (vLLM, text)

from vllm import LLM, SamplingParams

llm = LLM(
    model="drawais/Nemotron-3-Nano-Omni-30B-A3B-W4A16",
    trust_remote_code=True,
    max_model_len=65536,
)
params = SamplingParams(temperature=0.6, top_p=0.95, max_tokens=4096)
print(llm.generate(["Hello, world!"], params)[0].outputs[0].text)

Serve (vLLM, OpenAI-compatible)

vllm serve drawais/Nemotron-3-Nano-Omni-30B-A3B-W4A16 \
  --trust-remote-code \
  --max-model-len 65536 \
  --gpu-memory-utilization 0.94

Then point any OpenAI client at the local endpoint:

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="dummy")
print(client.chat.completions.create(
    model="drawais/Nemotron-3-Nano-Omni-30B-A3B-W4A16",
    messages=[{"role": "user", "content": "Hello"}],
    max_tokens=64,
).choices[0].message.content)

For multimodal usage (image / audio / video), reasoning controls, recommended --reasoning-parser nemotron_v3, tool-calling flags, and per-modality serving recommendations, follow the upstream Nemotron-3-Nano-Omni model card. If audio inputs are used: pip install vllm[audio].

Footprint

~22 GB on disk for the weights. Total VRAM should leave headroom for KV cache and multimodal-encoder activations; recommended:

• ≥ 32 GB consumer for full context + multimodal
• ≥ 24 GB with reduced --max-model-len and text-only usage

trust_remote_code=True is required.

Acknowledgements

Source model © NVIDIA Corporation, released under the NVIDIA Open Model Agreement. This artifact is a Derivative Work as defined in that agreement. See LICENSE and NOTICE for full text and required attribution.

License

NVIDIA Open Model Agreement (Release Date: April 2, 2026).

Commercially usable. You are free to create and distribute Derivative Works. NVIDIA does not claim ownership of outputs.

The full agreement text is included in LICENSE. The attribution notice required by Section 3(c) is in NOTICE:

Licensed by NVIDIA Corporation under the NVIDIA Open Model Agreement.