oakmindai/Qwen3.6-35B-A3B-NVFP4-FTW

Qwen3.6-35B-A3B NVFP4 — Spark Lab FTW

This is a ready-to-run Spark Lab checkpoint for Qwen3.6-35B-A3B, optimized for the NVIDIA DGX Spark and its Grace Blackwell GB10 Superchip.

Spark Lab is a GB10-native frontier inference lab. It packages tested model recipes, hardware readiness checks, artifact preparation, unified-memory planning, NVMe-backed MoE execution, and OpenAI-/Anthropic-compatible APIs for one DGX Spark.

SparkLab GitHub: https://github.com/sixteen-miles-labs/sparklab

The commands and settings below follow Spark Lab's supported DGX Spark path: ARM64 Linux or DGX OS, CUDA 13, SM121 kernels, 128 GB coherent unified memory, and local NVMe.

Runtime scope: this Spark Lab artifact is validated for text input and text output. Image and video fields in the inherited upstream metadata are not enabled by this deployment path.

What this repository contains

This repository does not introduce a new model or a new quantization. It repackages NVIDIA's existing NVFP4 checkpoint into FreeToken Weight (FTW) format:

1. Qwen developed the original Qwen3.6-35B-A3B model.
2. NVIDIA produced the Qwen3.6-35B-A3B-NVFP4 quantized checkpoint with NVIDIA Model Optimizer.
3. FreeToken provides the native inference backend, FTW format, conversion tooling, NVFP4 kernels, and MoE expert-bank layout.
4. Spark Lab provides the DGX Spark product experience: model recipes, GB10 readiness checks, capacity planning, artifact lifecycle, deployment policy, and serving workflow.
5. OakMind AI performed, validated, documented, and published this FTW conversion.

The exact NVIDIA source revision is 491c2f1ea524c639598bf8fa787a93fed5a6fbce. Conversion is precision-preserving: the model remains NVFP4, while its tensors are aligned and sharded for SparkLab's native loader. The upstream BF16 MTP layer is included as a fourth FTW shard; no training or additional quantization was performed.

Why use FTW?

FreeToken can load the original Hugging Face safetensors checkpoint directly. FTW is an optional deployment format that performs the layout work ahead of time. It stores routed MoE experts in independently addressable expert banks and lets FreeToken use its native fast-loading path.

On NVIDIA DGX Spark, the GB10 exposes 128 GB of coherent unified memory shared by its Grace CPU and Blackwell GPU. This 35B-A3B checkpoint is small enough for Spark Lab to keep all expert banks resident in that memory. FTW is therefore mainly useful for simpler, repeatable deployment and faster loading; it does not change the model's expected output quality, and it should not be assumed to increase steady-state decode speed. The published artifact passed index/shard validation and a real OpenAI-compatible API generation smoke test on one GB10.

Run with Spark Lab on NVIDIA DGX Spark

This is the recommended path for a DGX Spark with its standard 128 GB unified-memory GB10 configuration. Spark Lab checks the machine before model loading and fails clearly when required DGX Spark, CUDA, memory, dependency, or storage conditions are not met.

Install SparkLab from its repository, then download this checkpoint. The hf download command creates a local model directory containing the FTW shards, index, configuration, and tokenizer:

git clone https://github.com/sixteen-miles-labs/sparklab.git
cd sparklab
./install.sh

hf download oakmindai/Qwen3.6-35B-A3B-NVFP4-FTW \
  --local-dir ~/models/Qwen3.6-35B-A3B-NVFP4-FTW

Ask Spark Lab to verify the DGX Spark environment before loading the model. The doctor checks the GB10/SM121 target, CUDA 13 runtime, available unified memory, dependencies, swap state, and whether the checkpoint is on suitable local NVMe storage:

sparklab doctor \
  --storage-path ~/models/Qwen3.6-35B-A3B-NVFP4-FTW

Start Spark Lab's OpenAI-compatible API server using the resident configuration tested for this checkpoint on one DGX Spark:

sparklab serve \
  --model ~/models/Qwen3.6-35B-A3B-NVFP4-FTW \
  --moe-backend offload \
  --moe-storage ram \
  --nvfp4-backend triton \
  --moe-cache-rate 1.0 \
  --num-tokens 8192 \
  --moe-prefill-hit-d2d \
  --host 127.0.0.1 \
  --port 8000

The native three-token MTP path is experimental and opt-in:

sparklab serve \
  --model ~/models/Qwen3.6-35B-A3B-NVFP4-FTW \
  --moe-backend offload \
  --moe-storage ram \
  --nvfp4-backend triton \
  --moe-cache-rate 1.0 \
  --num-tokens 8192 \
  --speculative-method mtp \
  --speculative-tokens 3 \
  --host 127.0.0.1 \
  --port 8000

On one GB10, the three-token profile accepted 41 of 50 drafts and reduced 64 output tokens to 23 target forwards, but reached only 8.57 tok/s versus 52.63 tok/s for the controlled target-only run. It reproduced the target-only greedy output hash; widths one and two did not. Keep MTP disabled for performance on this hardware. The sidecar is published so the runtime path can be evaluated and optimized without downloading the original safetensor shard separately.

The important options are:

• --moe-backend offload: uses FreeToken's cached MoE execution path.
• --moe-storage ram: keeps the FTW expert banks in unified memory instead of reading expert rows from disk during inference.
• --nvfp4-backend triton: selects the kernel layout baked into this artifact.
• --moe-cache-rate 1.0: admits all routed experts to the device/unified-memory cache.
• --num-tokens 8192: reserves KV-cache capacity for up to 8,192 tokens in the validated startup configuration; this is not a claim that 8K recall or long-context quality has been certified.
• --moe-prefill-hit-d2d: reuses cache-resident experts during prompt processing.

Once the readiness message appears, send a standard OpenAI chat-completions request:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen3.6-35B-A3B-NVFP4-FTW",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": false
  }'

Credits and license

Please credit the projects responsible for each part of this artifact:

• Model architecture and base weights: Qwen
• NVFP4 quantization: NVIDIA Model Optimizer
• DGX Spark product, orchestration, model workflow, and deployment guidance: SparkLab
• Native inference engine, FTW format, conversion, and kernels: FreeToken
• SparkLab source: sixteen-miles-labs/sparklab
• FTW conversion and publishing: OakMind AI

The upstream model is distributed under Apache License 2.0. This repository preserves the source model card and metadata. See the original repositories for their complete license terms, limitations, and intended-use guidance.

---

Selected content from the original NVIDIA model card follows below. Instructions for other runtime formats are omitted because this repository contains an FTW artifact for the Spark Lab workflow documented above.

Model Overview

Description:

The NVIDIA Qwen3.6-35B-A3B-NVFP4 model is the quantized version of Alibaba's Qwen3.6-35B-A3B model, which is an auto-regressive language model that uses an optimized transformer architecture. For more information, please check here. The NVIDIA Qwen3.6-35B-A3B-NVFP4 model is quantized with Model Optimizer.

This model is ready for commercial/non-commercial use.

Third-Party Community Consideration

This model is not owned or developed by NVIDIA. This model has been developed and built to a third-party’s requirements for this application and use case; see link to Non-NVIDIA (Qwen3.6-35B-A3B) Model Card from Alibaba.

References

NVIDIA Model Optimizer: https://github.com/NVIDIA/Model-Optimizer

License/Terms of Use:

Apache license 2.0

Deployment Geography:

Global

Use Case:

Developers looking to take off-the-shelf, pre-quantized models for deployment in AI Agent systems, chatbots, RAG systems, and other AI-powered applications.

Release Date:

Hugging Face on 05/28/2026 via https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4

Model Architecture:

Architecture Type: Transformers
Network Architecture: Mixture-of-Experts (MoE) with Hybrid Attention
Number of Model Parameters: 35B in total and 3B activated

Input:

Input Type(s): Text, Image, Video
Input Format(s): String, Red, Green, Blue (RGB), Video (MP4/WebM)
Input Parameters: One-Dimensional (1D), Two-Dimensional (2D), Three-Dimensional (3D)
Other Properties Related to Input: Context length up to 262K

Output:

Output Type(s): Text
Output Format: String
Output Parameters: One-Dimensional(1D): Sequences
Other Properties Related to Output: None

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Supported Hardware Microarchitecture Compatibility:

• NVIDIA Hopper, NVIDIA Blackwell
  

Preferred Operating System(s):

• Linux
  

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

Model Version(s):

The model version is NVFP4 1.0 version and is Quantized with nvidia-modelopt v0.44.0

Training and Evaluation Datasets:

Calibration Dataset:

Link: cnn_dailymail, Nemotron-Post-Training-Dataset-v2
Data Collection Method by dataset: Automated.
Labeling Method by dataset: Automated.
Properties: The cnn_dailymail dataset is an English-language dataset containing just over 300k unique news articles as written by journalists at CNN and the Daily Mail. The Nemotron-Post-Training-Dataset-v2 is a post-training dataset curated by NVIDIA containing multi-turn conversations across diverse topics.

Training Dataset:

Data Modality: Undisclosed
Data Collection Method by dataset: Undisclosed
Labeling Method by dataset: Undisclosed
Data Size: Undisclosed
Properties: Undisclosed

Evaluation Dataset:

Datasets: MMLU Pro, GPQA Diamond, τ²-Bench Telecom, MMMU Pro, SciCode, AIME 2025, AA-LCR, IFBench
Data Collection Method by dataset: Hybrid: Automated, Human
Labeling Method by dataset: Hybrid: Human, Automated
Properties: We evaluated the model on text-based reasoning and coding benchmarks: MMLU Pro is a multi-task language understanding benchmark with challenging multiple-choice questions across diverse academic domains; GPQA Diamond contains 448 graduate-level multiple-choice questions written by domain experts in biology, physics, and chemistry; τ²-Bench Telecom evaluates agentic tool-use and policy-adherence capabilities in dual-control telecom customer-service scenarios where the model interacts with a simulated user and external tools to resolve account issues; MMMU Pro is the more challenging version of the Massive Multi-discipline Multimodal Understanding benchmark, measuring college-level multimodal reasoning across diverse disciplines with expanded answer choices and a vision-only input setting; SciCode evaluates scientific coding capabilities; AIME 2025 contains problems from the American Invitational Mathematics Examination; AA-LCR (Artificial Analysis Long Context Recall) evaluates a model's ability to accurately retrieve and recall information from long input contexts; IFBench is a benchmark for evaluating instruction-following capabilities across diverse and structured task constraints.

Post Training Quantization

This model was obtained by quantizing the weights of Qwen3.6-35B-A3B to NVFP4 data type. Only the weights and activations of the linear operators within transformer blocks in MoE are quantized. This optimization reduces the number of bits per parameter from 16 to 4, reducing the disk size and GPU memory requirements by approximately 3.06x.

Evaluation

The accuracy benchmark results are presented in the table below:

Precision	MMLU Pro	GPQA Diamond	τ²-Bench Telecom	SciCode	AIME 2025	AA-LCR	IFBench	MMMU PRO
BF16	85.6	84.9	95.5	40.8	89.2	62.0	62.3	74.1
NVFP4	85.0	84.8	94.7	40.6	88.8	62.0	62.8	74.5

Baseline: Qwen3.6-35B-A3B. SciCode with temperature=0.6, top_p=0.95, max num tokens 131072; the others with temperature=1.0, top_p=0.95, max num tokens 131072

Model Limitations:

The base model was trained on data that contains toxic language and societal biases originally crawled from the internet. Therefore, the model may amplify those biases and return toxic responses especially when prompted with toxic prompts. The model may generate answers that may be inaccurate, omit key information, or include irrelevant or redundant text producing socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive.

Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included.

Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.