Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Zhipu AI
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

REPORT.md

@@ -0,0 +1,112 @@
+# Pruning GLM-5.2 to 504B with REAP + Router-KD: an honest, well-powered study of what it costs — and a free fix
+
+**A technical report on expert-pruning GLM-5.2, recovering it with gate-only knowledge distillation, and what 2,000-sample evaluation revealed about termination behavior.**
+
+*Compute: 8× NVIDIA B200, sponsored by [Lambda](https://lambda.ai). Models: [0xSero on HuggingFace](https://huggingface.co/0xSero).*
+
+---
+
+## Abstract
+
+We prune **GLM-5.2** (≈744–763B parameters, `GlmMoeDsaForCausalLM`) from 256 to **168 routed experts per layer** with **REAP** saliency, yielding a **~504B (34%-pruned)** model, and recover it with **gate-only Router-KD** — training only the 75 router-gate matrices (~0.016% of parameters) to KL-match the unpruned teacher. We release the model in NVFP4 and GGUF.
+
+Our central finding is methodological as much as it is about the model. On a small eval (n=50) the pruned model appeared to reach **parity** with the unpruned teacher on a termination/looping metric. Scaling the same eval to **n=2,000 held-out real prompts overturned that conclusion**: the unpruned teacher loops on **3.6%** of prompts, the pruned model on **7.2%** — pruning **roughly doubles the loop rate** (two-proportion z≈5.0, p<0.0001). Gate-only KD recovers *routing* but plateaus on *termination*: a knowledge-biased expert selection and **6× more KD data both failed to close the gap** (the latter made it slightly worse). However, a **free, no-retraining sampler guardrail** (`min_p=0.05, repetition_penalty=1.10`) drops the pruned model to **2.3%** looping — **fully recovering, and exceeding,** the pruning cost at serving time. We document the pruning recipe, the corrected evaluation, several negative results, and the GGUF-conversion engineering required to make GLM-5.2's shared sparse-attention indexer loadable in llama.cpp.
+
+---
+
+## 1. Background
+
+**GLM-5.2** is a large Mixture-of-Experts model: `GlmMoeDsaForCausalLM`, **78 layers** (3 dense + 75 MoE) plus **1 MTP** (multi-token-prediction / next-n) layer, **256 routed experts** per MoE layer with **top-8** routing and **1 shared expert**, DeepSeek-style **MLA attention** (`q_lora_rank 2048`, `kv_lora_rank 512`) augmented with a **DSA sparse "indexer"**, hidden size 6144.
+
+**REAP** (Router-weighted Expert Activation Pruning) scores each expert by saliency = `gate_weight × ‖expert_output‖` over a calibration set, and keeps the top-K per layer. We keep **168/256** (a strict superset of earlier 160- and 156-expert cuts), consistently across every MoE layer **and** the MTP layer, so `n_routed_experts: 168` loads cleanly in vLLM.
+
+**The objective** was not raw accuracy but **agent-grade behavior**: does the pruned model *terminate*, or fall into repeat / `</think>`-restart loops? Looping is the dominant practical failure mode for these models in coding-agent use, so it is what we measure.
+
+---
+
+## 2. Methods
+
+### 2.1 Pruning + recovery
+- **Prune:** REAP saliency → top-168 experts/layer (incl. MTP), `n_routed_experts: 168`. ~504B params.
+- **Quantize:** NVFP4 (NVIDIA modelopt) on routed experts; BF16 router / attention / shared expert.
+- **Recover (Router-KD):** freeze the entire network; train **only the 75 router-gate matrices** (~0.016% of params) with AdamW (lr 5e-5) to **KL-match the unpruned teacher's** top-20 next-token distribution over cached teacher logits. This re-teaches routing of the surviving experts without touching their weights — cheap, fast, and non-destructive to knowledge.
+
+### 2.2 Evaluation
+- **Probes:** real held-out prompts harvested from the author's own coding-agent traces (codex, opencode-cli, cursor, claude-code) — the actual target distribution.
+- **Protocol:** raw sampling, **no `max_tokens`, no timeout**. Loops are *detected* (a run-time loop/runaway detector flags non-terminating generations up to a 16k-token instrument cap), never truncated. Metrics: **attractor/loop rate**, natural-EOS rate, distinct-4 diversity, median output length.
+- **The key methodological choice:** we scaled from **n=50 → n=2,000** probes. At n=50 a "win" is 4-vs-5 failures — pure noise. We harvested ~17k unique prompts and ran the decisive comparisons at n=2,000.
+
+---
+
+## 3. Results
+
+### 3.1 The leaderboard (n=2,000, raw sampling)
+
+| model / config | loop rate | 95% CI | EOS | distinct-4 | median tok |
+|---|---|---|---|---|---|
+| **unpruned teacher** (≈744B) | **3.6%** | [2.8–4.5] | 0.965 | 0.921 | 1207 |
+| floor keep-168, 3k-KD **(shipped)** | 7.2% | [6.1–8.4] | 0.928 | 0.880 | 1267 |
+| K-cut keep-168-K, 3k-KD | 7.6% | [6.5–8.8] | 0.925 | 0.886 | 1230 |
+| K-cut keep-168-K, full 18.6k-KD | 7.8% | [6.7–9.1] | 0.924 | 0.881 | 1232 |
+| floor keep-168, full 18.6k-KD | 8.6% | [7.4–9.9] | 0.915 | 0.872 | 1301 |
+
+### 3.2 Pruning roughly doubles the loop rate
+The pruned floor (7.2%) vs the unpruned teacher (3.6%) is a **statistically significant** gap: two-proportion **z ≈ 5.0, p < 0.0001**. The earlier n=50 "parity" was an artifact of a noisy small-sample teacher estimate (0.10 at n=50 vs its true 0.036). **Honest conclusion: REAP pruning costs ~3.6 points of termination robustness.** Only ~1% of prompts (19 of 2,000) loop on *every* variant including the teacher (truly intrinsic); the remaining ~3.6 points are **pruning-induced**.
+
+The cost concentrates on the hardest sources: **codex and opencode-cli loop ~13%** on the pruned model vs **~6%** on the teacher; cursor-composer/cursor-chat stay low (1–3%) on both.
+
+### 3.3 Gate-only KD plateaus
+Two interventions aimed at closing the gap both **failed**:
+- **Knowledge-biased experts (the "K-cut"):** replacing 8 experts/layer with the highest-saliency *knowledge/reasoning* experts that coding-saliency pruning drops — **no improvement** (7.6–7.8%, within noise).
+- **6× more KD data** (18,599 vs 2,999 calibration traces) — **slightly worse** (floor 7.2→8.6%, K-cut 7.6→7.8%). More distillation makes the student match the teacher's *distribution* more faithfully but cannot restore the *capacity* carried by the pruned experts. Gate-only KD has a ceiling.
+
+### 3.4 The free fix: a sampler guardrail
+What gate-KD could not do, a serving-time sampler knob does — for free (n=2,000, on the shipped floor):
+
+| guardrail | loop rate |
+|---|---|
+| none (raw) | 7.2% |
+| `min_p=0.05, rep_pen=1.05` | 4.9% (p=0.002 vs raw) |
+| `min_p=0.10, rep_pen=1.10` | 3.3% |
+| **`min_p=0.05, rep_pen=1.10`** | **2.3%** |
+
+A light repetition penalty (1.10) takes the pruned model to **2.3%** — below the teacher's *raw* 3.6%, and with distinct-4 rising to 0.95. **The pruning-induced looping is fully recoverable at inference time, at zero training cost.** (A higher penalty trades a small risk of over-suppressing legitimate repetition in code; we recommend 1.05 by default, 1.10 if loops appear.) Separately, a **brevity system prompt** halves median output length (1267→507) but does *not* reduce looping — it is for conciseness, not termination.
+
+---
+
+## 4. GGUF conversion: making GLM-5.2's DSA indexer loadable
+
+GLM-5.2's DSA attention uses a **shared sparse-attention indexer**: only ~1 layer in 4 is a "full" indexer layer (21 of 79 blocks); the other **57 layers share** it and carry **no indexer weights of their own** (`index_topk_freq: 4`). Stock llama.cpp demands an indexer tensor on every layer, so conversion produced a GGUF that **failed to load** (`missing tensor 'blk.3.indexer.k_norm.weight'`).
+
+**Fix:** we filled each of the 57 shared layers' indexer tensors (285 tensors total) by duplicating them from the nearest preceding full layer. The patched GGUF loads and generates coherently. A second snag — the quantizer rejecting the **MTP layer** (`blk.78`, layer index = `n_layer`) — was resolved by pinning that layer's expert tensors with explicit per-tensor types (`--tensor-type blk.78.*=q6_K`), **preserving MTP** (self-speculative decoding) in every quant. *Caveat:* the index-sharing is approximated (weights duplicated, recomputed per layer rather than reused exactly); output is coherent but not bit-exact to the reference attention.
+
+We release **BF16 + dynamic Q4_K_XL / Q3_K_XL / Q2_K_XL** (per-tensor precision: attention/shared-expert/embeddings/output kept high, routed experts pushed low). Imatrix calibration was not applied (GPU-time constraints).
+
+---
+
+## 5. Negative results (reported for honesty)
+- **Knowledge-augmented expert selection** did not reduce looping.
+- **6× more KD data** did not help (slightly hurt).
+- **A knowledge-recovery LoRA** (the intuitively right lever — add capacity back, distilled from the teacher) was attempted 6× and **did not train**: the 933 GB BF16 model needs ~170 GiB/GPU just to load (uneven MoE packing), leaving no room on GPU0 for the LoRA forward (a 15.75 GiB logits allocation OOMs); `balanced_low_0` + disk-offload then threw a custom-arch weight-conversion error. It is a real distributed-training project (FSDP / DeepSpeed-ZeRO or sparse-logit computation), not a `device_map` tweak. Left as future work.
+
+---
+
+## 6. Limitations
+- The eval measures **termination/looping on one author's agent-trace distribution**, not general capability or knowledge benchmarks. Pruned-vs-teacher *knowledge* differences are not quantified here.
+- The loop detector's 16k-token instrument cap means borderline-long generations are classified as loops; ~86% of flagged loops are variant-specific (near the cutoff), only ~1–2% are a stable intrinsic core.
+- GGUF index-sharing is approximated (§4). Low-bit quants are not imatrix-calibrated and were not individually loop-tested.
+- Teacher-with-guardrail was not measured (the pod was returned), so "2.3% beats the teacher" is vs the teacher's *raw* 3.6%.
+
+---
+
+## 7. Released artifacts
+- **[`0xSero/GLM-5.2-504B`](https://huggingface.co/0xSero/GLM-5.2-504B)** — the recommended model (keep-168, NVFP4). Serve with `min_p=0.05, repetition_penalty=1.05–1.10`.
+- **[`0xSero/GLM-5.2-REAP-504B-GGUF`](https://huggingface.co/0xSero/GLM-5.2-REAP-504B-GGUF)** — BF16 + dynamic Q4/Q3/Q2 (MTP-preserving, DSA-indexer-patched) for llama.cpp.
+- **[`0xSero/GLM-5.2-504B-K`](https://huggingface.co/0xSero/GLM-5.2-504B-K)** — knowledge-augmented variant (full-data KD).
+- **[`0xSero/GLM-5.2-504B-FullKD`](https://huggingface.co/0xSero/GLM-5.2-504B-FullKD)** — full-data plain-KD variant (ablation).
+
+## 8. Conclusion
+REAP + gate-only Router-KD produces a usable, 34%-smaller GLM-5.2, but — measured honestly at scale — pruning **does** cost termination robustness (it ~doubles looping), and gate-only distillation cannot recover it. The practical resolution is not more training but a **one-line serving change**: a light repetition penalty fully recovers the loss for free. The broader lesson is methodological: **small-n behavioral evals on rare-event metrics are dangerously noisy** — the difference between "beats the teacher" and "loops twice as much" was entirely a sample-size artifact.
+
+---
+*Pruning, distillation, evaluation, and analysis on 8× NVIDIA B200 sponsored by **[Lambda](https://lambda.ai)**. 🙏*

chat_template.jinja

@@ -0,0 +1,119 @@
+[gMASK]<sop>
+{%- set effective_reasoning_effort = 'high' if reasoning_effort is defined and reasoning_effort == 'high' else 'max' -%}
+{%- if (enable_thinking is not defined or enable_thinking) and effective_reasoning_effort is not none -%}<|system|>Reasoning Effort: {{ effective_reasoning_effort | capitalize }}{%- endif -%}
+{%- if tools -%}
+{%- macro tool_to_json(tool) -%}
+    {%- set ns_tool = namespace(first=true) -%}
+    {{ '{' -}}
+    {%- for k, v in tool.items() -%}
+        {%- if k != 'defer_loading' and k != 'strict' -%}
+            {%- if not ns_tool.first -%}{{- ', ' -}}{%- endif -%}
+            {%- set ns_tool.first = false -%}
+            "{{ k }}": {{ v | tojson(ensure_ascii=False) }}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- '}' -}}
+{%- endmacro -%}
+<|system|>
+# Tools
+
+You may call one or more functions to assist with the user query.
+
+You are provided with function signatures within <tools></tools> XML tags:
+<tools>
+{% for tool in tools %}
+{%- if 'function' in tool -%}
+    {%- set tool = tool['function'] -%}
+{%- endif -%}
+{% if tool.defer_loading is not defined or not tool.defer_loading %}
+{{ tool_to_json(tool) }}
+{% endif %}
+{% endfor %}
+</tools>
+
+For each function call, output the function name and arguments within the following XML format:
+<tool_call>{function-name}<arg_key>{arg-key-1}</arg_key><arg_value>{arg-value-1}</arg_value><arg_key>{arg-key-2}</arg_key><arg_value>{arg-value-2}</arg_value>...</tool_call>{%- endif -%}
+{%- macro visible_text(content) -%}
+    {%- if content is string -%}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping -%}
+        {%- for item in content -%}
+            {%- if item is mapping and item.type == 'text' -%}
+                {{- item.text }}
+            {%- elif item is string -%}
+                {{- item }}
+            {%- elif item is mapping and item.type in ['image', 'image_url', 'video', 'video_url', 'audio', 'audio_url', 'input_audio'] -%}
+                {%- set media_type = item.type | replace('_url', '') | replace('input_', '') -%}
+                {{- "<reminder>You are unable to process this " ~ media_type ~ " because you don't have multi-modal input ability. Try different methods.</reminder>" }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{- content }}
+    {%- endif -%}
+{%- endmacro -%}
+{%- set ns = namespace(last_user_index=-1) -%}
+{%- for m in messages %}
+    {%- if m.role == 'user' %}
+        {%- set ns.last_user_index = loop.index0 -%}
+    {%- endif %}
+{%- endfor %}
+{%- for m in messages -%}
+{%- if m.role == 'user' -%}<|user|>{{ visible_text(m.content) }}
+{%- elif m.role == 'assistant' -%}
+<|assistant|>
+{%- set content = visible_text(m.content) %}
+{%- if m.reasoning_content is string %}
+    {%- set reasoning_content = m.reasoning_content %}
+{%- elif '</think>' in content %}
+    {%- set reasoning_content = content.split('</think>')[0].split('<think>')[-1] %}
+    {%- set content = content.split('</think>')[-1] %}
+{%- endif %}
+{%- if ((clear_thinking is defined and not clear_thinking) or loop.index0 > ns.last_user_index) and reasoning_content is defined -%}
+{{ '<think>' + reasoning_content +  '</think>'}}
+{%- else -%}
+{{ '<think></think>' }}
+{%- endif -%}
+{%- if content.strip() -%}
+{{ content.strip() }}
+{%- endif -%}
+{% if m.tool_calls %}
+{% for tc in m.tool_calls %}
+{%- if tc.function %}
+    {%- set tc = tc.function %}
+{%- endif %}
+{{- '<tool_call>' + tc.name -}}
+{% set _args = tc.arguments %}{% for k, v in _args.items() %}<arg_key>{{ k }}</arg_key><arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>{% endfor %}</tool_call>{% endfor %}
+{% endif %}
+{%- elif m.role == 'tool' -%}
+{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+    {{- '<|observation|>' -}}
+{%- endif %}
+{%- if m.content is string -%}
+    {{- '<tool_response>' + m.content + '</tool_response>' -}}
+{%- elif m.content is iterable and m.content is not mapping and m.content and m.content.0.type == "tool_reference" -%}
+    {{- '<tool_response><tools>\n' -}}
+    {% for tr in m.content %}
+        {%- for tool in tools -%}
+            {%- if 'function' in tool -%}
+                {%- set tool = tool['function'] -%}
+            {%- endif -%}
+            {%- if tool.name == tr.name -%}
+                {{- tool_to_json(tool) + '\n' -}}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- endfor -%}
+    {{- '</tools></tool_response>' -}}
+{%- elif m.content is iterable and m.content is not mapping and m.content and m.content.0 is mapping and m.content.0.output is defined -%}
+    {%- for tr in m.content -%}
+        {{- '<tool_response>' + tr.output + '</tool_response>' -}}
+    {%- endfor -%}
+{%- else -%}
+    {{- '<tool_response>' + visible_text(m.content) + '</tool_response>' -}}
+{% endif -%}
+{%- elif m.role == 'system' -%}
+<|system|>{{ visible_text(m.content) }}
+{%- endif -%}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    <|assistant|>{{- '<think></think>' if (enable_thinking is defined and not enable_thinking) else '<think>' -}}
+{%- endif -%}

config.json

@@ -0,0 +1,418 @@
+{
+  "architectures": [
+    "GlmMoeDsaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "dtype": "bfloat16",
+  "eos_token_id": [
+    154820,
+    154827,
+    154829
+  ],
+  "ep_size": 1,
+  "first_k_dense_replace": 3,
+  "head_dim": 192,
+  "hidden_act": "silu",
+  "hidden_size": 6144,
+  "index_head_dim": 128,
+  "index_n_heads": 32,
+  "index_share_for_mtp_iteration": true,
+  "index_skip_topk_offset": 3,
+  "index_topk": 2048,
+  "index_topk_freq": 4,
+  "index_topk_pattern": null,
+  "indexer_rope_interleave": true,
+  "indexer_types": [
+    "full",
+    "full",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared"
+  ],
+  "initializer_range": 0.02,
+  "intermediate_size": 12288,
+  "kv_lora_rank": 512,
+  "max_position_embeddings": 1048576,
+  "mlp_layer_types": [
+    "dense",
+    "dense",
+    "dense",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse"
+  ],
+  "model_type": "glm_moe_dsa",
+  "moe_intermediate_size": 2048,
+  "moe_layer_freq": 1,
+  "n_group": 1,
+  "n_routed_experts": 168,
+  "n_shared_experts": 1,
+  "norm_topk_prob": true,
+  "num_attention_heads": 64,
+  "num_experts_per_tok": 8,
+  "num_hidden_layers": 78,
+  "num_key_value_heads": 64,
+  "num_nextn_predict_layers": 1,
+  "pad_token_id": 154820,
+  "pretraining_tp": 1,
+  "q_lora_rank": 2048,
+  "qk_head_dim": 256,
+  "qk_nope_head_dim": 192,
+  "qk_rope_head_dim": 64,
+  "rms_norm_eps": 1e-05,
+  "rope_interleave": true,
+  "rope_parameters": {
+    "rope_theta": 8000000,
+    "rope_type": "default"
+  },
+  "routed_scaling_factor": 2.5,
+  "scoring_func": "sigmoid",
+  "tie_word_embeddings": false,
+  "topk_group": 1,
+  "topk_method": "noaux_tc",
+  "transformers_version": "5.12.0",
+  "use_cache": true,
+  "v_head_dim": 256,
+  "vocab_size": 154880,
+  "quantization_config": {
+    "config_groups": {
+      "group_0": {
+        "input_activations": {
+          "dynamic": false,
+          "num_bits": 4,
+          "type": "float",
+          "group_size": 16
+        },
+        "weights": {
+          "dynamic": false,
+          "num_bits": 4,
+          "type": "float",
+          "group_size": 16
+        },
+        "targets": [
+          "Linear"
+        ]
+      }
+    },
+    "ignore": [
+      "lm_head",
+      "model.layers.0.mlp*",
+      "model.layers.0.self_attn*",
+      "model.layers.0.self_attn.indexer*",
+      "model.layers.1.mlp*",
+      "model.layers.1.self_attn*",
+      "model.layers.1.self_attn.indexer*",
+      "*shared_experts*",
+      "model.layers.10.self_attn*",
+      "model.layers.10.self_attn.indexer*",
+      "model.layers.11.self_attn*",
+      "model.layers.11.self_attn.indexer*",
+      "model.layers.12.self_attn*",
+      "model.layers.12.self_attn.indexer*",
+      "model.layers.13.self_attn*",
+      "model.layers.13.self_attn.indexer*",
+      "model.layers.14.self_attn*",
+      "model.layers.14.self_attn.indexer*",
+      "model.layers.15.self_attn*",
+      "model.layers.15.self_attn.indexer*",
+      "model.layers.16.self_attn*",
+      "model.layers.16.self_attn.indexer*",
+      "model.layers.17.self_attn*",
+      "model.layers.17.self_attn.indexer*",
+      "model.layers.18.self_attn*",
+      "model.layers.18.self_attn.indexer*",
+      "model.layers.19.self_attn*",
+      "model.layers.19.self_attn.indexer*",
+      "model.layers.2.mlp*",
+      "model.layers.2.self_attn*",
+      "model.layers.2.self_attn.indexer*",
+      "model.layers.20.self_attn*",
+      "model.layers.20.self_attn.indexer*",
+      "model.layers.21.self_attn*",
+      "model.layers.21.self_attn.indexer*",
+      "model.layers.22.self_attn*",
+      "model.layers.22.self_attn.indexer*",
+      "model.layers.23.self_attn*",
+      "model.layers.23.self_attn.indexer*",
+      "model.layers.24.self_attn*",
+      "model.layers.24.self_attn.indexer*",
+      "model.layers.25.self_attn*",
+      "model.layers.25.self_attn.indexer*",
+      "model.layers.26.self_attn*",
+      "model.layers.26.self_attn.indexer*",
+      "model.layers.27.self_attn*",
+      "model.layers.27.self_attn.indexer*",
+      "model.layers.28.self_attn*",
+      "model.layers.28.self_attn.indexer*",
+      "model.layers.29.self_attn*",
+      "model.layers.29.self_attn.indexer*",
+      "model.layers.3.self_attn*",
+      "model.layers.3.self_attn.indexer*",
+      "model.layers.30.self_attn*",
+      "model.layers.30.self_attn.indexer*",
+      "model.layers.31.self_attn*",
+      "model.layers.31.self_attn.indexer*",
+      "model.layers.32.self_attn*",
+      "model.layers.32.self_attn.indexer*",
+      "model.layers.33.self_attn*",
+      "model.layers.33.self_attn.indexer*",
+      "model.layers.34.self_attn*",
+      "model.layers.34.self_attn.indexer*",
+      "model.layers.35.self_attn*",
+      "model.layers.35.self_attn.indexer*",
+      "model.layers.36.self_attn*",
+      "model.layers.36.self_attn.indexer*",
+      "model.layers.37.self_attn*",
+      "model.layers.37.self_attn.indexer*",
+      "model.layers.38.self_attn*",
+      "model.layers.38.self_attn.indexer*",
+      "model.layers.39.self_attn*",
+      "model.layers.39.self_attn.indexer*",
+      "model.layers.4.self_attn*",
+      "model.layers.4.self_attn.indexer*",
+      "model.layers.40.self_attn*",
+      "model.layers.40.self_attn.indexer*",
+      "model.layers.41.self_attn*",
+      "model.layers.41.self_attn.indexer*",
+      "model.layers.42.self_attn*",
+      "model.layers.42.self_attn.indexer*",
+      "model.layers.43.self_attn*",
+      "model.layers.43.self_attn.indexer*",
+      "model.layers.44.self_attn*",
+      "model.layers.44.self_attn.indexer*",
+      "model.layers.45.self_attn*",
+      "model.layers.45.self_attn.indexer*",
+      "model.layers.46.self_attn*",
+      "model.layers.46.self_attn.indexer*",
+      "model.layers.47.self_attn*",
+      "model.layers.47.self_attn.indexer*",
+      "model.layers.48.self_attn*",
+      "model.layers.48.self_attn.indexer*",
+      "model.layers.49.self_attn*",
+      "model.layers.49.self_attn.indexer*",
+      "model.layers.5.self_attn*",
+      "model.layers.5.self_attn.indexer*",
+      "model.layers.50.self_attn*",
+      "model.layers.50.self_attn.indexer*",
+      "model.layers.51.self_attn*",
+      "model.layers.51.self_attn.indexer*",
+      "model.layers.52.self_attn*",
+      "model.layers.52.self_attn.indexer*",
+      "model.layers.53.self_attn*",
+      "model.layers.53.self_attn.indexer*",
+      "model.layers.54.self_attn*",
+      "model.layers.54.self_attn.indexer*",
+      "model.layers.55.self_attn*",
+      "model.layers.55.self_attn.indexer*",
+      "model.layers.56.self_attn*",
+      "model.layers.56.self_attn.indexer*",
+      "model.layers.57.self_attn*",
+      "model.layers.57.self_attn.indexer*",
+      "model.layers.58.self_attn*",
+      "model.layers.58.self_attn.indexer*",
+      "model.layers.59.self_attn*",
+      "model.layers.59.self_attn.indexer*",
+      "model.layers.6.self_attn*",
+      "model.layers.6.self_attn.indexer*",
+      "model.layers.60.self_attn*",
+      "model.layers.60.self_attn.indexer*",
+      "model.layers.61.self_attn*",
+      "model.layers.61.self_attn.indexer*",
+      "model.layers.62.self_attn*",
+      "model.layers.62.self_attn.indexer*",
+      "model.layers.63.self_attn*",
+      "model.layers.63.self_attn.indexer*",
+      "model.layers.64.self_attn*",
+      "model.layers.64.self_attn.indexer*",
+      "model.layers.65.self_attn*",
+      "model.layers.65.self_attn.indexer*",
+      "model.layers.66.self_attn*",
+      "model.layers.66.self_attn.indexer*",
+      "model.layers.67.self_attn*",
+      "model.layers.67.self_attn.indexer*",
+      "model.layers.68.self_attn*",
+      "model.layers.68.self_attn.indexer*",
+      "model.layers.69.self_attn*",
+      "model.layers.69.self_attn.indexer*",
+      "model.layers.7.self_attn*",
+      "model.layers.7.self_attn.indexer*",
+      "model.layers.70.self_attn*",
+      "model.layers.70.self_attn.indexer*",
+      "model.layers.71.self_attn*",
+      "model.layers.71.self_attn.indexer*",
+      "model.layers.72.self_attn*",
+      "model.layers.72.self_attn.indexer*",
+      "model.layers.73.self_attn*",
+      "model.layers.73.self_attn.indexer*",
+      "model.layers.74.self_attn*",
+      "model.layers.74.self_attn.indexer*",
+      "model.layers.75.self_attn*",
+      "model.layers.75.self_attn.indexer*",
+      "model.layers.76.self_attn*",
+      "model.layers.76.self_attn.indexer*",
+      "model.layers.77.self_attn*",
+      "model.layers.77.self_attn.indexer*",
+      "model.layers.8.self_attn*",
+      "model.layers.8.self_attn.indexer*",
+      "model.layers.9.self_attn*",
+      "model.layers.9.self_attn.indexer*",
+      "model.layers.78.eh_proj*",
+      "model.layers.78.self_attn*",
+      "model.layers.78.self_attn.indexer*",
+      "model.layers.78.mlp.gate*",
+      "model.layers.78.mlp.shared_experts*"
+    ],
+    "quant_algo": "NVFP4",
+    "producer": {
+      "name": "modelopt",
+      "version": "0.39.0.dev290+gf9d9a71de.d20260214"
+    },
+    "quant_method": "modelopt"
+  }
+}

filelist.txt

@@ -0,0 +1,72 @@
+chat_template.jinja
+config.json
+generation_config.json
+LICENSE
+model-00000.safetensors
+model-00001.safetensors
+model-00002.safetensors
+model-00003.safetensors
+model-00004.safetensors
+model-00005.safetensors
+model-00006.safetensors
+model-00007.safetensors
+model-00008.safetensors
+model-00009.safetensors
+model-00010.safetensors
+model-00011.safetensors
+model-00012.safetensors
+model-00013.safetensors
+model-00014.safetensors
+model-00015.safetensors
+model-00016.safetensors
+model-00017.safetensors
+model-00018.safetensors
+model-00019.safetensors
+model-00020.safetensors
+model-00021.safetensors
+model-00022.safetensors
+model-00023.safetensors
+model-00024.safetensors
+model-00025.safetensors
+model-00026.safetensors
+model-00027.safetensors
+model-00028.safetensors
+model-00029.safetensors
+model-00030.safetensors
+model-00031.safetensors
+model-00032.safetensors
+model-00033.safetensors
+model-00034.safetensors
+model-00035.safetensors
+model-00036.safetensors
+model-00037.safetensors
+model-00038.safetensors
+model-00039.safetensors
+model-00040.safetensors
+model-00041.safetensors
+model-00042.safetensors
+model-00043.safetensors
+model-00044.safetensors
+model-00045.safetensors
+model-00046.safetensors
+model-00047.safetensors
+model-00048.safetensors
+model-00049.safetensors
+model-00050.safetensors
+model-00051.safetensors
+model-00052.safetensors
+model-00053.safetensors
+model-00054.safetensors
+model-00055.safetensors
+model-00056.safetensors
+model-00057.safetensors
+model-00058.safetensors
+model-00059.safetensors
+model-00061.safetensors
+model-00062.safetensors
+model-00063.safetensors
+model.safetensors.index.json
+README.md
+REPORT.md
+tokenizer_config.json
+tokenizer.json

generation_config.json

@@ -0,0 +1,12 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": [
+    154820,
+    154827,
+    154829
+  ],
+  "pad_token_id": 154820,
+  "temperature": 1.0,
+  "top_p": 0.95,
+  "transformers_version": "5.12.0"
+}

model-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bdb653f84e8c4e16b9f3f151c7a2e1e8561443b7b30b6bf568b99c3c81c59df1
+size 5001422828

model-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c2250aa4dfe11e4d5d7068110604c0befce614d6319b223896b62bd65636198
+size 5166417984

model-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89eed13db315ea4b747b5d87d74eac85f58fffdf192cf1a564e1000c86f8fdb9
+size 5000910108

model-00009.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50026982d64f5aa2a774d5a597a12a287f88fc4c217f88c37d677148490f5a76
+size 5000241880

model-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e1d13802df07a7066834e24567b743d4e817bedb9e8a15e79f23758a83199036
+size 5001422812

model-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e27db0193332cade0223985f2efc39e18a30b4a631fe78597d02fcb48936855b
+size 5005594616

model-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:22af3b7229ebdbb7c4397acfd08e8340508fc0ae0d4b031a7727f3f60c3671f2
+size 5006010772

model-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d46c796254386f7f88caf1ac66f17b64e6d86bde5752d59c72ed8ea67aaf786a
+size 5001422900

model-00020.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a3f377919c90e7232112a4bba42d95ad61f801aa72552a0723d2ee0e131602d
+size 5002972352

model-00022.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02f0bd86e4b17577de16305070f69d8b0f1dd518f9055ab5ff374b0e6afcde7b
+size 5001422900

model-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:03eb7ef542c85706e296f8ee4ddc1e8f2141e3d2f9caed5108d898b670792ff8
+size 5001422860

model-00024.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8457c486f67da96a74826f449453d9927695c0cb329c0a78ba9ff0af8b6d1d76
+size 5109791912

model-00025.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39996fd80fe50bf232657c0a60d2863de9c036ba932d1ac665f163bb8733ac5d
+size 5000909684

model-00026.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:464fce91985b881cb646987f7c1326d79f621465208bbe0a65846969771021dc
+size 5001421884

model-00027.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7a9d6e123a1e7606b5f953efb60a5f887bcdeb8730a07ec17fecc61635dd152
+size 5001422788

model-00028.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d5cc9bce564aa6d5568aec0cf23f19e6ae0b62456216a7f2c06e626470427227
+size 5003105788

model-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7d1b9c2d359e028d3ac6b3c7491013838ac15966056c0bfc6d1fa730a57a2145
+size 5001422620

model-00030.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:15f8e07390617aec1228997be7c2ccaf5216ac5667625e1090af3909cab3e8b7
+size 5001422988

model-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88b4fda277e591a0fe111cfd7f290b122a1ec71c1c205e21db8bc789297c8ea9
+size 5003105764

model-00034.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e9a13b89e919886838eb00cc2284ad47edb0d4dc815b8f1b8c31923263598b7
+size 5001422060

model-00035.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f3397c003df9d83c4539aad137578593755e8d65a8d1efd781400a55eb37ee50
+size 5001422148

model-00037.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab9636004cd18f0dbc7cf4b4581f45082391ed3b378705e7b12143d4752abebe
+size 5001422748

model-00038.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0e7971ab68c4939a582f01d1f3c28833470335174a330d4daa36483df9a208a
+size 5001422940

model-00039.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b1499eb70d73be75f1aa58d41fea7e85b14bb2be108fc84495e6d8c24d78d344
+size 5006010668

model-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c85b7e382ef5ce8219490ec99e7f0c5c1c13bb6f1a2afdb7b2cdaf39169a5fcd
+size 5005596096

model-00041.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:21fda0f47338618fc712492fadc13f705aab31c62e22a5c185d8ba3e8799b0ed
+size 5001422884

model-00042.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fac772500878fb290f117dcd5105f5bb347ba7a3f6f65e0e5a604f34dcda7f05
+size 5006010756

model-00047.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c0194e4d54c87f7d5310a0cdbc1f41b336d6b72bfa7ce0608eb62c198782c38d
+size 5001422748

model-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c17b70da119a20bfa97ba70df6c6659b469555064af5f3a3508c49bd634cd88c
+size 5006010708

model-00050.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e0950b75e22a6ac144c9e2283e01c6d58d4f3c0d2337ba176cc0392c7b69605
+size 5001422948

model-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d005bfc7b4fccbb74f90dc259d6cefa53d4b66c4615f6a85c5a904e0a8ec72ba
+size 5001925072

model-00052.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3608cea3d93b5bb37b9d659d9e5bc7d433e8828e8a3aebc312e79fd33a214bfb
+size 5005091788

model-00053.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88e455ff611448511f84cb1c2381449c8e7cf200db2ae46b4d4bffadbb76ae00
+size 5006010748

model-00055.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b5b9937406891a0a3532dd0389769893d5e116946d5dc1d11304f05b936f59a
+size 5112282228

model-00056.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:855015f3fcd52385ad8541a43f049c23284e38bcecf30d05e01e8fce546b48c4
+size 5005498068

model-00057.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0268c3c6a7cc8c401e047a71e561f4fea01e8e760ae99a932c4aadeb8ba4a6cd
+size 5001422940

model-00058.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a398dd7866838963771387ffa26d3ed72e546819b513305fb4b29501bf59468b
+size 5001422804

model-00059.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c97c6064f25b3d35bd64e0cc1505e072ed50650125489b494d29ca85e8e74539
+size 1002980748

model-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d287024d255b45c03af1be471fd5335af7e608ab6e83fb7cd87c3c9df5fb2847
+size 4575950984

model-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:821378aa6125e34fcaa4fac78a1b1adb24a5397c3a0f3754e11e2f204c69ab52
+size 5008224104

model.safetensors.index.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb5b4fa9782aea5ffc66f9145d6e630f1045d385c30437c531bebe422c075f3f
+size 12706968

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19e773648cb4e65de8660ea6365e10acca112d42a854923df93db4a6f333a82d
+size 20217442

tokenizer_config.json

@@ -0,0 +1,33 @@
+{
+  "backend": "tokenizers",
+  "clean_up_tokenization_spaces": false,
+  "do_lower_case": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": [
+    "<|endoftext|>",
+    "[MASK]",
+    "[gMASK]",
+    "[sMASK]",
+    "<sop>",
+    "<eop>",
+    "<|system|>",
+    "<|user|>",
+    "<|assistant|>",
+    "<|observation|>",
+    "<|begin_of_image|>",
+    "<|end_of_image|>",
+    "<|begin_of_video|>",
+    "<|end_of_video|>",
+    "<|begin_of_audio|>",
+    "<|end_of_audio|>",
+    "<|begin_of_transcription|>",
+    "<|end_of_transcription|>"
+  ],
+  "is_local": true,
+  "model_max_length": 1048576,
+  "model_specific_special_tokens": {},
+  "pad_token": "<|endoftext|>",
+  "padding_side": "left",
+  "remove_space": false,
+  "tokenizer_class": "TokenizersBackend"
+}