Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,163 @@
+---
+license: other
+license_name: openmdw-1.1
+license_link: https://openmdw.ai/license/1-1/
+base_model: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
+base_model_relation: quantized
+library_name: mlx
+pipeline_tag: text-generation
+tags:
+  - mlx
+  - vmlx-swift
+  - osaurus
+  - jang
+  - nemotron
+  - nemotron-3.5
+  - moe
+  - mamba2
+  - hybrid
+language:
+  - en
+  - es
+  - fr
+  - de
+  - it
+  - ja
+---
+
+<p align="center"><a href="https://osaurus.ai"><img src="./osaurus-x-banner.png" alt="Osaurus AI"></a></p>
+
+# OsaurusAI/Nemotron-3.5-Lightning-30B-A3B-JANG_6M
+
+Near-lossless. 8-bit attention/routers/shared experts, 6-bit everything else.
+
+JANG mixed-precision affine (JANG_6M, group size 64) conversion of [`nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16) for Apple silicon (MLX).
+
+NVIDIA Nemotron 3.5 Lightning is a 30B-total / 3B-active hybrid model: 52 layers of
+**Mamba-2 + MoE + attention** (23 Mamba, 23 MoE, 6 full-attention), 128 routed experts
+with 6 active plus 1 shared expert, and a 256K context window. Only 6 of 52 layers hold
+a KV cache, which is what makes long context affordable on a laptop.
+
+## Bundles in this series
+
+| Bundle | Avg bits | Size | Notes |
+|---|---|---|---|
+| [`Nemotron-3.5-Lightning-30B-A3B-JANG_2L`](https://huggingface.co/OsaurusAI/Nemotron-3.5-Lightning-30B-A3B-JANG_2L) | 3.73 | 16.22 GiB | smallest |
+| [`Nemotron-3.5-Lightning-30B-A3B-JANG_4M`](https://huggingface.co/OsaurusAI/Nemotron-3.5-Lightning-30B-A3B-JANG_4M) | 4.12 | 17.72 GiB | **recommended** |
+| [`Nemotron-3.5-Lightning-30B-A3B-JANG_6M`](https://huggingface.co/OsaurusAI/Nemotron-3.5-Lightning-30B-A3B-JANG_6M) | 6.06 | 25.15 GiB | near-lossless |
+| [`Nemotron-3.5-Lightning-30B-A3B-MXFP8`](https://huggingface.co/OsaurusAI/Nemotron-3.5-Lightning-30B-A3B-MXFP8) | 8.00 | 31.62 GiB | MX FP8 reference |
+
+## Verified local row
+
+The exact directory uploaded to this repository was loaded and generated on an
+M5 Max MacBook on 2026-08-11. Greedy decoding, prompts rendered through this
+bundle's own `chat_template.jinja`.
+
+- Model class: `nemotron_h`
+- Weight shards: 49
+- Weight bytes: 27001832920 (25.147 GiB)
+- Average bits/weight: 6.06
+- Load time: 2.5 seconds
+- Reasoning ON: 101.2 tok/s
+- Reasoning OFF: 41.4 tok/s
+- Stop behaviour: normal stop on every turn (no length stops)
+- Visible output: coherent on every turn
+- Tool calling: emitted a well-formed `<tool_call>` block
+
+Probes: a factual question with reasoning on, the same with reasoning off, and a
+single-function tool-calling request.
+
+## Sampling
+
+NVIDIA's recommended settings, stamped into **both** `generation_config.json` and
+`jang_config.json` so servers and loaders agree:
+
+| Parameter | Value |
+|---|---|
+| `temperature` | 1.0 |
+| `top_p` | 0.95 |
+| `top_k` | 0 (disabled) |
+| `repetition_penalty` | 1.0 (none) |
+| `eos_token_id` | `[2, 11]` |
+
+`top_k` is unspecified upstream and is stamped as `0` so downstream servers do not
+apply their own default. Both EOS ids matter: `11` (`<|im_end|>`) is the chat
+terminator and `2` (`</s>`) is the legacy one. Do not prepend BOS —
+`add_bos_token` is `false`.
+
+## Reasoning
+
+Reasoning is **on by default**, matching upstream. There are exactly two states —
+there are no low/medium/high effort tiers.
+
+```python
+# Reasoning ON (default)
+prompt = tokenizer.apply_chat_template(msgs, add_generation_prompt=True,
+                                       tokenize=False, enable_thinking=True)
+# Reasoning OFF
+prompt = tokenizer.apply_chat_template(msgs, add_generation_prompt=True,
+                                       tokenize=False, enable_thinking=False)
+```
+
+The chat template opens the reasoning rail itself: with reasoning on the prompt ends
+in `<think>\n`, and with reasoning off it ends in a **prefilled empty** `<think></think>`.
+Reasoning-off is therefore not "omit the block" — a runtime that simply drops `<think>`
+will produce a model that opens one and never closes it.
+
+The template also defaults `truncate_history_thinking=True`, so reasoning traces in
+earlier assistant turns are dropped when a new user turn arrives.
+
+## Tool calling
+
+Tools are rendered into the system prompt and calls come back as XML, not JSON:
+
+```
+<tool_call>
+<function=get_weather>
+<parameter=city>
+Santa Clara
+</parameter>
+</function>
+</tool_call>
+```
+
+Tool results are passed back with `role: "tool"`; consecutive results are coalesced
+into a single user turn wrapped in `<tool_response>`.
+
+## Modality
+
+**Text only.** Verified against the weights, not the config: the checkpoint contains
+no vision, audio, or video tensors. This is not the Omni or Audex line.
+
+## Usage
+
+```python
+from mlx_lm import load, generate
+from mlx_lm.sample_utils import make_sampler
+
+model, tokenizer = load("OsaurusAI/Nemotron-3.5-Lightning-30B-A3B-JANG_6M")
+prompt = tokenizer.apply_chat_template(
+    [{"role": "user", "content": "Explain speculative decoding in two sentences."}],
+    add_generation_prompt=True, tokenize=False, enable_thinking=True,
+)
+print(generate(model, tokenizer, prompt=prompt, max_tokens=512,
+               sampler=make_sampler(temp=1.0, top_p=0.95), verbose=True))
+```
+
+## Multi-Token Prediction
+
+The upstream checkpoint ships a native MTP head and it is **retained** in this bundle
+(`mtp.*`, DeepSeek-V3 shape, sharing the embeddings and output head). Current MLX
+runtimes do not decode with it, so it is inert — it costs disk only, not RAM. The
+bundle metadata reflects this honestly: `mtp.artifact_available = true`,
+`mtp.runtime_available = false`. Presence of these weights is not a claim of active
+speculative-decoding acceleration.
+
+## License
+
+Released under the [OpenMDW-1.1](https://openmdw.ai/license/1-1/) license of the base model. Please review the
+base model's terms before use.
+
+---
+
+Converted by Jinho Jang — <eric@osaurus.ai> — with [JANG](https://osaurus.ai).

chat_template.jinja

@@ -0,0 +1,190 @@
+{% macro render_extra_keys(json_dict, handled_keys) %}
+    {%- if json_dict is mapping %}
+        {%- for json_key in json_dict if json_key not in handled_keys %}
+            {%- if json_dict[json_key] is mapping or (json_dict[json_key] is sequence and json_dict[json_key] is not string) %}
+                {{- '\n<' ~ json_key ~ '>' ~ (json_dict[json_key] | tojson | safe) ~ '</' ~ json_key ~ '>' }}
+            {%- else %}
+                {{-'\n<' ~ json_key ~ '>' ~ (json_dict[json_key] | string) ~ '</' ~ json_key ~ '>' }}
+            {%- endif %}
+        {%- endfor %}
+    {%- endif %}
+{% endmacro %}
+{%- set enable_thinking = enable_thinking if enable_thinking is defined else True %}
+{%- set truncate_history_thinking = truncate_history_thinking if truncate_history_thinking is defined else True %}
+{%- set ns = namespace(last_user_idx = -1) %}
+{%- set loop_messages = messages %}
+{%- for m in loop_messages %}
+  {%- if m["role"] == "user" %}
+    {%- set ns.last_user_idx = loop.index0 %}
+  {%- endif %}
+{%- endfor %}
+{%- if messages[0]["role"] == "system" %}
+    {%- set system_message = messages[0]["content"] %}
+    {%- set loop_messages = messages[1:] %}
+{%- else %}
+    {%- set system_message = "" %}
+    {%- set loop_messages = messages %}
+{%- endif %}
+{%- if not tools is defined %}
+    {%- set tools = [] %}
+{%- endif %}
+{%- set ns = namespace(last_user_idx = -1) %}
+{%- for m in loop_messages %}
+  {%- if m["role"] == "user" %}
+    {%- set ns.last_user_idx = loop.index0 %}
+  {%- endif %}
+{%- endfor %}
+{%- if system_message is defined %}
+    {{- "<|im_start|>system\n" + system_message }}
+{%- else %}
+    {%- if tools is iterable and tools | length > 0 %}
+        {{- "<|im_start|>system\n" }}
+    {%- endif %}
+{%- endif %}
+{%- if tools is iterable and tools | length > 0 %}
+    {%- if system_message is defined and system_message | length > 0 %}
+        {{- "\n\n" }}
+    {%- endif %}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n" }}
+    {{- "<tools>" }}
+    {%- for tool in tools %}
+        {%- if tool.function is defined %}
+            {%- set tool = tool.function %}
+        {%- endif %}
+        {{- "\n<function>\n<name>" ~ tool.name ~ "</name>" }}
+        {%- if tool.description is defined %}
+            {{- '\n<description>' ~ (tool.description | trim) ~ '</description>' }}
+        {%- endif %}
+        {{- '\n<parameters>' }}
+        {%- if tool.parameters is defined and tool.parameters is mapping and tool.parameters.properties is defined and tool.parameters.properties is mapping %}
+            {%- for param_name, param_fields in tool.parameters.properties|items %}
+                {{- '\n<parameter>' }}
+                {{- '\n<name>' ~ param_name ~ '</name>' }}
+                {%- if param_fields.type is defined %}
+                    {{- '\n<type>' ~ (param_fields.type | string) ~ '</type>' }}
+                {%- endif %}
+                {%- if param_fields.description is defined %}
+                    {{- '\n<description>' ~ (param_fields.description | trim) ~ '</description>' }}
+                {%- endif %}
+                {%- if param_fields.enum is defined %}
+                    {{- '\n<enum>' ~ (param_fields.enum | tojson | safe) ~ '</enum>' }}
+                {%- endif %}
+                {%- set handled_keys = ['name', 'type', 'description', 'enum'] %}
+                {{- render_extra_keys(param_fields, handled_keys) }}
+                {{- '\n</parameter>' }}
+            {%- endfor %}
+        {%- endif %}
+        {% set handled_keys = ['type', 'properties', 'required'] %}
+        {{- render_extra_keys(tool.parameters, handled_keys) }}
+        {%- if tool.parameters is defined and tool.parameters.required is defined %}
+            {{- '\n<required>' ~ (tool.parameters.required | tojson | safe) ~ '</required>' }}
+        {%- endif %}
+        {{- '\n</parameters>' }}
+        {%- set handled_keys = ['type', 'name', 'description', 'parameters'] %}
+        {{- render_extra_keys(tool, handled_keys) }}
+        {{- '\n</function>' }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+{%- endif %}
+{%- if system_message is defined %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if tools is iterable and tools | length > 0 %}
+        {{- '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in loop_messages %}
+    {%- if message.role == "assistant" %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is string and message.reasoning_content | trim | length > 0 %}
+            {%- set content = "<think>\n" ~ message.reasoning_content ~ "</think>" ~ (message.content | default('', true)) %}
+        {%- else %}
+            {%- set content = message.content | default('', true) %}
+            {%- if content is string -%}
+                {%- if '<think>' not in content and '</think>' not in content -%}
+                    {%- set content = "<think></think>" ~ content -%}
+                {%- endif -%}
+            {%- else -%}
+                {%- set content = content -%}
+            {%- endif -%}
+        {%- endif %}
+        {%- if message.tool_calls is defined and message.tool_calls is iterable and message.tool_calls | length > 0 %}
+            {{- '<|im_start|>assistant\n' }}
+                {%- set include_content = not (truncate_history_thinking and loop.index0 < ns.last_user_idx) %}
+                {%- if content is string and content | trim | length > 0 %}
+                    {%- if include_content %}
+                        {{- (content | trim) ~ '\n' -}}
+                    {%- else %}
+                        {%- set c = (content | string) %}
+                        {%- if '</think>' in c %}
+                            {%- set c = c.split('</think>')[-1] %}
+                        {%- elif '<think>' in c %}
+                            {%- set c = c.split('<think>')[0] %}
+                        {%- endif %}
+                        {%- set c = "<think></think>" ~ c %}
+                        {%- if c | length > 0 %}
+                            {{- c ~ '\n' -}}
+                        {%- endif %}
+                    {%- endif %}
+                {%- else %}
+                    {{- "<think></think>" -}}
+                {%- endif %}
+                {%- for tool_call in message.tool_calls %}
+                    {%- if tool_call.function is defined %}
+                        {%- set tool_call = tool_call.function %}
+                    {%- endif %}
+                    {{- '<tool_call>\n<function=' ~ tool_call.name ~ '>\n' -}}
+                        {%- if tool_call.arguments is defined %}
+                            {%- for args_name, args_value in tool_call.arguments|items %}
+                                {{- '<parameter=' ~ args_name ~ '>\n' -}}
+                                    {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                                {{- args_value ~ '\n</parameter>\n' -}}
+                            {%- endfor %}
+                        {%- endif %}
+                    {{- '</function>\n</tool_call>\n' -}}
+                {%- endfor %}
+                {{- '<|im_end|>\n' }}
+        {%- else %}
+            {%- if not (truncate_history_thinking and loop.index0 < ns.last_user_idx) %}
+                {{- '<|im_start|>assistant\n' ~ (content | default('', true) | string | trim) ~ '<|im_end|>\n' }}
+            {%- else %}
+                {%- set c = (content | default('', true) | string) %}
+                {%- if '<think>' in c and '</think>' in c %}
+                    {%- set c = "<think></think>" ~ c.split('</think>')[-1] %}
+                {%- endif %}
+                {%- set c = c | trim %}
+                {%- if c | length > 0 %}
+                    {{- '<|im_start|>assistant\n' ~ c ~ '<|im_end|>\n' }}
+                {%- else %}
+                    {{- '<|im_start|>assistant\n<|im_end|>\n' }}
+                {%- endif %}
+            {%- endif %}
+        {%- endif %}
+    {%- elif message.role == "user" or message.role == "system" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- set content = message.content | string %}
+        {{- content }}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user\n' }}
+        {%- endif %}
+        {{- '<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>\n' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {%- if enable_thinking %}
+        {{- '<|im_start|>assistant\n<think>\n' }}
+    {%- else %}
+        {{- '<|im_start|>assistant\n<think></think>' }}
+    {%- endif %}
+{%- endif %}

config.json

@@ -0,0 +1,1053 @@
+{
+  "architectures": [
+    "NemotronHForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "chunk_size": 128,
+  "conv_kernel": 4,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "expand": 2,
+  "head_dim": 128,
+  "hidden_dropout": 0.0,
+  "hidden_size": 2688,
+  "initializer_range": 0.02,
+  "intermediate_size": 1856,
+  "layer_norm_epsilon": 1e-05,
+  "mamba_head_dim": 64,
+  "mamba_hidden_act": "silu",
+  "mamba_num_heads": 64,
+  "mamba_proj_bias": false,
+  "mamba_ssm_cache_dtype": "float32",
+  "max_position_embeddings": 262144,
+  "mlp_bias": false,
+  "mlp_hidden_act": "relu2",
+  "model_type": "nemotron_h",
+  "moe_intermediate_size": 1856,
+  "moe_latent_size": null,
+  "moe_shared_expert_intermediate_size": 3712,
+  "moe_shared_expert_overlap": true,
+  "n_group": 1,
+  "n_groups": 8,
+  "n_routed_experts": 128,
+  "n_shared_experts": 1,
+  "norm_eps": 1e-05,
+  "norm_topk_prob": true,
+  "num_attention_heads": 32,
+  "num_experts_per_tok": 6,
+  "num_hidden_layers": 52,
+  "num_key_value_heads": 2,
+  "num_logits_to_keep": 1,
+  "num_nextn_predict_layers": 1,
+  "pad_token_id": 0,
+  "partial_rotary_factor": 1.0,
+  "rescale_prenorm_residual": true,
+  "residual_in_fp32": false,
+  "rope_theta": 10000,
+  "routed_scaling_factor": 2.5,
+  "sliding_window": null,
+  "ssm_state_size": 128,
+  "tie_word_embeddings": false,
+  "time_step_floor": 0.0001,
+  "time_step_max": 0.1,
+  "time_step_min": 0.001,
+  "topk_group": 1,
+  "transformers_version": "4.57.6",
+  "use_bias": false,
+  "use_cache": true,
+  "use_conv_bias": true,
+  "use_mamba_kernels": true,
+  "vocab_size": 131072,
+  "layers_block_type": [
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe"
+  ],
+  "mtp_layers_block_type": [
+    "attention",
+    "moe"
+  ],
+  "quantization": {
+    "group_size": 64,
+    "bits": 6,
+    "mode": "affine",
+    "backbone.embeddings": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.0.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.0.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.1.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.1.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.1.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.1.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.2.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.2.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.3.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.3.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.3.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.3.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.4.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.4.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.5.mixer.k_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.5.mixer.o_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.5.mixer.q_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.5.mixer.v_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.6.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.6.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.6.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.6.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.7.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.7.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.10.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.10.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.10.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.10.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.11.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.11.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.12.mixer.k_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.12.mixer.o_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.12.mixer.q_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.12.mixer.v_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.8.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.8.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.8.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.8.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.9.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.9.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.13.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.13.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.13.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.13.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.14.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.14.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.15.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.15.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.15.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.15.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.16.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.16.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.17.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.17.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.17.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.17.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.18.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.18.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.19.mixer.k_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.19.mixer.o_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.19.mixer.q_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.19.mixer.v_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.20.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.20.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.20.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.20.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.21.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.21.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.22.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.22.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.22.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.22.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.23.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.23.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.24.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.24.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.24.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.24.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.25.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.25.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.26.mixer.k_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.26.mixer.o_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.26.mixer.q_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.26.mixer.v_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.27.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.27.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.27.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.27.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.28.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.28.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.29.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.29.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.29.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.29.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.30.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.30.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.31.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.31.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.31.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.31.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.32.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.32.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.33.mixer.k_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.33.mixer.o_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.33.mixer.q_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.33.mixer.v_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.34.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.34.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.34.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.34.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.35.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.35.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.36.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.36.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.36.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.36.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.37.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.37.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.38.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.38.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.38.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.38.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.39.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.39.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.40.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.40.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.40.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.40.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.41.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.41.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.42.mixer.k_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.42.mixer.o_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.42.mixer.q_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.42.mixer.v_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.43.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.43.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.43.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.43.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.44.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.44.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.45.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.45.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.45.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.45.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.46.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.46.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.47.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.47.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.47.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.47.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.48.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.48.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.49.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.49.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.49.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.49.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.50.mixer.in_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.50.mixer.out_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.51.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.51.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "backbone.layers.51.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "backbone.layers.51.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "lm_head": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "mtp.layers.0.eh_proj": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "mtp.layers.0.mixer.k_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "mtp.layers.0.mixer.o_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "mtp.layers.0.mixer.q_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "mtp.layers.0.mixer.v_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "mtp.layers.1.mixer.switch_mlp.fc2": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "mtp.layers.1.mixer.switch_mlp.fc1": {
+      "group_size": 64,
+      "bits": 6,
+      "mode": "affine"
+    },
+    "mtp.layers.1.mixer.shared_experts.down_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    },
+    "mtp.layers.1.mixer.shared_experts.up_proj": {
+      "group_size": 64,
+      "bits": 8,
+      "mode": "affine"
+    }
+  },
+  "capabilities": {
+    "reasoning_parser": "deepseek_r1",
+    "tool_parser": "nemotron",
+    "think_in_template": true,
+    "supports_tools": true,
+    "supports_thinking": true,
+    "family": "nemotron_h",
+    "modality": "text",
+    "modalities": {
+      "text": true,
+      "vision": false,
+      "audio": false,
+      "video": false
+    },
+    "has_vision": false,
+    "has_audio": false,
+    "has_video": false,
+    "cache_type": "hybrid",
+    "default_reasoning": "on"
+  },
+  "jang_config": {
+    "chat": {
+      "sampling_defaults": {
+        "temperature": 1.0,
+        "top_p": 0.95,
+        "top_k": 0,
+        "repetition_penalty": 1.0,
+        "source": "vendor_card+generation_config_2026-08-11"
+      },
+      "add_bos_token": false,
+      "stop_token_ids": [
+        2,
+        11
+      ]
+    },
+    "reasoning": {
+      "supported": true,
+      "parser": "deepseek_r1",
+      "default": "on",
+      "think_in_template": true,
+      "tiers": null,
+      "budget": "server_side",
+      "enable_kwarg": "enable_thinking",
+      "off_is_prefilled_empty_block": true,
+      "truncate_history_thinking_default": true
+    },
+    "tools": {
+      "supported": true,
+      "parser": "nemotron",
+      "dialect": "xml_function",
+      "tools_in_system_prompt": true,
+      "tool_results_role": "tool",
+      "force_nonempty_content_supported": false
+    },
+    "mtp": {
+      "artifact_available": true,
+      "runtime_available": false,
+      "num_nextn_predict_layers": 1,
+      "shape": "deepseek_v3_eh_proj",
+      "shares_embeddings_and_lm_head": true
+    }
+  }
+}

generation_config.json

@@ -0,0 +1,14 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "do_sample": true,
+  "eos_token_id": [
+    2,
+    11
+  ],
+  "pad_token_id": 0,
+  "top_p": 0.95,
+  "transformers_version": "4.57.6",
+  "temperature": 1.0,
+  "top_k": 0
+}

model-00001-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0f173cbf1e76f52d1ea8991be2b3169791e18423b8045f7aa50321ccbb0b703
+size 836560752

model-00002-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ec9ea6d5320cd75e4bf2cfcaa7e09c51071175f76b4d979e038d79e63eb7bf1
+size 529450776

model-00003-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f187c0fe016d439919002048f7f5786914860d10ca1a84c890d8d99d1363d4bf
+size 560901048

model-00004-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ca535b96ea4c8702fdeb3ac312cb5ae50152edd56edbf574d66c2c8ee8bd060
+size 529450776

model-00005-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:256902eacda91c0a984eac1c103da709de0078a6ef6d3462e4f5fcb369d49a88
+size 585760992

model-00006-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ed0f5ecfc2656e02c70e4dfd521968fc7f92daca3506ea5f23157dae488c6b91
+size 529450776

model-00008-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a951c8e1562eeb3dcdaf7da97b7da2063f8bea8149a3f23220e795eaf00f9752
+size 529450784

model-00009-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d5d316f329c75e12b458efd95f7d9b354b36b53a84da1b9a559ef73d2aebb39
+size 585761016

model-00010-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bdd96234a11f0af6a9a05234eb8be5e13956752fdae6de9376ab3e3aa77e69ef
+size 529450776

model-00012-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3f5704c266f9939a8cf45d3d8eeccbddb0e56209c01eb8fae7121e02b9fcb54c
+size 529450784

model-00013-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4b1ae3829ea88fe9572018d339233297a816ced03066e9dff61843ded92d4d5
+size 560901056

model-00015-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dfc57e8e18a39b2d5effb85a77c675d22548a1df8b85f8fbfdaee9c05216ac31
+size 560901056

model-00016-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b967ff58d75996550cb52aa75d3d52495f855037e810690df9ee3ba4d1eeaf0
+size 529450784

model-00017-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:792d61349d9ee4d90156fa5694f0dcb2435eaad33b6cbbf9c5c30eea47789d18
+size 585761016

model-00018-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7fa337240f5b356fe26cde1cd79be37f479e5736a8011d9adda9570938c13bcf
+size 529450784

model-00019-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da634f8054181e726ed1ab1c04716cf1f9d73607653f8341d7ee446c44f291f5
+size 560901056

model-00020-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d134061528b314ba922e9bcc2ccccffc09c3ef387877be1f512b71c8eb8bfe71
+size 529450784

model-00021-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac71b2771824a631e8fbb554b137cda96016047080bac36331d377876761ab21
+size 560901056

model-00022-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33a506d1466960e22b0b09e559bedde81190cc866f3560e55de2816cadf416d2
+size 529450784

model-00024-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e545668b970d14ba52c607d530dd0df0fb072d07ea0a30d7c9d4055ba0ce371
+size 529450784

model-00025-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25997a62b8cdf7e8b644c53b605480cc7673f721808a96a4efa3064090a3c3d9
+size 560901056

model-00027-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cab312ea2af1487be62a27731b18988b68685142191a0f8f0ed1d048318a74c7
+size 560901056

model-00028-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa758f1b8520603a78efddce99aeefa1b3c14be69fdf5bca95736ce613d46adf
+size 529450784

model-00032-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c189d118bcecae18d5125ed27e1c813950393dc2aee3ec98936d1f2fbf3c8d4
+size 529450784

model-00033-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b960e50059be0fdef5204972dcaa20c1552ef3f1d1b0de6e6d804901185c020
+size 560901056

model-00034-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd0c9c47e092fb8aa933f66c8cb7297ef01977f4f69ab0e4bf864c64350b072c
+size 529450784

model-00036-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4aced11f139c8ea478f7625cedb866c5f6504af711ecbae202799b1bea0696d9
+size 529450784

model-00037-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d1b937e60a406a2ce36b2852a21686d90cd5bd568e75c84fc8acf8c9cb85152
+size 585761016

model-00038-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9cf1391836c3d9b02387c8e84de3abfef2aab5c044aa7870a1d19ab522bf01d
+size 529450784

model-00039-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b254d00d83d71b788e8352cdbb30d923ad7270291f7bdf980b4f8c1fdd2d0eb5
+size 560901056

model-00040-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c425a955a74b92797b076c76b988bf82213e7a97f2d186efa3d60bb405926cd
+size 529450784

model-00041-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8541c03309497be67058b306ba4af5bdbc3fd3de408b11ef0d340e34fe6dffd
+size 560901056

model-00042-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72e7be0d783afe40b96b9f18f9d15ac788ba0e910b506449c2b0d3384fe1f8a7
+size 529450784

model-00043-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b96f8047f49234288be53846839d71bbde652b6e60ad2be00cf84e5d04dab73
+size 560901056

model-00044-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cddfc3608419b2e5d1df88619ec44ba805329fe1f535d309a09dd917167675d3
+size 529450784

model-00045-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7cdecd9a09ed5fac4e29989e0ca38b0b5aebb999acbf8188b64ac47fb767ec4
+size 560901056

model-00046-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93e0440687d4246fbbf2ac09f770979f14a64b0e43637d1b5a6efafdaee51069
+size 529450784

model-00047-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:082f93b7dccec5443c4c5c76acf3d25ab440e1f91d5c00d18a851da14c41be45
+size 940394040

model-00048-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:705a768701a5f7747a3cad7be4b372cdd31c5b74fba2ccf547a773a4709ccbe4
+size 529450744

model-00049-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f780665195e8bf9cf677fe62846840285efdfa3c9e80196bc0e028a4ec1d584
+size 29076424

model.safetensors.index.json

@@ -0,0 +1,772 @@
+{
+  "metadata": {
+    "format": "jang",
+    "format_version": "2.0",
+    "total_size": 27001832920
+  },
+  "weight_map": {
+    "backbone.embeddings.weight": "model-00001-of-00049.safetensors",
+    "backbone.embeddings.scales": "model-00001-of-00049.safetensors",
+    "backbone.embeddings.biases": "model-00001-of-00049.safetensors",
+    "backbone.layers.0.mixer.in_proj.weight": "model-00001-of-00049.safetensors",
+    "backbone.layers.0.mixer.in_proj.scales": "model-00001-of-00049.safetensors",
+    "backbone.layers.0.mixer.in_proj.biases": "model-00001-of-00049.safetensors",
+    "backbone.layers.0.mixer.out_proj.weight": "model-00001-of-00049.safetensors",
+    "backbone.layers.0.mixer.out_proj.scales": "model-00001-of-00049.safetensors",
+    "backbone.layers.0.mixer.out_proj.biases": "model-00001-of-00049.safetensors",
+    "backbone.layers.1.mixer.switch_mlp.fc2.weight": "model-00001-of-00049.safetensors",
+    "backbone.layers.1.mixer.switch_mlp.fc2.scales": "model-00001-of-00049.safetensors",
+    "backbone.layers.1.mixer.switch_mlp.fc2.biases": "model-00001-of-00049.safetensors",
+    "backbone.layers.1.mixer.switch_mlp.fc1.weight": "model-00002-of-00049.safetensors",
+    "backbone.layers.1.mixer.switch_mlp.fc1.scales": "model-00002-of-00049.safetensors",
+    "backbone.layers.1.mixer.switch_mlp.fc1.biases": "model-00002-of-00049.safetensors",
+    "backbone.layers.1.mixer.shared_experts.down_proj.weight": "model-00002-of-00049.safetensors",
+    "backbone.layers.1.mixer.shared_experts.down_proj.scales": "model-00002-of-00049.safetensors",
+    "backbone.layers.1.mixer.shared_experts.down_proj.biases": "model-00002-of-00049.safetensors",
+    "backbone.layers.1.mixer.shared_experts.up_proj.weight": "model-00003-of-00049.safetensors",
+    "backbone.layers.1.mixer.shared_experts.up_proj.scales": "model-00003-of-00049.safetensors",
+    "backbone.layers.1.mixer.shared_experts.up_proj.biases": "model-00003-of-00049.safetensors",
+    "backbone.layers.2.mixer.in_proj.weight": "model-00003-of-00049.safetensors",
+    "backbone.layers.2.mixer.in_proj.scales": "model-00003-of-00049.safetensors",
+    "backbone.layers.2.mixer.in_proj.biases": "model-00003-of-00049.safetensors",
+    "backbone.layers.2.mixer.out_proj.weight": "model-00003-of-00049.safetensors",
+    "backbone.layers.2.mixer.out_proj.scales": "model-00003-of-00049.safetensors",
+    "backbone.layers.2.mixer.out_proj.biases": "model-00003-of-00049.safetensors",
+    "backbone.layers.3.mixer.switch_mlp.fc2.weight": "model-00003-of-00049.safetensors",
+    "backbone.layers.3.mixer.switch_mlp.fc2.scales": "model-00003-of-00049.safetensors",
+    "backbone.layers.3.mixer.switch_mlp.fc2.biases": "model-00003-of-00049.safetensors",
+    "backbone.layers.3.mixer.switch_mlp.fc1.weight": "model-00004-of-00049.safetensors",
+    "backbone.layers.3.mixer.switch_mlp.fc1.scales": "model-00004-of-00049.safetensors",
+    "backbone.layers.3.mixer.switch_mlp.fc1.biases": "model-00004-of-00049.safetensors",
+    "backbone.layers.3.mixer.shared_experts.down_proj.weight": "model-00004-of-00049.safetensors",
+    "backbone.layers.3.mixer.shared_experts.down_proj.scales": "model-00004-of-00049.safetensors",
+    "backbone.layers.3.mixer.shared_experts.down_proj.biases": "model-00004-of-00049.safetensors",
+    "backbone.layers.3.mixer.shared_experts.up_proj.weight": "model-00005-of-00049.safetensors",
+    "backbone.layers.3.mixer.shared_experts.up_proj.scales": "model-00005-of-00049.safetensors",
+    "backbone.layers.3.mixer.shared_experts.up_proj.biases": "model-00005-of-00049.safetensors",
+    "backbone.layers.4.mixer.in_proj.weight": "model-00005-of-00049.safetensors",
+    "backbone.layers.4.mixer.in_proj.scales": "model-00005-of-00049.safetensors",
+    "backbone.layers.4.mixer.in_proj.biases": "model-00005-of-00049.safetensors",
+    "backbone.layers.4.mixer.out_proj.weight": "model-00005-of-00049.safetensors",
+    "backbone.layers.4.mixer.out_proj.scales": "model-00005-of-00049.safetensors",
+    "backbone.layers.4.mixer.out_proj.biases": "model-00005-of-00049.safetensors",
+    "backbone.layers.5.mixer.k_proj.weight": "model-00005-of-00049.safetensors",
+    "backbone.layers.5.mixer.k_proj.scales": "model-00005-of-00049.safetensors",
+    "backbone.layers.5.mixer.k_proj.biases": "model-00005-of-00049.safetensors",
+    "backbone.layers.5.mixer.o_proj.weight": "model-00005-of-00049.safetensors",
+    "backbone.layers.5.mixer.o_proj.scales": "model-00005-of-00049.safetensors",
+    "backbone.layers.5.mixer.o_proj.biases": "model-00005-of-00049.safetensors",
+    "backbone.layers.5.mixer.q_proj.weight": "model-00005-of-00049.safetensors",
+    "backbone.layers.5.mixer.q_proj.scales": "model-00005-of-00049.safetensors",
+    "backbone.layers.5.mixer.q_proj.biases": "model-00005-of-00049.safetensors",
+    "backbone.layers.5.mixer.v_proj.weight": "model-00005-of-00049.safetensors",
+    "backbone.layers.5.mixer.v_proj.scales": "model-00005-of-00049.safetensors",
+    "backbone.layers.5.mixer.v_proj.biases": "model-00005-of-00049.safetensors",
+    "backbone.layers.6.mixer.switch_mlp.fc2.weight": "model-00005-of-00049.safetensors",
+    "backbone.layers.6.mixer.switch_mlp.fc2.scales": "model-00005-of-00049.safetensors",
+    "backbone.layers.6.mixer.switch_mlp.fc2.biases": "model-00005-of-00049.safetensors",
+    "backbone.layers.6.mixer.switch_mlp.fc1.weight": "model-00006-of-00049.safetensors",
+    "backbone.layers.6.mixer.switch_mlp.fc1.scales": "model-00006-of-00049.safetensors",
+    "backbone.layers.6.mixer.switch_mlp.fc1.biases": "model-00006-of-00049.safetensors",
+    "backbone.layers.6.mixer.shared_experts.down_proj.weight": "model-00006-of-00049.safetensors",
+    "backbone.layers.6.mixer.shared_experts.down_proj.scales": "model-00006-of-00049.safetensors",
+    "backbone.layers.6.mixer.shared_experts.down_proj.biases": "model-00006-of-00049.safetensors",
+    "backbone.layers.6.mixer.shared_experts.up_proj.weight": "model-00007-of-00049.safetensors",
+    "backbone.layers.6.mixer.shared_experts.up_proj.scales": "model-00007-of-00049.safetensors",
+    "backbone.layers.6.mixer.shared_experts.up_proj.biases": "model-00007-of-00049.safetensors",
+    "backbone.layers.7.mixer.in_proj.weight": "model-00007-of-00049.safetensors",
+    "backbone.layers.7.mixer.in_proj.scales": "model-00007-of-00049.safetensors",
+    "backbone.layers.7.mixer.in_proj.biases": "model-00007-of-00049.safetensors",
+    "backbone.layers.7.mixer.out_proj.weight": "model-00007-of-00049.safetensors",
+    "backbone.layers.7.mixer.out_proj.scales": "model-00007-of-00049.safetensors",
+    "backbone.layers.7.mixer.out_proj.biases": "model-00007-of-00049.safetensors",
+    "backbone.layers.10.mixer.switch_mlp.fc2.weight": "model-00007-of-00049.safetensors",
+    "backbone.layers.10.mixer.switch_mlp.fc2.scales": "model-00007-of-00049.safetensors",
+    "backbone.layers.10.mixer.switch_mlp.fc2.biases": "model-00007-of-00049.safetensors",
+    "backbone.layers.10.mixer.switch_mlp.fc1.weight": "model-00008-of-00049.safetensors",
+    "backbone.layers.10.mixer.switch_mlp.fc1.scales": "model-00008-of-00049.safetensors",
+    "backbone.layers.10.mixer.switch_mlp.fc1.biases": "model-00008-of-00049.safetensors",
+    "backbone.layers.10.mixer.shared_experts.down_proj.weight": "model-00008-of-00049.safetensors",
+    "backbone.layers.10.mixer.shared_experts.down_proj.scales": "model-00008-of-00049.safetensors",
+    "backbone.layers.10.mixer.shared_experts.down_proj.biases": "model-00008-of-00049.safetensors",
+    "backbone.layers.10.mixer.shared_experts.up_proj.weight": "model-00009-of-00049.safetensors",
+    "backbone.layers.10.mixer.shared_experts.up_proj.scales": "model-00009-of-00049.safetensors",
+    "backbone.layers.10.mixer.shared_experts.up_proj.biases": "model-00009-of-00049.safetensors",
+    "backbone.layers.11.mixer.in_proj.weight": "model-00009-of-00049.safetensors",
+    "backbone.layers.11.mixer.in_proj.scales": "model-00009-of-00049.safetensors",
+    "backbone.layers.11.mixer.in_proj.biases": "model-00009-of-00049.safetensors",
+    "backbone.layers.11.mixer.out_proj.weight": "model-00009-of-00049.safetensors",
+    "backbone.layers.11.mixer.out_proj.scales": "model-00009-of-00049.safetensors",
+    "backbone.layers.11.mixer.out_proj.biases": "model-00009-of-00049.safetensors",
+    "backbone.layers.12.mixer.k_proj.weight": "model-00009-of-00049.safetensors",
+    "backbone.layers.12.mixer.k_proj.scales": "model-00009-of-00049.safetensors",
+    "backbone.layers.12.mixer.k_proj.biases": "model-00009-of-00049.safetensors",
+    "backbone.layers.12.mixer.o_proj.weight": "model-00009-of-00049.safetensors",
+    "backbone.layers.12.mixer.o_proj.scales": "model-00009-of-00049.safetensors",
+    "backbone.layers.12.mixer.o_proj.biases": "model-00009-of-00049.safetensors",
+    "backbone.layers.12.mixer.q_proj.weight": "model-00009-of-00049.safetensors",
+    "backbone.layers.12.mixer.q_proj.scales": "model-00009-of-00049.safetensors",
+    "backbone.layers.12.mixer.q_proj.biases": "model-00009-of-00049.safetensors",
+    "backbone.layers.12.mixer.v_proj.weight": "model-00009-of-00049.safetensors",
+    "backbone.layers.12.mixer.v_proj.scales": "model-00009-of-00049.safetensors",
+    "backbone.layers.12.mixer.v_proj.biases": "model-00009-of-00049.safetensors",
+    "backbone.layers.8.mixer.switch_mlp.fc2.weight": "model-00009-of-00049.safetensors",
+    "backbone.layers.8.mixer.switch_mlp.fc2.scales": "model-00009-of-00049.safetensors",
+    "backbone.layers.8.mixer.switch_mlp.fc2.biases": "model-00009-of-00049.safetensors",
+    "backbone.layers.8.mixer.switch_mlp.fc1.weight": "model-00010-of-00049.safetensors",
+    "backbone.layers.8.mixer.switch_mlp.fc1.scales": "model-00010-of-00049.safetensors",
+    "backbone.layers.8.mixer.switch_mlp.fc1.biases": "model-00010-of-00049.safetensors",
+    "backbone.layers.8.mixer.shared_experts.down_proj.weight": "model-00010-of-00049.safetensors",
+    "backbone.layers.8.mixer.shared_experts.down_proj.scales": "model-00010-of-00049.safetensors",
+    "backbone.layers.8.mixer.shared_experts.down_proj.biases": "model-00010-of-00049.safetensors",
+    "backbone.layers.8.mixer.shared_experts.up_proj.weight": "model-00011-of-00049.safetensors",
+    "backbone.layers.8.mixer.shared_experts.up_proj.scales": "model-00011-of-00049.safetensors",
+    "backbone.layers.8.mixer.shared_experts.up_proj.biases": "model-00011-of-00049.safetensors",
+    "backbone.layers.9.mixer.in_proj.weight": "model-00011-of-00049.safetensors",
+    "backbone.layers.9.mixer.in_proj.scales": "model-00011-of-00049.safetensors",
+    "backbone.layers.9.mixer.in_proj.biases": "model-00011-of-00049.safetensors",
+    "backbone.layers.9.mixer.out_proj.weight": "model-00011-of-00049.safetensors",
+    "backbone.layers.9.mixer.out_proj.scales": "model-00011-of-00049.safetensors",
+    "backbone.layers.9.mixer.out_proj.biases": "model-00011-of-00049.safetensors",
+    "backbone.layers.13.mixer.switch_mlp.fc2.weight": "model-00011-of-00049.safetensors",
+    "backbone.layers.13.mixer.switch_mlp.fc2.scales": "model-00011-of-00049.safetensors",
+    "backbone.layers.13.mixer.switch_mlp.fc2.biases": "model-00011-of-00049.safetensors",
+    "backbone.layers.13.mixer.switch_mlp.fc1.weight": "model-00012-of-00049.safetensors",
+    "backbone.layers.13.mixer.switch_mlp.fc1.scales": "model-00012-of-00049.safetensors",
+    "backbone.layers.13.mixer.switch_mlp.fc1.biases": "model-00012-of-00049.safetensors",
+    "backbone.layers.13.mixer.shared_experts.down_proj.weight": "model-00012-of-00049.safetensors",
+    "backbone.layers.13.mixer.shared_experts.down_proj.scales": "model-00012-of-00049.safetensors",
+    "backbone.layers.13.mixer.shared_experts.down_proj.biases": "model-00012-of-00049.safetensors",
+    "backbone.layers.13.mixer.shared_experts.up_proj.weight": "model-00013-of-00049.safetensors",
+    "backbone.layers.13.mixer.shared_experts.up_proj.scales": "model-00013-of-00049.safetensors",
+    "backbone.layers.13.mixer.shared_experts.up_proj.biases": "model-00013-of-00049.safetensors",
+    "backbone.layers.14.mixer.in_proj.weight": "model-00013-of-00049.safetensors",
+    "backbone.layers.14.mixer.in_proj.scales": "model-00013-of-00049.safetensors",
+    "backbone.layers.14.mixer.in_proj.biases": "model-00013-of-00049.safetensors",
+    "backbone.layers.14.mixer.out_proj.weight": "model-00013-of-00049.safetensors",
+    "backbone.layers.14.mixer.out_proj.scales": "model-00013-of-00049.safetensors",
+    "backbone.layers.14.mixer.out_proj.biases": "model-00013-of-00049.safetensors",
+    "backbone.layers.15.mixer.switch_mlp.fc2.weight": "model-00013-of-00049.safetensors",
+    "backbone.layers.15.mixer.switch_mlp.fc2.scales": "model-00013-of-00049.safetensors",
+    "backbone.layers.15.mixer.switch_mlp.fc2.biases": "model-00013-of-00049.safetensors",
+    "backbone.layers.15.mixer.switch_mlp.fc1.weight": "model-00014-of-00049.safetensors",
+    "backbone.layers.15.mixer.switch_mlp.fc1.scales": "model-00014-of-00049.safetensors",
+    "backbone.layers.15.mixer.switch_mlp.fc1.biases": "model-00014-of-00049.safetensors",
+    "backbone.layers.15.mixer.shared_experts.down_proj.weight": "model-00014-of-00049.safetensors",
+    "backbone.layers.15.mixer.shared_experts.down_proj.scales": "model-00014-of-00049.safetensors",
+    "backbone.layers.15.mixer.shared_experts.down_proj.biases": "model-00014-of-00049.safetensors",
+    "backbone.layers.15.mixer.shared_experts.up_proj.weight": "model-00015-of-00049.safetensors",
+    "backbone.layers.15.mixer.shared_experts.up_proj.scales": "model-00015-of-00049.safetensors",
+    "backbone.layers.15.mixer.shared_experts.up_proj.biases": "model-00015-of-00049.safetensors",
+    "backbone.layers.16.mixer.in_proj.weight": "model-00015-of-00049.safetensors",
+    "backbone.layers.16.mixer.in_proj.scales": "model-00015-of-00049.safetensors",
+    "backbone.layers.16.mixer.in_proj.biases": "model-00015-of-00049.safetensors",
+    "backbone.layers.16.mixer.out_proj.weight": "model-00015-of-00049.safetensors",
+    "backbone.layers.16.mixer.out_proj.scales": "model-00015-of-00049.safetensors",
+    "backbone.layers.16.mixer.out_proj.biases": "model-00015-of-00049.safetensors",
+    "backbone.layers.17.mixer.switch_mlp.fc2.weight": "model-00015-of-00049.safetensors",
+    "backbone.layers.17.mixer.switch_mlp.fc2.scales": "model-00015-of-00049.safetensors",
+    "backbone.layers.17.mixer.switch_mlp.fc2.biases": "model-00015-of-00049.safetensors",
+    "backbone.layers.17.mixer.switch_mlp.fc1.weight": "model-00016-of-00049.safetensors",
+    "backbone.layers.17.mixer.switch_mlp.fc1.scales": "model-00016-of-00049.safetensors",
+    "backbone.layers.17.mixer.switch_mlp.fc1.biases": "model-00016-of-00049.safetensors",
+    "backbone.layers.17.mixer.shared_experts.down_proj.weight": "model-00016-of-00049.safetensors",
+    "backbone.layers.17.mixer.shared_experts.down_proj.scales": "model-00016-of-00049.safetensors",
+    "backbone.layers.17.mixer.shared_experts.down_proj.biases": "model-00016-of-00049.safetensors",
+    "backbone.layers.17.mixer.shared_experts.up_proj.weight": "model-00017-of-00049.safetensors",
+    "backbone.layers.17.mixer.shared_experts.up_proj.scales": "model-00017-of-00049.safetensors",
+    "backbone.layers.17.mixer.shared_experts.up_proj.biases": "model-00017-of-00049.safetensors",
+    "backbone.layers.18.mixer.in_proj.weight": "model-00017-of-00049.safetensors",
+    "backbone.layers.18.mixer.in_proj.scales": "model-00017-of-00049.safetensors",
+    "backbone.layers.18.mixer.in_proj.biases": "model-00017-of-00049.safetensors",
+    "backbone.layers.18.mixer.out_proj.weight": "model-00017-of-00049.safetensors",
+    "backbone.layers.18.mixer.out_proj.scales": "model-00017-of-00049.safetensors",
+    "backbone.layers.18.mixer.out_proj.biases": "model-00017-of-00049.safetensors",
+    "backbone.layers.19.mixer.k_proj.weight": "model-00017-of-00049.safetensors",
+    "backbone.layers.19.mixer.k_proj.scales": "model-00017-of-00049.safetensors",
+    "backbone.layers.19.mixer.k_proj.biases": "model-00017-of-00049.safetensors",
+    "backbone.layers.19.mixer.o_proj.weight": "model-00017-of-00049.safetensors",
+    "backbone.layers.19.mixer.o_proj.scales": "model-00017-of-00049.safetensors",
+    "backbone.layers.19.mixer.o_proj.biases": "model-00017-of-00049.safetensors",
+    "backbone.layers.19.mixer.q_proj.weight": "model-00017-of-00049.safetensors",
+    "backbone.layers.19.mixer.q_proj.scales": "model-00017-of-00049.safetensors",
+    "backbone.layers.19.mixer.q_proj.biases": "model-00017-of-00049.safetensors",
+    "backbone.layers.19.mixer.v_proj.weight": "model-00017-of-00049.safetensors",
+    "backbone.layers.19.mixer.v_proj.scales": "model-00017-of-00049.safetensors",
+    "backbone.layers.19.mixer.v_proj.biases": "model-00017-of-00049.safetensors",
+    "backbone.layers.20.mixer.switch_mlp.fc2.weight": "model-00017-of-00049.safetensors",
+    "backbone.layers.20.mixer.switch_mlp.fc2.scales": "model-00017-of-00049.safetensors",
+    "backbone.layers.20.mixer.switch_mlp.fc2.biases": "model-00017-of-00049.safetensors",
+    "backbone.layers.20.mixer.switch_mlp.fc1.weight": "model-00018-of-00049.safetensors",
+    "backbone.layers.20.mixer.switch_mlp.fc1.scales": "model-00018-of-00049.safetensors",
+    "backbone.layers.20.mixer.switch_mlp.fc1.biases": "model-00018-of-00049.safetensors",
+    "backbone.layers.20.mixer.shared_experts.down_proj.weight": "model-00018-of-00049.safetensors",
+    "backbone.layers.20.mixer.shared_experts.down_proj.scales": "model-00018-of-00049.safetensors",
+    "backbone.layers.20.mixer.shared_experts.down_proj.biases": "model-00018-of-00049.safetensors",
+    "backbone.layers.20.mixer.shared_experts.up_proj.weight": "model-00019-of-00049.safetensors",
+    "backbone.layers.20.mixer.shared_experts.up_proj.scales": "model-00019-of-00049.safetensors",
+    "backbone.layers.20.mixer.shared_experts.up_proj.biases": "model-00019-of-00049.safetensors",
+    "backbone.layers.21.mixer.in_proj.weight": "model-00019-of-00049.safetensors",
+    "backbone.layers.21.mixer.in_proj.scales": "model-00019-of-00049.safetensors",
+    "backbone.layers.21.mixer.in_proj.biases": "model-00019-of-00049.safetensors",
+    "backbone.layers.21.mixer.out_proj.weight": "model-00019-of-00049.safetensors",
+    "backbone.layers.21.mixer.out_proj.scales": "model-00019-of-00049.safetensors",
+    "backbone.layers.21.mixer.out_proj.biases": "model-00019-of-00049.safetensors",
+    "backbone.layers.22.mixer.switch_mlp.fc2.weight": "model-00019-of-00049.safetensors",
+    "backbone.layers.22.mixer.switch_mlp.fc2.scales": "model-00019-of-00049.safetensors",
+    "backbone.layers.22.mixer.switch_mlp.fc2.biases": "model-00019-of-00049.safetensors",
+    "backbone.layers.22.mixer.switch_mlp.fc1.weight": "model-00020-of-00049.safetensors",
+    "backbone.layers.22.mixer.switch_mlp.fc1.scales": "model-00020-of-00049.safetensors",
+    "backbone.layers.22.mixer.switch_mlp.fc1.biases": "model-00020-of-00049.safetensors",
+    "backbone.layers.22.mixer.shared_experts.down_proj.weight": "model-00020-of-00049.safetensors",
+    "backbone.layers.22.mixer.shared_experts.down_proj.scales": "model-00020-of-00049.safetensors",
+    "backbone.layers.22.mixer.shared_experts.down_proj.biases": "model-00020-of-00049.safetensors",
+    "backbone.layers.22.mixer.shared_experts.up_proj.weight": "model-00021-of-00049.safetensors",
+    "backbone.layers.22.mixer.shared_experts.up_proj.scales": "model-00021-of-00049.safetensors",
+    "backbone.layers.22.mixer.shared_experts.up_proj.biases": "model-00021-of-00049.safetensors",
+    "backbone.layers.23.mixer.in_proj.weight": "model-00021-of-00049.safetensors",
+    "backbone.layers.23.mixer.in_proj.scales": "model-00021-of-00049.safetensors",
+    "backbone.layers.23.mixer.in_proj.biases": "model-00021-of-00049.safetensors",
+    "backbone.layers.23.mixer.out_proj.weight": "model-00021-of-00049.safetensors",
+    "backbone.layers.23.mixer.out_proj.scales": "model-00021-of-00049.safetensors",
+    "backbone.layers.23.mixer.out_proj.biases": "model-00021-of-00049.safetensors",
+    "backbone.layers.24.mixer.switch_mlp.fc2.weight": "model-00021-of-00049.safetensors",
+    "backbone.layers.24.mixer.switch_mlp.fc2.scales": "model-00021-of-00049.safetensors",
+    "backbone.layers.24.mixer.switch_mlp.fc2.biases": "model-00021-of-00049.safetensors",
+    "backbone.layers.24.mixer.switch_mlp.fc1.weight": "model-00022-of-00049.safetensors",
+    "backbone.layers.24.mixer.switch_mlp.fc1.scales": "model-00022-of-00049.safetensors",
+    "backbone.layers.24.mixer.switch_mlp.fc1.biases": "model-00022-of-00049.safetensors",
+    "backbone.layers.24.mixer.shared_experts.down_proj.weight": "model-00022-of-00049.safetensors",
+    "backbone.layers.24.mixer.shared_experts.down_proj.scales": "model-00022-of-00049.safetensors",
+    "backbone.layers.24.mixer.shared_experts.down_proj.biases": "model-00022-of-00049.safetensors",
+    "backbone.layers.24.mixer.shared_experts.up_proj.weight": "model-00023-of-00049.safetensors",
+    "backbone.layers.24.mixer.shared_experts.up_proj.scales": "model-00023-of-00049.safetensors",
+    "backbone.layers.24.mixer.shared_experts.up_proj.biases": "model-00023-of-00049.safetensors",
+    "backbone.layers.25.mixer.in_proj.weight": "model-00023-of-00049.safetensors",
+    "backbone.layers.25.mixer.in_proj.scales": "model-00023-of-00049.safetensors",
+    "backbone.layers.25.mixer.in_proj.biases": "model-00023-of-00049.safetensors",
+    "backbone.layers.25.mixer.out_proj.weight": "model-00023-of-00049.safetensors",
+    "backbone.layers.25.mixer.out_proj.scales": "model-00023-of-00049.safetensors",
+    "backbone.layers.25.mixer.out_proj.biases": "model-00023-of-00049.safetensors",
+    "backbone.layers.26.mixer.k_proj.weight": "model-00023-of-00049.safetensors",
+    "backbone.layers.26.mixer.k_proj.scales": "model-00023-of-00049.safetensors",
+    "backbone.layers.26.mixer.k_proj.biases": "model-00023-of-00049.safetensors",
+    "backbone.layers.26.mixer.o_proj.weight": "model-00023-of-00049.safetensors",
+    "backbone.layers.26.mixer.o_proj.scales": "model-00023-of-00049.safetensors",
+    "backbone.layers.26.mixer.o_proj.biases": "model-00023-of-00049.safetensors",
+    "backbone.layers.26.mixer.q_proj.weight": "model-00023-of-00049.safetensors",
+    "backbone.layers.26.mixer.q_proj.scales": "model-00023-of-00049.safetensors",
+    "backbone.layers.26.mixer.q_proj.biases": "model-00023-of-00049.safetensors",
+    "backbone.layers.26.mixer.v_proj.weight": "model-00023-of-00049.safetensors",
+    "backbone.layers.26.mixer.v_proj.scales": "model-00023-of-00049.safetensors",
+    "backbone.layers.26.mixer.v_proj.biases": "model-00023-of-00049.safetensors",
+    "backbone.layers.27.mixer.switch_mlp.fc2.weight": "model-00023-of-00049.safetensors",
+    "backbone.layers.27.mixer.switch_mlp.fc2.scales": "model-00023-of-00049.safetensors",
+    "backbone.layers.27.mixer.switch_mlp.fc2.biases": "model-00023-of-00049.safetensors",
+    "backbone.layers.27.mixer.switch_mlp.fc1.weight": "model-00024-of-00049.safetensors",
+    "backbone.layers.27.mixer.switch_mlp.fc1.scales": "model-00024-of-00049.safetensors",
+    "backbone.layers.27.mixer.switch_mlp.fc1.biases": "model-00024-of-00049.safetensors",
+    "backbone.layers.27.mixer.shared_experts.down_proj.weight": "model-00024-of-00049.safetensors",
+    "backbone.layers.27.mixer.shared_experts.down_proj.scales": "model-00024-of-00049.safetensors",
+    "backbone.layers.27.mixer.shared_experts.down_proj.biases": "model-00024-of-00049.safetensors",
+    "backbone.layers.27.mixer.shared_experts.up_proj.weight": "model-00025-of-00049.safetensors",
+    "backbone.layers.27.mixer.shared_experts.up_proj.scales": "model-00025-of-00049.safetensors",
+    "backbone.layers.27.mixer.shared_experts.up_proj.biases": "model-00025-of-00049.safetensors",
+    "backbone.layers.28.mixer.in_proj.weight": "model-00025-of-00049.safetensors",
+    "backbone.layers.28.mixer.in_proj.scales": "model-00025-of-00049.safetensors",
+    "backbone.layers.28.mixer.in_proj.biases": "model-00025-of-00049.safetensors",
+    "backbone.layers.28.mixer.out_proj.weight": "model-00025-of-00049.safetensors",
+    "backbone.layers.28.mixer.out_proj.scales": "model-00025-of-00049.safetensors",
+    "backbone.layers.28.mixer.out_proj.biases": "model-00025-of-00049.safetensors",
+    "backbone.layers.29.mixer.switch_mlp.fc2.weight": "model-00025-of-00049.safetensors",
+    "backbone.layers.29.mixer.switch_mlp.fc2.scales": "model-00025-of-00049.safetensors",
+    "backbone.layers.29.mixer.switch_mlp.fc2.biases": "model-00025-of-00049.safetensors",
+    "backbone.layers.29.mixer.switch_mlp.fc1.weight": "model-00026-of-00049.safetensors",
+    "backbone.layers.29.mixer.switch_mlp.fc1.scales": "model-00026-of-00049.safetensors",
+    "backbone.layers.29.mixer.switch_mlp.fc1.biases": "model-00026-of-00049.safetensors",
+    "backbone.layers.29.mixer.shared_experts.down_proj.weight": "model-00026-of-00049.safetensors",
+    "backbone.layers.29.mixer.shared_experts.down_proj.scales": "model-00026-of-00049.safetensors",
+    "backbone.layers.29.mixer.shared_experts.down_proj.biases": "model-00026-of-00049.safetensors",
+    "backbone.layers.29.mixer.shared_experts.up_proj.weight": "model-00027-of-00049.safetensors",
+    "backbone.layers.29.mixer.shared_experts.up_proj.scales": "model-00027-of-00049.safetensors",
+    "backbone.layers.29.mixer.shared_experts.up_proj.biases": "model-00027-of-00049.safetensors",
+    "backbone.layers.30.mixer.in_proj.weight": "model-00027-of-00049.safetensors",
+    "backbone.layers.30.mixer.in_proj.scales": "model-00027-of-00049.safetensors",
+    "backbone.layers.30.mixer.in_proj.biases": "model-00027-of-00049.safetensors",
+    "backbone.layers.30.mixer.out_proj.weight": "model-00027-of-00049.safetensors",
+    "backbone.layers.30.mixer.out_proj.scales": "model-00027-of-00049.safetensors",
+    "backbone.layers.30.mixer.out_proj.biases": "model-00027-of-00049.safetensors",
+    "backbone.layers.31.mixer.switch_mlp.fc2.weight": "model-00027-of-00049.safetensors",
+    "backbone.layers.31.mixer.switch_mlp.fc2.scales": "model-00027-of-00049.safetensors",
+    "backbone.layers.31.mixer.switch_mlp.fc2.biases": "model-00027-of-00049.safetensors",
+    "backbone.layers.31.mixer.switch_mlp.fc1.weight": "model-00028-of-00049.safetensors",
+    "backbone.layers.31.mixer.switch_mlp.fc1.scales": "model-00028-of-00049.safetensors",
+    "backbone.layers.31.mixer.switch_mlp.fc1.biases": "model-00028-of-00049.safetensors",
+    "backbone.layers.31.mixer.shared_experts.down_proj.weight": "model-00028-of-00049.safetensors",
+    "backbone.layers.31.mixer.shared_experts.down_proj.scales": "model-00028-of-00049.safetensors",
+    "backbone.layers.31.mixer.shared_experts.down_proj.biases": "model-00028-of-00049.safetensors",
+    "backbone.layers.31.mixer.shared_experts.up_proj.weight": "model-00029-of-00049.safetensors",
+    "backbone.layers.31.mixer.shared_experts.up_proj.scales": "model-00029-of-00049.safetensors",
+    "backbone.layers.31.mixer.shared_experts.up_proj.biases": "model-00029-of-00049.safetensors",
+    "backbone.layers.32.mixer.in_proj.weight": "model-00029-of-00049.safetensors",
+    "backbone.layers.32.mixer.in_proj.scales": "model-00029-of-00049.safetensors",
+    "backbone.layers.32.mixer.in_proj.biases": "model-00029-of-00049.safetensors",
+    "backbone.layers.32.mixer.out_proj.weight": "model-00029-of-00049.safetensors",
+    "backbone.layers.32.mixer.out_proj.scales": "model-00029-of-00049.safetensors",
+    "backbone.layers.32.mixer.out_proj.biases": "model-00029-of-00049.safetensors",
+    "backbone.layers.33.mixer.k_proj.weight": "model-00029-of-00049.safetensors",
+    "backbone.layers.33.mixer.k_proj.scales": "model-00029-of-00049.safetensors",
+    "backbone.layers.33.mixer.k_proj.biases": "model-00029-of-00049.safetensors",
+    "backbone.layers.33.mixer.o_proj.weight": "model-00029-of-00049.safetensors",
+    "backbone.layers.33.mixer.o_proj.scales": "model-00029-of-00049.safetensors",
+    "backbone.layers.33.mixer.o_proj.biases": "model-00029-of-00049.safetensors",
+    "backbone.layers.33.mixer.q_proj.weight": "model-00029-of-00049.safetensors",
+    "backbone.layers.33.mixer.q_proj.scales": "model-00029-of-00049.safetensors",
+    "backbone.layers.33.mixer.q_proj.biases": "model-00029-of-00049.safetensors",
+    "backbone.layers.33.mixer.v_proj.weight": "model-00029-of-00049.safetensors",
+    "backbone.layers.33.mixer.v_proj.scales": "model-00029-of-00049.safetensors",
+    "backbone.layers.33.mixer.v_proj.biases": "model-00029-of-00049.safetensors",
+    "backbone.layers.34.mixer.switch_mlp.fc2.weight": "model-00029-of-00049.safetensors",
+    "backbone.layers.34.mixer.switch_mlp.fc2.scales": "model-00029-of-00049.safetensors",
+    "backbone.layers.34.mixer.switch_mlp.fc2.biases": "model-00029-of-00049.safetensors",
+    "backbone.layers.34.mixer.switch_mlp.fc1.weight": "model-00030-of-00049.safetensors",
+    "backbone.layers.34.mixer.switch_mlp.fc1.scales": "model-00030-of-00049.safetensors",
+    "backbone.layers.34.mixer.switch_mlp.fc1.biases": "model-00030-of-00049.safetensors",
+    "backbone.layers.34.mixer.shared_experts.down_proj.weight": "model-00030-of-00049.safetensors",
+    "backbone.layers.34.mixer.shared_experts.down_proj.scales": "model-00030-of-00049.safetensors",
+    "backbone.layers.34.mixer.shared_experts.down_proj.biases": "model-00030-of-00049.safetensors",
+    "backbone.layers.34.mixer.shared_experts.up_proj.weight": "model-00031-of-00049.safetensors",
+    "backbone.layers.34.mixer.shared_experts.up_proj.scales": "model-00031-of-00049.safetensors",
+    "backbone.layers.34.mixer.shared_experts.up_proj.biases": "model-00031-of-00049.safetensors",
+    "backbone.layers.35.mixer.in_proj.weight": "model-00031-of-00049.safetensors",
+    "backbone.layers.35.mixer.in_proj.scales": "model-00031-of-00049.safetensors",
+    "backbone.layers.35.mixer.in_proj.biases": "model-00031-of-00049.safetensors",
+    "backbone.layers.35.mixer.out_proj.weight": "model-00031-of-00049.safetensors",
+    "backbone.layers.35.mixer.out_proj.scales": "model-00031-of-00049.safetensors",
+    "backbone.layers.35.mixer.out_proj.biases": "model-00031-of-00049.safetensors",
+    "backbone.layers.36.mixer.switch_mlp.fc2.weight": "model-00031-of-00049.safetensors",
+    "backbone.layers.36.mixer.switch_mlp.fc2.scales": "model-00031-of-00049.safetensors",
+    "backbone.layers.36.mixer.switch_mlp.fc2.biases": "model-00031-of-00049.safetensors",
+    "backbone.layers.36.mixer.switch_mlp.fc1.weight": "model-00032-of-00049.safetensors",
+    "backbone.layers.36.mixer.switch_mlp.fc1.scales": "model-00032-of-00049.safetensors",
+    "backbone.layers.36.mixer.switch_mlp.fc1.biases": "model-00032-of-00049.safetensors",
+    "backbone.layers.36.mixer.shared_experts.down_proj.weight": "model-00032-of-00049.safetensors",
+    "backbone.layers.36.mixer.shared_experts.down_proj.scales": "model-00032-of-00049.safetensors",
+    "backbone.layers.36.mixer.shared_experts.down_proj.biases": "model-00032-of-00049.safetensors",
+    "backbone.layers.36.mixer.shared_experts.up_proj.weight": "model-00033-of-00049.safetensors",
+    "backbone.layers.36.mixer.shared_experts.up_proj.scales": "model-00033-of-00049.safetensors",
+    "backbone.layers.36.mixer.shared_experts.up_proj.biases": "model-00033-of-00049.safetensors",
+    "backbone.layers.37.mixer.in_proj.weight": "model-00033-of-00049.safetensors",
+    "backbone.layers.37.mixer.in_proj.scales": "model-00033-of-00049.safetensors",
+    "backbone.layers.37.mixer.in_proj.biases": "model-00033-of-00049.safetensors",
+    "backbone.layers.37.mixer.out_proj.weight": "model-00033-of-00049.safetensors",
+    "backbone.layers.37.mixer.out_proj.scales": "model-00033-of-00049.safetensors",
+    "backbone.layers.37.mixer.out_proj.biases": "model-00033-of-00049.safetensors",
+    "backbone.layers.38.mixer.switch_mlp.fc2.weight": "model-00033-of-00049.safetensors",
+    "backbone.layers.38.mixer.switch_mlp.fc2.scales": "model-00033-of-00049.safetensors",
+    "backbone.layers.38.mixer.switch_mlp.fc2.biases": "model-00033-of-00049.safetensors",
+    "backbone.layers.38.mixer.switch_mlp.fc1.weight": "model-00034-of-00049.safetensors",
+    "backbone.layers.38.mixer.switch_mlp.fc1.scales": "model-00034-of-00049.safetensors",
+    "backbone.layers.38.mixer.switch_mlp.fc1.biases": "model-00034-of-00049.safetensors",
+    "backbone.layers.38.mixer.shared_experts.down_proj.weight": "model-00034-of-00049.safetensors",
+    "backbone.layers.38.mixer.shared_experts.down_proj.scales": "model-00034-of-00049.safetensors",
+    "backbone.layers.38.mixer.shared_experts.down_proj.biases": "model-00034-of-00049.safetensors",
+    "backbone.layers.38.mixer.shared_experts.up_proj.weight": "model-00035-of-00049.safetensors",
+    "backbone.layers.38.mixer.shared_experts.up_proj.scales": "model-00035-of-00049.safetensors",
+    "backbone.layers.38.mixer.shared_experts.up_proj.biases": "model-00035-of-00049.safetensors",
+    "backbone.layers.39.mixer.in_proj.weight": "model-00035-of-00049.safetensors",
+    "backbone.layers.39.mixer.in_proj.scales": "model-00035-of-00049.safetensors",
+    "backbone.layers.39.mixer.in_proj.biases": "model-00035-of-00049.safetensors",
+    "backbone.layers.39.mixer.out_proj.weight": "model-00035-of-00049.safetensors",
+    "backbone.layers.39.mixer.out_proj.scales": "model-00035-of-00049.safetensors",
+    "backbone.layers.39.mixer.out_proj.biases": "model-00035-of-00049.safetensors",
+    "backbone.layers.40.mixer.switch_mlp.fc2.weight": "model-00035-of-00049.safetensors",
+    "backbone.layers.40.mixer.switch_mlp.fc2.scales": "model-00035-of-00049.safetensors",
+    "backbone.layers.40.mixer.switch_mlp.fc2.biases": "model-00035-of-00049.safetensors",
+    "backbone.layers.40.mixer.switch_mlp.fc1.weight": "model-00036-of-00049.safetensors",
+    "backbone.layers.40.mixer.switch_mlp.fc1.scales": "model-00036-of-00049.safetensors",
+    "backbone.layers.40.mixer.switch_mlp.fc1.biases": "model-00036-of-00049.safetensors",
+    "backbone.layers.40.mixer.shared_experts.down_proj.weight": "model-00036-of-00049.safetensors",
+    "backbone.layers.40.mixer.shared_experts.down_proj.scales": "model-00036-of-00049.safetensors",
+    "backbone.layers.40.mixer.shared_experts.down_proj.biases": "model-00036-of-00049.safetensors",
+    "backbone.layers.40.mixer.shared_experts.up_proj.weight": "model-00037-of-00049.safetensors",
+    "backbone.layers.40.mixer.shared_experts.up_proj.scales": "model-00037-of-00049.safetensors",
+    "backbone.layers.40.mixer.shared_experts.up_proj.biases": "model-00037-of-00049.safetensors",
+    "backbone.layers.41.mixer.in_proj.weight": "model-00037-of-00049.safetensors",
+    "backbone.layers.41.mixer.in_proj.scales": "model-00037-of-00049.safetensors",
+    "backbone.layers.41.mixer.in_proj.biases": "model-00037-of-00049.safetensors",
+    "backbone.layers.41.mixer.out_proj.weight": "model-00037-of-00049.safetensors",
+    "backbone.layers.41.mixer.out_proj.scales": "model-00037-of-00049.safetensors",
+    "backbone.layers.41.mixer.out_proj.biases": "model-00037-of-00049.safetensors",
+    "backbone.layers.42.mixer.k_proj.weight": "model-00037-of-00049.safetensors",
+    "backbone.layers.42.mixer.k_proj.scales": "model-00037-of-00049.safetensors",
+    "backbone.layers.42.mixer.k_proj.biases": "model-00037-of-00049.safetensors",
+    "backbone.layers.42.mixer.o_proj.weight": "model-00037-of-00049.safetensors",
+    "backbone.layers.42.mixer.o_proj.scales": "model-00037-of-00049.safetensors",
+    "backbone.layers.42.mixer.o_proj.biases": "model-00037-of-00049.safetensors",
+    "backbone.layers.42.mixer.q_proj.weight": "model-00037-of-00049.safetensors",
+    "backbone.layers.42.mixer.q_proj.scales": "model-00037-of-00049.safetensors",
+    "backbone.layers.42.mixer.q_proj.biases": "model-00037-of-00049.safetensors",
+    "backbone.layers.42.mixer.v_proj.weight": "model-00037-of-00049.safetensors",
+    "backbone.layers.42.mixer.v_proj.scales": "model-00037-of-00049.safetensors",
+    "backbone.layers.42.mixer.v_proj.biases": "model-00037-of-00049.safetensors",
+    "backbone.layers.43.mixer.switch_mlp.fc2.weight": "model-00037-of-00049.safetensors",
+    "backbone.layers.43.mixer.switch_mlp.fc2.scales": "model-00037-of-00049.safetensors",
+    "backbone.layers.43.mixer.switch_mlp.fc2.biases": "model-00037-of-00049.safetensors",
+    "backbone.layers.43.mixer.switch_mlp.fc1.weight": "model-00038-of-00049.safetensors",
+    "backbone.layers.43.mixer.switch_mlp.fc1.scales": "model-00038-of-00049.safetensors",
+    "backbone.layers.43.mixer.switch_mlp.fc1.biases": "model-00038-of-00049.safetensors",
+    "backbone.layers.43.mixer.shared_experts.down_proj.weight": "model-00038-of-00049.safetensors",
+    "backbone.layers.43.mixer.shared_experts.down_proj.scales": "model-00038-of-00049.safetensors",
+    "backbone.layers.43.mixer.shared_experts.down_proj.biases": "model-00038-of-00049.safetensors",
+    "backbone.layers.43.mixer.shared_experts.up_proj.weight": "model-00039-of-00049.safetensors",
+    "backbone.layers.43.mixer.shared_experts.up_proj.scales": "model-00039-of-00049.safetensors",
+    "backbone.layers.43.mixer.shared_experts.up_proj.biases": "model-00039-of-00049.safetensors",
+    "backbone.layers.44.mixer.in_proj.weight": "model-00039-of-00049.safetensors",
+    "backbone.layers.44.mixer.in_proj.scales": "model-00039-of-00049.safetensors",
+    "backbone.layers.44.mixer.in_proj.biases": "model-00039-of-00049.safetensors",
+    "backbone.layers.44.mixer.out_proj.weight": "model-00039-of-00049.safetensors",
+    "backbone.layers.44.mixer.out_proj.scales": "model-00039-of-00049.safetensors",
+    "backbone.layers.44.mixer.out_proj.biases": "model-00039-of-00049.safetensors",
+    "backbone.layers.45.mixer.switch_mlp.fc2.weight": "model-00039-of-00049.safetensors",
+    "backbone.layers.45.mixer.switch_mlp.fc2.scales": "model-00039-of-00049.safetensors",
+    "backbone.layers.45.mixer.switch_mlp.fc2.biases": "model-00039-of-00049.safetensors",
+    "backbone.layers.45.mixer.switch_mlp.fc1.weight": "model-00040-of-00049.safetensors",
+    "backbone.layers.45.mixer.switch_mlp.fc1.scales": "model-00040-of-00049.safetensors",
+    "backbone.layers.45.mixer.switch_mlp.fc1.biases": "model-00040-of-00049.safetensors",
+    "backbone.layers.45.mixer.shared_experts.down_proj.weight": "model-00040-of-00049.safetensors",
+    "backbone.layers.45.mixer.shared_experts.down_proj.scales": "model-00040-of-00049.safetensors",
+    "backbone.layers.45.mixer.shared_experts.down_proj.biases": "model-00040-of-00049.safetensors",
+    "backbone.layers.45.mixer.shared_experts.up_proj.weight": "model-00041-of-00049.safetensors",
+    "backbone.layers.45.mixer.shared_experts.up_proj.scales": "model-00041-of-00049.safetensors",
+    "backbone.layers.45.mixer.shared_experts.up_proj.biases": "model-00041-of-00049.safetensors",
+    "backbone.layers.46.mixer.in_proj.weight": "model-00041-of-00049.safetensors",
+    "backbone.layers.46.mixer.in_proj.scales": "model-00041-of-00049.safetensors",
+    "backbone.layers.46.mixer.in_proj.biases": "model-00041-of-00049.safetensors",
+    "backbone.layers.46.mixer.out_proj.weight": "model-00041-of-00049.safetensors",
+    "backbone.layers.46.mixer.out_proj.scales": "model-00041-of-00049.safetensors",
+    "backbone.layers.46.mixer.out_proj.biases": "model-00041-of-00049.safetensors",
+    "backbone.layers.47.mixer.switch_mlp.fc2.weight": "model-00041-of-00049.safetensors",
+    "backbone.layers.47.mixer.switch_mlp.fc2.scales": "model-00041-of-00049.safetensors",
+    "backbone.layers.47.mixer.switch_mlp.fc2.biases": "model-00041-of-00049.safetensors",
+    "backbone.layers.47.mixer.switch_mlp.fc1.weight": "model-00042-of-00049.safetensors",
+    "backbone.layers.47.mixer.switch_mlp.fc1.scales": "model-00042-of-00049.safetensors",
+    "backbone.layers.47.mixer.switch_mlp.fc1.biases": "model-00042-of-00049.safetensors",
+    "backbone.layers.47.mixer.shared_experts.down_proj.weight": "model-00042-of-00049.safetensors",
+    "backbone.layers.47.mixer.shared_experts.down_proj.scales": "model-00042-of-00049.safetensors",
+    "backbone.layers.47.mixer.shared_experts.down_proj.biases": "model-00042-of-00049.safetensors",
+    "backbone.layers.47.mixer.shared_experts.up_proj.weight": "model-00043-of-00049.safetensors",
+    "backbone.layers.47.mixer.shared_experts.up_proj.scales": "model-00043-of-00049.safetensors",
+    "backbone.layers.47.mixer.shared_experts.up_proj.biases": "model-00043-of-00049.safetensors",
+    "backbone.layers.48.mixer.in_proj.weight": "model-00043-of-00049.safetensors",
+    "backbone.layers.48.mixer.in_proj.scales": "model-00043-of-00049.safetensors",
+    "backbone.layers.48.mixer.in_proj.biases": "model-00043-of-00049.safetensors",
+    "backbone.layers.48.mixer.out_proj.weight": "model-00043-of-00049.safetensors",
+    "backbone.layers.48.mixer.out_proj.scales": "model-00043-of-00049.safetensors",
+    "backbone.layers.48.mixer.out_proj.biases": "model-00043-of-00049.safetensors",
+    "backbone.layers.49.mixer.switch_mlp.fc2.weight": "model-00043-of-00049.safetensors",
+    "backbone.layers.49.mixer.switch_mlp.fc2.scales": "model-00043-of-00049.safetensors",
+    "backbone.layers.49.mixer.switch_mlp.fc2.biases": "model-00043-of-00049.safetensors",
+    "backbone.layers.49.mixer.switch_mlp.fc1.weight": "model-00044-of-00049.safetensors",
+    "backbone.layers.49.mixer.switch_mlp.fc1.scales": "model-00044-of-00049.safetensors",
+    "backbone.layers.49.mixer.switch_mlp.fc1.biases": "model-00044-of-00049.safetensors",
+    "backbone.layers.49.mixer.shared_experts.down_proj.weight": "model-00044-of-00049.safetensors",
+    "backbone.layers.49.mixer.shared_experts.down_proj.scales": "model-00044-of-00049.safetensors",
+    "backbone.layers.49.mixer.shared_experts.down_proj.biases": "model-00044-of-00049.safetensors",
+    "backbone.layers.49.mixer.shared_experts.up_proj.weight": "model-00045-of-00049.safetensors",
+    "backbone.layers.49.mixer.shared_experts.up_proj.scales": "model-00045-of-00049.safetensors",
+    "backbone.layers.49.mixer.shared_experts.up_proj.biases": "model-00045-of-00049.safetensors",
+    "backbone.layers.50.mixer.in_proj.weight": "model-00045-of-00049.safetensors",
+    "backbone.layers.50.mixer.in_proj.scales": "model-00045-of-00049.safetensors",
+    "backbone.layers.50.mixer.in_proj.biases": "model-00045-of-00049.safetensors",
+    "backbone.layers.50.mixer.out_proj.weight": "model-00045-of-00049.safetensors",
+    "backbone.layers.50.mixer.out_proj.scales": "model-00045-of-00049.safetensors",
+    "backbone.layers.50.mixer.out_proj.biases": "model-00045-of-00049.safetensors",
+    "backbone.layers.51.mixer.switch_mlp.fc2.weight": "model-00045-of-00049.safetensors",
+    "backbone.layers.51.mixer.switch_mlp.fc2.scales": "model-00045-of-00049.safetensors",
+    "backbone.layers.51.mixer.switch_mlp.fc2.biases": "model-00045-of-00049.safetensors",
+    "backbone.layers.51.mixer.switch_mlp.fc1.weight": "model-00046-of-00049.safetensors",
+    "backbone.layers.51.mixer.switch_mlp.fc1.scales": "model-00046-of-00049.safetensors",
+    "backbone.layers.51.mixer.switch_mlp.fc1.biases": "model-00046-of-00049.safetensors",
+    "backbone.layers.51.mixer.shared_experts.down_proj.weight": "model-00046-of-00049.safetensors",
+    "backbone.layers.51.mixer.shared_experts.down_proj.scales": "model-00046-of-00049.safetensors",
+    "backbone.layers.51.mixer.shared_experts.down_proj.biases": "model-00046-of-00049.safetensors",
+    "backbone.layers.51.mixer.shared_experts.up_proj.weight": "model-00047-of-00049.safetensors",
+    "backbone.layers.51.mixer.shared_experts.up_proj.scales": "model-00047-of-00049.safetensors",
+    "backbone.layers.51.mixer.shared_experts.up_proj.biases": "model-00047-of-00049.safetensors",
+    "lm_head.weight": "model-00047-of-00049.safetensors",
+    "lm_head.scales": "model-00047-of-00049.safetensors",
+    "lm_head.biases": "model-00047-of-00049.safetensors",
+    "mtp.layers.0.eh_proj.weight": "model-00047-of-00049.safetensors",
+    "mtp.layers.0.eh_proj.scales": "model-00047-of-00049.safetensors",
+    "mtp.layers.0.eh_proj.biases": "model-00047-of-00049.safetensors",
+    "mtp.layers.0.mixer.k_proj.weight": "model-00047-of-00049.safetensors",
+    "mtp.layers.0.mixer.k_proj.scales": "model-00047-of-00049.safetensors",
+    "mtp.layers.0.mixer.k_proj.biases": "model-00047-of-00049.safetensors",
+    "mtp.layers.0.mixer.o_proj.weight": "model-00047-of-00049.safetensors",
+    "mtp.layers.0.mixer.o_proj.scales": "model-00047-of-00049.safetensors",
+    "mtp.layers.0.mixer.o_proj.biases": "model-00047-of-00049.safetensors",
+    "mtp.layers.0.mixer.q_proj.weight": "model-00047-of-00049.safetensors",
+    "mtp.layers.0.mixer.q_proj.scales": "model-00047-of-00049.safetensors",
+    "mtp.layers.0.mixer.q_proj.biases": "model-00047-of-00049.safetensors",
+    "mtp.layers.0.mixer.v_proj.weight": "model-00047-of-00049.safetensors",
+    "mtp.layers.0.mixer.v_proj.scales": "model-00047-of-00049.safetensors",
+    "mtp.layers.0.mixer.v_proj.biases": "model-00047-of-00049.safetensors",
+    "mtp.layers.1.mixer.switch_mlp.fc2.weight": "model-00047-of-00049.safetensors",
+    "mtp.layers.1.mixer.switch_mlp.fc2.scales": "model-00047-of-00049.safetensors",
+    "mtp.layers.1.mixer.switch_mlp.fc2.biases": "model-00047-of-00049.safetensors",
+    "mtp.layers.1.mixer.switch_mlp.fc1.weight": "model-00048-of-00049.safetensors",
+    "mtp.layers.1.mixer.switch_mlp.fc1.scales": "model-00048-of-00049.safetensors",
+    "mtp.layers.1.mixer.switch_mlp.fc1.biases": "model-00048-of-00049.safetensors",
+    "mtp.layers.1.mixer.shared_experts.down_proj.weight": "model-00048-of-00049.safetensors",
+    "mtp.layers.1.mixer.shared_experts.down_proj.scales": "model-00048-of-00049.safetensors",
+    "mtp.layers.1.mixer.shared_experts.down_proj.biases": "model-00048-of-00049.safetensors",
+    "mtp.layers.1.mixer.shared_experts.up_proj.weight": "model-00049-of-00049.safetensors",
+    "mtp.layers.1.mixer.shared_experts.up_proj.scales": "model-00049-of-00049.safetensors",
+    "mtp.layers.1.mixer.shared_experts.up_proj.biases": "model-00049-of-00049.safetensors",
+    "backbone.layers.1.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.1.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.3.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.3.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.6.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.6.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.10.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.10.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.8.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.8.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.13.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.13.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.15.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.15.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.17.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.17.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.20.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.20.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.22.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.22.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.24.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.24.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.27.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.27.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.29.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.29.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.31.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.31.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.34.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.34.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.36.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.36.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.38.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.38.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.40.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.40.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.43.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.43.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.45.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.45.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.47.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.47.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.49.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.49.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.51.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.51.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "mtp.layers.1.mixer.gate.weight": "model-00049-of-00049.safetensors",
+    "mtp.layers.1.mixer.gate.e_score_correction_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.0.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.0.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.0.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.0.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.0.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.0.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.0.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.1.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.2.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.2.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.2.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.2.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.2.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.2.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.2.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.3.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.4.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.4.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.4.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.4.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.4.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.4.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.4.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.5.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.6.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.7.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.7.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.7.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.7.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.7.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.7.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.7.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.8.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.10.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.11.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.11.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.11.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.11.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.11.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.11.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.11.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.12.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.13.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.9.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.9.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.9.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.9.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.9.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.9.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.9.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.14.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.14.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.14.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.14.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.14.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.14.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.14.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.15.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.16.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.16.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.16.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.16.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.16.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.16.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.16.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.17.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.18.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.18.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.18.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.18.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.18.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.18.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.18.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.19.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.20.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.21.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.21.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.21.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.21.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.21.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.21.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.21.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.22.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.23.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.23.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.23.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.23.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.23.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.23.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.23.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.24.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.25.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.25.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.25.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.25.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.25.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.25.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.25.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.26.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.27.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.28.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.28.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.28.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.28.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.28.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.28.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.28.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.29.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.30.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.30.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.30.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.30.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.30.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.30.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.30.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.31.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.32.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.32.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.32.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.32.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.32.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.32.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.32.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.33.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.34.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.35.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.35.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.35.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.35.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.35.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.35.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.35.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.36.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.37.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.37.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.37.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.37.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.37.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.37.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.37.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.38.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.39.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.39.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.39.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.39.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.39.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.39.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.39.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.40.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.41.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.41.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.41.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.41.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.41.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.41.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.41.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.42.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.43.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.44.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.44.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.44.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.44.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.44.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.44.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.44.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.45.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.46.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.46.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.46.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.46.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.46.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.46.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.46.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.47.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.48.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.48.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.48.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.48.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.48.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.48.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.48.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.49.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.50.mixer.A_log": "model-00049-of-00049.safetensors",
+    "backbone.layers.50.mixer.D": "model-00049-of-00049.safetensors",
+    "backbone.layers.50.mixer.dt_bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.50.mixer.conv1d.bias": "model-00049-of-00049.safetensors",
+    "backbone.layers.50.mixer.conv1d.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.50.mixer.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.50.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.layers.51.norm.weight": "model-00049-of-00049.safetensors",
+    "backbone.norm_f.weight": "model-00049-of-00049.safetensors",
+    "mtp.layers.0.enorm.weight": "model-00049-of-00049.safetensors",
+    "mtp.layers.0.hnorm.weight": "model-00049-of-00049.safetensors",
+    "mtp.layers.0.norm.weight": "model-00049-of-00049.safetensors",
+    "mtp.layers.1.final_layernorm.weight": "model-00049-of-00049.safetensors",
+    "mtp.layers.1.norm.weight": "model-00049-of-00049.safetensors"
+  }
+}

special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6021eb6847e682f89aa52d5eb6e8c7d902a23acfc8137e25211cf84828f1592
+size 17077485