Text Generation
MLX
Safetensors
mistral
apple-silicon
quantized
mixed-precision
axquant
axq
development
mistral-devstral
4bit
4-bit precision
v2
conversational
Instructions to use AutomatosX/AX-Devstral-Small-2505-MLX-AXQ-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use AutomatosX/AX-Devstral-Small-2505-MLX-AXQ-4bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("AutomatosX/AX-Devstral-Small-2505-MLX-AXQ-4bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use AutomatosX/AX-Devstral-Small-2505-MLX-AXQ-4bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "AutomatosX/AX-Devstral-Small-2505-MLX-AXQ-4bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "AutomatosX/AX-Devstral-Small-2505-MLX-AXQ-4bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AutomatosX/AX-Devstral-Small-2505-MLX-AXQ-4bit", "messages": [ {"role": "user", "content": "Hello"} ] }' - Atomic Chat
Promote audited AXQuant v2 artifact at stable repository name
Browse filesPreserve backward-compatible repository identity; v1 remains tagged legacy-pre-v2 and this exact revision is tagged v2 after verification.
- README.md +22 -22
- axquant_manifest.json +16 -21
- axquant_plan.json +5 -4
- axquant_quantizer_execution.json +2 -2
- axquant_runtime.json +3 -3
- model-00001-of-00003.safetensors +1 -1
- model-manifest.json +0 -0
README.md
CHANGED
|
@@ -15,6 +15,7 @@ tags:
|
|
| 15 |
- mistral-devstral
|
| 16 |
- 4bit
|
| 17 |
- 4-bit
|
|
|
|
| 18 |
---
|
| 19 |
|
| 20 |
# AX-Devstral-Small-2505-MLX-AXQ-4bit
|
|
@@ -26,17 +27,21 @@ the BF16 source model. The language path is quantized under AXQuant protection f
|
|
| 26 |
> artifact-integrity records, but it does not publish measured quality, long-context, kernel-speed,
|
| 27 |
> or MTP-speed evidence. Do not interpret the AXQ product label as a benchmark claim.
|
| 28 |
|
|
|
|
|
|
|
|
|
|
| 29 |
## Model details
|
| 30 |
|
| 31 |
| Property | Value |
|
| 32 |
| --- | --- |
|
| 33 |
-
| Base model | [mistralai/Devstral-Small-2505](https://huggingface.co/mistralai/Devstral-Small-2505) |
|
| 34 |
-
| Source revision | `
|
| 35 |
| Product family | `mistral-devstral` |
|
| 36 |
| Source architecture | `MistralForCausalLM` (dense); text path optimized |
|
| 37 |
| Main-model parameters | 23.57B logical parameters |
|
| 38 |
-
| Quantizer | AXQuant `1.
|
| 39 |
| Hub budget class | `4bit` |
|
|
|
|
| 40 |
| AXQuant base precision class | `4bit` |
|
| 41 |
| Planned storage-adjusted BPW | 4.9499 |
|
| 42 |
| Measured main-model BPW | 4.9500 |
|
|
@@ -44,8 +49,8 @@ the BF16 source model. The language path is quantized under AXQuant protection f
|
|
| 44 |
| Safetensors weight size | 14.59 GB |
|
| 45 |
| Approximate complete download | 14.60 GB |
|
| 46 |
| Configured maximum context | 131,072 tokens; practical limits depend on unified memory |
|
| 47 |
-
|
|
| 48 |
-
|
|
| 49 |
| MTP present | `False` |
|
| 50 |
| Vision sidecar present | `False` |
|
| 51 |
|
|
@@ -63,7 +68,7 @@ protected models.
|
|
| 63 |
| Sibling | Intended trade-off |
|
| 64 |
| --- | --- |
|
| 65 |
| [4bit sibling](https://huggingface.co/AutomatosX/AX-Devstral-Small-2505-MLX-AXQ-4bit) | Lower-storage AXQ budget; check its exact BPW |
|
| 66 |
-
| [6bit sibling](https://huggingface.co/AutomatosX/AX-Devstral-Small-2505-MLX-AXQ-6bit) | Higher average precision near
|
| 67 |
|
| 68 |
See the [AutomatosX MLX model catalog](https://huggingface.co/collections/AutomatosX/automatosx-mlx-model-catalog)
|
| 69 |
for related MLX and OptiQ alternatives.
|
|
@@ -94,19 +99,13 @@ metadata and optional sidecars (`vision.safetensors`, `mtp.safetensors`); this c
|
|
| 94 |
does not establish MTP acceleration or vision-language quality. The artifact records MLX
|
| 95 |
`0.32.0` and MLX-LM `0.31.3` from conversion.
|
| 96 |
|
| 97 |
-
##
|
| 98 |
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
```
|
| 105 |
-
|
| 106 |
-
AX Engine is the authority for the AXQ runtime contract.
|
| 107 |
-
This development package does not claim runtime speedups until identical-checkpoint benchmarks are
|
| 108 |
-
published. The artifact records AX Engine version `not recorded`. Native
|
| 109 |
-
`model-manifest.json` status: included as `model-manifest.json`.
|
| 110 |
|
| 111 |
## Quantization layout
|
| 112 |
|
|
@@ -133,7 +132,7 @@ establish MTP acceleration or vision-language quality.
|
|
| 133 |
| Planning evidence | `architecture_prior` |
|
| 134 |
| Calibration | none; the allocation is based on architecture priors |
|
| 135 |
| Quantizer execution | 281/281 recorded module conversions succeeded; 0 fallbacks |
|
| 136 |
-
| AX Engine native manifest | included
|
| 137 |
| Quality versus BF16 or uniform baselines | Not published; no quality-retention claim |
|
| 138 |
| MTP acceptance and speed | not measured; no MTP speedup claim |
|
| 139 |
| AX Engine kernel evidence | `unmeasured` |
|
|
@@ -149,6 +148,8 @@ establish MTP acceleration or vision-language quality.
|
|
| 149 |
- Architecture-prior allocation is not measured sensitivity. It must not be presented as measured
|
| 150 |
model quality.
|
| 151 |
- The configured context window can require substantially more memory as the KV cache grows.
|
|
|
|
|
|
|
| 152 |
- Upstream capabilities, limitations, biases, and responsible-use guidance still apply.
|
| 153 |
|
| 154 |
## Provenance and audit files
|
|
@@ -158,8 +159,7 @@ establish MTP acceleration or vision-language quality.
|
|
| 158 |
- [`axquant_plan.json`](axquant_plan.json): per-tensor precision decisions and planning evidence.
|
| 159 |
- [`axquant_quantizer_execution.json`](axquant_quantizer_execution.json): conversion coverage and
|
| 160 |
fallback records.
|
| 161 |
-
- [`axquant_runtime.json`](axquant_runtime.json): AX Engine and MLX-LM compatibility
|
| 162 |
-
- [`model-manifest.json`](model-manifest.json): AX Engine native tensor manifest.
|
| 163 |
|
| 164 |
All published provenance uses repository-relative paths. Local source paths are stripped before
|
| 165 |
publication. The checkpoint was converted from BF16 rather than re-quantized from an OptiQ
|
|
@@ -169,5 +169,5 @@ identical BPW or quality.
|
|
| 169 |
## License
|
| 170 |
|
| 171 |
The checkpoint follows the upstream model license where applicable (often Apache License 2.0). See
|
| 172 |
-
the [mistralai/Devstral-Small-2505 model card](https://huggingface.co/mistralai/Devstral-Small-2505) for license terms, model
|
| 173 |
limitations, and responsible-use guidance.
|
|
|
|
| 15 |
- mistral-devstral
|
| 16 |
- 4bit
|
| 17 |
- 4-bit
|
| 18 |
+
- v2
|
| 19 |
---
|
| 20 |
|
| 21 |
# AX-Devstral-Small-2505-MLX-AXQ-4bit
|
|
|
|
| 27 |
> artifact-integrity records, but it does not publish measured quality, long-context, kernel-speed,
|
| 28 |
> or MTP-speed evidence. Do not interpret the AXQ product label as a benchmark claim.
|
| 29 |
|
| 30 |
+
> **Stable-name v2.** `main` serves the audited v2 artifact for backward compatibility. The same revision is tagged `v2`; the replaced artifact remains recoverable at `legacy-pre-v2`.
|
| 31 |
+
|
| 32 |
+
|
| 33 |
## Model details
|
| 34 |
|
| 35 |
| Property | Value |
|
| 36 |
| --- | --- |
|
| 37 |
+
| Base model | [mistralai/Devstral-Small-2505](https://huggingface.co/mistralai/Devstral-Small-2505/tree/c2a9d81a2989af566682b4cecc828c84556076c5) |
|
| 38 |
+
| Source revision | `c2a9d81a2989af566682b4cecc828c84556076c5` |
|
| 39 |
| Product family | `mistral-devstral` |
|
| 40 |
| Source architecture | `MistralForCausalLM` (dense); text path optimized |
|
| 41 |
| Main-model parameters | 23.57B logical parameters |
|
| 42 |
+
| Quantizer | AXQuant `1.2.0` |
|
| 43 |
| Hub budget class | `4bit` |
|
| 44 |
+
| Artifact edition | `v2` |
|
| 45 |
| AXQuant base precision class | `4bit` |
|
| 46 |
| Planned storage-adjusted BPW | 4.9499 |
|
| 47 |
| Measured main-model BPW | 4.9500 |
|
|
|
|
| 49 |
| Safetensors weight size | 14.59 GB |
|
| 50 |
| Approximate complete download | 14.60 GB |
|
| 51 |
| Configured maximum context | 131,072 tokens; practical limits depend on unified memory |
|
| 52 |
+
| MLX-LM compatibility | Standard text inference, compatibility level B |
|
| 53 |
+
| AX Engine native execution | Not established; no validated native manifest is included |
|
| 54 |
| MTP present | `False` |
|
| 55 |
| Vision sidecar present | `False` |
|
| 56 |
|
|
|
|
| 68 |
| Sibling | Intended trade-off |
|
| 69 |
| --- | --- |
|
| 70 |
| [4bit sibling](https://huggingface.co/AutomatosX/AX-Devstral-Small-2505-MLX-AXQ-4bit) | Lower-storage AXQ budget; check its exact BPW |
|
| 71 |
+
| [6bit sibling](https://huggingface.co/AutomatosX/AX-Devstral-Small-2505-MLX-AXQ-6bit) | Higher average precision near the 6-BPW budget |
|
| 72 |
|
| 73 |
See the [AutomatosX MLX model catalog](https://huggingface.co/collections/AutomatosX/automatosx-mlx-model-catalog)
|
| 74 |
for related MLX and OptiQ alternatives.
|
|
|
|
| 99 |
does not establish MTP acceleration or vision-language quality. The artifact records MLX
|
| 100 |
`0.32.0` and MLX-LM `0.31.3` from conversion.
|
| 101 |
|
| 102 |
+
## AX Engine status
|
| 103 |
|
| 104 |
+
This package does **not** include a validated native `model-manifest.json`, so AX Engine execution
|
| 105 |
+
is not established by this release. The AX Engine fields in `axquant_runtime.json` describe the
|
| 106 |
+
intended compatibility contract, not observed runtime evidence. Use the MLX-LM path above for
|
| 107 |
+
standard text/backbone inference. The artifact records AX Engine version
|
| 108 |
+
`not recorded`, but version discovery alone is not a runtime check.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 109 |
|
| 110 |
## Quantization layout
|
| 111 |
|
|
|
|
| 132 |
| Planning evidence | `architecture_prior` |
|
| 133 |
| Calibration | none; the allocation is based on architecture priors |
|
| 134 |
| Quantizer execution | 281/281 recorded module conversions succeeded; 0 fallbacks |
|
| 135 |
+
| AX Engine native manifest | not included |
|
| 136 |
| Quality versus BF16 or uniform baselines | Not published; no quality-retention claim |
|
| 137 |
| MTP acceptance and speed | not measured; no MTP speedup claim |
|
| 138 |
| AX Engine kernel evidence | `unmeasured` |
|
|
|
|
| 148 |
- Architecture-prior allocation is not measured sensitivity. It must not be presented as measured
|
| 149 |
model quality.
|
| 150 |
- The configured context window can require substantially more memory as the KV cache grows.
|
| 151 |
+
- AX Engine execution is not established because this package has no validated native manifest.
|
| 152 |
+
|
| 153 |
- Upstream capabilities, limitations, biases, and responsible-use guidance still apply.
|
| 154 |
|
| 155 |
## Provenance and audit files
|
|
|
|
| 159 |
- [`axquant_plan.json`](axquant_plan.json): per-tensor precision decisions and planning evidence.
|
| 160 |
- [`axquant_quantizer_execution.json`](axquant_quantizer_execution.json): conversion coverage and
|
| 161 |
fallback records.
|
| 162 |
+
- [`axquant_runtime.json`](axquant_runtime.json): declared AX Engine and MLX-LM compatibility metadata; runtime checks remain separate evidence.
|
|
|
|
| 163 |
|
| 164 |
All published provenance uses repository-relative paths. Local source paths are stripped before
|
| 165 |
publication. The checkpoint was converted from BF16 rather than re-quantized from an OptiQ
|
|
|
|
| 169 |
## License
|
| 170 |
|
| 171 |
The checkpoint follows the upstream model license where applicable (often Apache License 2.0). See
|
| 172 |
+
the [mistralai/Devstral-Small-2505 model card](https://huggingface.co/mistralai/Devstral-Small-2505/tree/c2a9d81a2989af566682b4cecc828c84556076c5) for license terms, model
|
| 173 |
limitations, and responsible-use guidance.
|
axquant_manifest.json
CHANGED
|
@@ -1,28 +1,28 @@
|
|
| 1 |
{
|
| 2 |
-
"axquant_version": "1.
|
| 3 |
"calibration": null,
|
| 4 |
-
"created_at": "2026-08-
|
| 5 |
"effective_bpw": 4.949927508451917,
|
| 6 |
"files": [
|
| 7 |
{
|
| 8 |
"path": "README.md",
|
| 9 |
-
"sha256": "
|
| 10 |
-
"size_bytes":
|
| 11 |
},
|
| 12 |
{
|
| 13 |
"path": "axquant_plan.json",
|
| 14 |
-
"sha256": "
|
| 15 |
-
"size_bytes":
|
| 16 |
},
|
| 17 |
{
|
| 18 |
"path": "axquant_quantizer_execution.json",
|
| 19 |
-
"sha256": "
|
| 20 |
"size_bytes": 68529
|
| 21 |
},
|
| 22 |
{
|
| 23 |
"path": "axquant_runtime.json",
|
| 24 |
-
"sha256": "
|
| 25 |
-
"size_bytes":
|
| 26 |
},
|
| 27 |
{
|
| 28 |
"path": "chat_template.jinja",
|
|
@@ -41,7 +41,7 @@
|
|
| 41 |
},
|
| 42 |
{
|
| 43 |
"path": "model-00001-of-00003.safetensors",
|
| 44 |
-
"sha256": "
|
| 45 |
"size_bytes": 5308086419
|
| 46 |
},
|
| 47 |
{
|
|
@@ -54,11 +54,6 @@
|
|
| 54 |
"sha256": "9060d2f447d0203a0459a36cba4ef510e3c4eebb4cae92c8228129bc123407e5",
|
| 55 |
"size_bytes": 3962522975
|
| 56 |
},
|
| 57 |
-
{
|
| 58 |
-
"path": "model-manifest.json",
|
| 59 |
-
"sha256": "0ebe4e82e362ae95227b6d92490009672f6d8d6ca1ade22c588cdd9e3bb95f8f",
|
| 60 |
-
"size_bytes": 150358
|
| 61 |
-
},
|
| 62 |
{
|
| 63 |
"path": "model.safetensors.index.json",
|
| 64 |
"sha256": "49f2e5cc33c02b3040dbff542839e016af6fe9b0ba8aaeea67e31df919ac715a",
|
|
@@ -98,7 +93,7 @@
|
|
| 98 |
},
|
| 99 |
"mtp_present": false,
|
| 100 |
"mtp_weight_file_size_bytes": 0,
|
| 101 |
-
"plan_sha256": "
|
| 102 |
"profile": "general",
|
| 103 |
"protected_weight_file_size_bytes": 0,
|
| 104 |
"quantizer": "axquant",
|
|
@@ -114,7 +109,7 @@
|
|
| 114 |
{
|
| 115 |
"compatibility_level": "B",
|
| 116 |
"manifest": "config.json",
|
| 117 |
-
"mtp_support": "
|
| 118 |
"name": "mlx-lm",
|
| 119 |
"notes": [
|
| 120 |
"Standard backbone inference is the compatibility target.",
|
|
@@ -125,7 +120,7 @@
|
|
| 125 |
"support_level": "standard-inference"
|
| 126 |
}
|
| 127 |
],
|
| 128 |
-
"created_at": "2026-08-
|
| 129 |
"kv_cache": null,
|
| 130 |
"memory_policy": {
|
| 131 |
"kv_cache_precision": "runtime-default",
|
|
@@ -149,7 +144,7 @@
|
|
| 149 |
"primary_runtime": {
|
| 150 |
"compatibility_level": "A",
|
| 151 |
"manifest": "model-manifest.json",
|
| 152 |
-
"mtp_support": "
|
| 153 |
"name": "ax-engine",
|
| 154 |
"notes": [
|
| 155 |
"Runtime claims require a passing AX Engine doctor and benchmark report."
|
|
@@ -163,7 +158,7 @@
|
|
| 163 |
"schema_version": "axquant.artifact.v2",
|
| 164 |
"software_versions": {
|
| 165 |
"ax_engine": null,
|
| 166 |
-
"axquant": "1.
|
| 167 |
"mlx": "0.32.0",
|
| 168 |
"mlx_lm": "0.31.3",
|
| 169 |
"pydantic": "2.13.4",
|
|
@@ -175,7 +170,7 @@
|
|
| 175 |
"format": "mlx",
|
| 176 |
"local_path": null,
|
| 177 |
"model_id": "mistralai/Devstral-Small-2505",
|
| 178 |
-
"revision":
|
| 179 |
},
|
| 180 |
"target_class": "4bit",
|
| 181 |
"weight_distribution": {
|
|
|
|
| 1 |
{
|
| 2 |
+
"axquant_version": "1.2.0",
|
| 3 |
"calibration": null,
|
| 4 |
+
"created_at": "2026-08-05T08:23:55.481291Z",
|
| 5 |
"effective_bpw": 4.949927508451917,
|
| 6 |
"files": [
|
| 7 |
{
|
| 8 |
"path": "README.md",
|
| 9 |
+
"sha256": "f944a3bed85ad156433c73ce4227cc8dd7e5de07a21b281f132ff8b5bf5c0dca",
|
| 10 |
+
"size_bytes": 7796
|
| 11 |
},
|
| 12 |
{
|
| 13 |
"path": "axquant_plan.json",
|
| 14 |
+
"sha256": "6f90c1d59273dd8c482a7b14bccc90b8f7947e244ec71a572fd30ee71fe015c1",
|
| 15 |
+
"size_bytes": 337379
|
| 16 |
},
|
| 17 |
{
|
| 18 |
"path": "axquant_quantizer_execution.json",
|
| 19 |
+
"sha256": "226b42e3e7a08581a1c8128c52ed319d5263807fafe71b47e356f324933a8b61",
|
| 20 |
"size_bytes": 68529
|
| 21 |
},
|
| 22 |
{
|
| 23 |
"path": "axquant_runtime.json",
|
| 24 |
+
"sha256": "3c218a96a5a8d62d687f22d0a91985dc9164f5c8b15770573d0eaeb3e5811897",
|
| 25 |
+
"size_bytes": 1628
|
| 26 |
},
|
| 27 |
{
|
| 28 |
"path": "chat_template.jinja",
|
|
|
|
| 41 |
},
|
| 42 |
{
|
| 43 |
"path": "model-00001-of-00003.safetensors",
|
| 44 |
+
"sha256": "b6580465a51a48c0d4541aaaf3017ace61f17e7bee36c49a5e3eb5e53ce0bf27",
|
| 45 |
"size_bytes": 5308086419
|
| 46 |
},
|
| 47 |
{
|
|
|
|
| 54 |
"sha256": "9060d2f447d0203a0459a36cba4ef510e3c4eebb4cae92c8228129bc123407e5",
|
| 55 |
"size_bytes": 3962522975
|
| 56 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
{
|
| 58 |
"path": "model.safetensors.index.json",
|
| 59 |
"sha256": "49f2e5cc33c02b3040dbff542839e016af6fe9b0ba8aaeea67e31df919ac715a",
|
|
|
|
| 93 |
},
|
| 94 |
"mtp_present": false,
|
| 95 |
"mtp_weight_file_size_bytes": 0,
|
| 96 |
+
"plan_sha256": "7bd8665657133cce8bda462f969b954f466519dbff00600f596688ed5e84888f",
|
| 97 |
"profile": "general",
|
| 98 |
"protected_weight_file_size_bytes": 0,
|
| 99 |
"quantizer": "axquant",
|
|
|
|
| 109 |
{
|
| 110 |
"compatibility_level": "B",
|
| 111 |
"manifest": "config.json",
|
| 112 |
+
"mtp_support": "none",
|
| 113 |
"name": "mlx-lm",
|
| 114 |
"notes": [
|
| 115 |
"Standard backbone inference is the compatibility target.",
|
|
|
|
| 120 |
"support_level": "standard-inference"
|
| 121 |
}
|
| 122 |
],
|
| 123 |
+
"created_at": "2026-08-05T08:23:50.819054Z",
|
| 124 |
"kv_cache": null,
|
| 125 |
"memory_policy": {
|
| 126 |
"kv_cache_precision": "runtime-default",
|
|
|
|
| 144 |
"primary_runtime": {
|
| 145 |
"compatibility_level": "A",
|
| 146 |
"manifest": "model-manifest.json",
|
| 147 |
+
"mtp_support": "none",
|
| 148 |
"name": "ax-engine",
|
| 149 |
"notes": [
|
| 150 |
"Runtime claims require a passing AX Engine doctor and benchmark report."
|
|
|
|
| 158 |
"schema_version": "axquant.artifact.v2",
|
| 159 |
"software_versions": {
|
| 160 |
"ax_engine": null,
|
| 161 |
+
"axquant": "1.2.0",
|
| 162 |
"mlx": "0.32.0",
|
| 163 |
"mlx_lm": "0.31.3",
|
| 164 |
"pydantic": "2.13.4",
|
|
|
|
| 170 |
"format": "mlx",
|
| 171 |
"local_path": null,
|
| 172 |
"model_id": "mistralai/Devstral-Small-2505",
|
| 173 |
+
"revision": "c2a9d81a2989af566682b4cecc828c84556076c5"
|
| 174 |
},
|
| 175 |
"target_class": "4bit",
|
| 176 |
"weight_distribution": {
|
axquant_plan.json
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
{
|
| 2 |
-
"analysis_sha256": "
|
| 3 |
"architecture_profile": {
|
| 4 |
"adapter_id": "mistral-devstral-dense-v1",
|
| 5 |
"config_model_type": "mistral",
|
|
@@ -10564,7 +10564,7 @@
|
|
| 10564 |
"minimum_mtp_speedup": 1.2,
|
| 10565 |
"minimum_quality_retention": 0.98
|
| 10566 |
},
|
| 10567 |
-
"created_at": "2026-08-
|
| 10568 |
"effective_bpw": 4.949927508451917,
|
| 10569 |
"evidence_kind": "architecture_prior",
|
| 10570 |
"global_validation_required": true,
|
|
@@ -10589,6 +10589,7 @@
|
|
| 10589 |
"affine",
|
| 10590 |
"awq",
|
| 10591 |
"dwq",
|
|
|
|
| 10592 |
"bf16"
|
| 10593 |
]
|
| 10594 |
},
|
|
@@ -10626,7 +10627,7 @@
|
|
| 10626 |
"schema_version": "axquant.plan.v1",
|
| 10627 |
"software_versions": {
|
| 10628 |
"ax_engine": null,
|
| 10629 |
-
"axquant": "1.
|
| 10630 |
"mlx": "0.32.0",
|
| 10631 |
"mlx_lm": "0.31.3",
|
| 10632 |
"pydantic": "2.13.4",
|
|
@@ -10638,7 +10639,7 @@
|
|
| 10638 |
"format": "mlx",
|
| 10639 |
"local_path": null,
|
| 10640 |
"model_id": "mistralai/Devstral-Small-2505",
|
| 10641 |
-
"revision":
|
| 10642 |
},
|
| 10643 |
"status": "planned",
|
| 10644 |
"target_bpw": 4.95,
|
|
|
|
| 1 |
{
|
| 2 |
+
"analysis_sha256": "9c2c4dbc8887f3c2b65a95370cbef6855c3bef3054a0e0ea47e1380e279ad7bb",
|
| 3 |
"architecture_profile": {
|
| 4 |
"adapter_id": "mistral-devstral-dense-v1",
|
| 5 |
"config_model_type": "mistral",
|
|
|
|
| 10564 |
"minimum_mtp_speedup": 1.2,
|
| 10565 |
"minimum_quality_retention": 0.98
|
| 10566 |
},
|
| 10567 |
+
"created_at": "2026-08-05T08:22:48.557763Z",
|
| 10568 |
"effective_bpw": 4.949927508451917,
|
| 10569 |
"evidence_kind": "architecture_prior",
|
| 10570 |
"global_validation_required": true,
|
|
|
|
| 10589 |
"affine",
|
| 10590 |
"awq",
|
| 10591 |
"dwq",
|
| 10592 |
+
"gptq",
|
| 10593 |
"bf16"
|
| 10594 |
]
|
| 10595 |
},
|
|
|
|
| 10627 |
"schema_version": "axquant.plan.v1",
|
| 10628 |
"software_versions": {
|
| 10629 |
"ax_engine": null,
|
| 10630 |
+
"axquant": "1.2.0",
|
| 10631 |
"mlx": "0.32.0",
|
| 10632 |
"mlx_lm": "0.31.3",
|
| 10633 |
"pydantic": "2.13.4",
|
|
|
|
| 10639 |
"format": "mlx",
|
| 10640 |
"local_path": null,
|
| 10641 |
"model_id": "mistralai/Devstral-Small-2505",
|
| 10642 |
+
"revision": "c2a9d81a2989af566682b4cecc828c84556076c5"
|
| 10643 |
},
|
| 10644 |
"status": "planned",
|
| 10645 |
"target_bpw": 4.95,
|
axquant_quantizer_execution.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
{
|
| 2 |
-
"created_at": "2026-08-
|
| 3 |
-
"plan_sha256": "
|
| 4 |
"records": [
|
| 5 |
{
|
| 6 |
"bits": 8,
|
|
|
|
| 1 |
{
|
| 2 |
+
"created_at": "2026-08-05T08:23:50.814845Z",
|
| 3 |
+
"plan_sha256": "7bd8665657133cce8bda462f969b954f466519dbff00600f596688ed5e84888f",
|
| 4 |
"records": [
|
| 5 |
{
|
| 6 |
"bits": 8,
|
axquant_runtime.json
CHANGED
|
@@ -10,7 +10,7 @@
|
|
| 10 |
{
|
| 11 |
"compatibility_level": "B",
|
| 12 |
"manifest": "config.json",
|
| 13 |
-
"mtp_support": "
|
| 14 |
"name": "mlx-lm",
|
| 15 |
"notes": [
|
| 16 |
"Standard backbone inference is the compatibility target.",
|
|
@@ -21,7 +21,7 @@
|
|
| 21 |
"support_level": "standard-inference"
|
| 22 |
}
|
| 23 |
],
|
| 24 |
-
"created_at": "2026-08-
|
| 25 |
"kv_cache": null,
|
| 26 |
"memory_policy": {
|
| 27 |
"kv_cache_precision": "runtime-default",
|
|
@@ -45,7 +45,7 @@
|
|
| 45 |
"primary_runtime": {
|
| 46 |
"compatibility_level": "A",
|
| 47 |
"manifest": "model-manifest.json",
|
| 48 |
-
"mtp_support": "
|
| 49 |
"name": "ax-engine",
|
| 50 |
"notes": [
|
| 51 |
"Runtime claims require a passing AX Engine doctor and benchmark report."
|
|
|
|
| 10 |
{
|
| 11 |
"compatibility_level": "B",
|
| 12 |
"manifest": "config.json",
|
| 13 |
+
"mtp_support": "none",
|
| 14 |
"name": "mlx-lm",
|
| 15 |
"notes": [
|
| 16 |
"Standard backbone inference is the compatibility target.",
|
|
|
|
| 21 |
"support_level": "standard-inference"
|
| 22 |
}
|
| 23 |
],
|
| 24 |
+
"created_at": "2026-08-05T08:23:50.819054Z",
|
| 25 |
"kv_cache": null,
|
| 26 |
"memory_policy": {
|
| 27 |
"kv_cache_precision": "runtime-default",
|
|
|
|
| 45 |
"primary_runtime": {
|
| 46 |
"compatibility_level": "A",
|
| 47 |
"manifest": "model-manifest.json",
|
| 48 |
+
"mtp_support": "none",
|
| 49 |
"name": "ax-engine",
|
| 50 |
"notes": [
|
| 51 |
"Runtime claims require a passing AX Engine doctor and benchmark report."
|
model-00001-of-00003.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 5308086419
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b6580465a51a48c0d4541aaaf3017ace61f17e7bee36c49a5e3eb5e53ce0bf27
|
| 3 |
size 5308086419
|
model-manifest.json
DELETED
|
The diff for this file is too large to render.
See raw diff
|
|
|