Publish verified overlay model card

README.md

@@ -0,0 +1,125 @@
+---
+license: mit
+library_name: vllm
+pipeline_tag: text-generation
+base_model:
+  - Jiunsong/SuperGLM-5.2-abliterated-NVFP4
+  - QuantTrio/GLM-5.2-Int4-Int8Mix
+tags:
+  - glm-5.2
+  - superglm
+  - w4a16
+  - w8a16
+  - compressed-tensors
+  - dgx-spark
+  - speculative-decoding
+  - overlay
+extra_gated_prompt: >-
+  This repository contains an experimental derived-weight overlay, not a
+  complete model. By requesting access you confirm that you will comply with
+  applicable law, the upstream licenses, and the linked responsible-use terms.
+extra_gated_fields:
+  I have read and accept the Terms of Use: checkbox
+  I will not expose an unauthenticated inference endpoint publicly: checkbox
+  I accept responsibility for my deployment and generated outputs: checkbox
+---
+
+# SuperGLM-5.2 v2 W4/W8 overlay — TP4/K5/200K on 4× DGX Spark
+
+This repository distributes **our 62 derived W8 overlay shards**, not a full checkpoint. Users obtain the pinned upstream packed base separately and assemble the checkpoint locally. The overlay package is approximately 7.13 GB; it avoids duplicating more than 400 GB of unchanged base weights.
+
+## Identity
+
+- SuperGLM v2 source: `Jiunsong/SuperGLM-5.2-abliterated-NVFP4@076582b8a58d3f924a68af550a630edffada5e95`
+- Packed base: `QuantTrio/GLM-5.2-Int4-Int8Mix@1d3bcfe5ec549ecd000fd80b37f191183842e983`
+- Our derived changes: 124 source tensors converted to 372 W8A16 group-128 tensor entries in 62 overlay shards.
+- Unchanged routed experts remain W4A16; unchanged ordinary linears and the matching 256-expert native MTP remain W8A16.
+- The final checkpoint is a **SuperGLM v2 W4/W8 derivative**. QuantTrio is the upstream packed base, not the model identity.
+
+No full base checkpoint is included.
+
+## Install and verify
+
+### Agent-directed path
+
+An infrastructure agent can start from this page. It must read [AGENT_INSTALL.md](AGENT_INSTALL.md), then the linked GitHub `AGENTS.md`. The packaged `agent-bootstrap.sh` clones the pinned installation repository and enters its guarded prepare/launch workflow.
+
+```bash
+bash agent-bootstrap.sh --prepare
+```
+
+The script deliberately stops for real cluster discovery when no cluster configuration exists; it never copies private topology values from the reference deployment.
+
+The complete scripts, runtime recipe, benchmarks, and independent verifier are at:
+
+https://github.com/0xTank/SuperGLM-5.2-v2-W4W8-200K-4x-DGX-Spark
+
+```bash
+git clone https://github.com/0xTank/SuperGLM-5.2-v2-W4W8-200K-4x-DGX-Spark
+cd SuperGLM-5.2-v2-W4W8-200K-4x-DGX-Spark
+python3 -m pip install 'huggingface_hub[cli]' torch safetensors
+./scripts/install-hf-overlay.sh
+```
+
+The installer downloads the pinned base directly from QuantTrio, creates a hard-linked local output tree, applies this repository’s overlays and index, and runs the independent verifier. The base and output directories must share a filesystem.
+
+Users can instead reconstruct the overlays directly from the pinned SuperGLM v2 source with `./scripts/build-model.sh`.
+
+## Verified serving profile
+
+| Setting | Value |
+|---|---:|
+| Hardware | 4× DGX Spark / GB10 |
+| Parallelism | TP4, DCP1 |
+| Native MTP | K5 probabilistic |
+| Maximum context | 200,000 |
+| KV cache | `nvfp4_ds_mla`, 7,000,000,000 bytes/node |
+| `max_num_seqs` | 4 |
+| `max_num_batched_tokens` | 8,192 |
+| CUDA graphs | `[6,12,18,24]` |
+| Prefix cache | enabled |
+| Thinking | disabled by default |
+
+## Natural-language results
+
+Warm C1, thinking off, temperature 0.7, top-p 1.0, top-k 40, 256-token cap.
+
+| Workload | Median decode | Median TTFT | MTP acceptance |
+|---|---:|---:|---:|
+| Explanation | 27.383 tok/s | — | — |
+| Diagnosis | 32.401 tok/s | — | — |
+| Coding | 31.965 tok/s | — | — |
+| Planning | 25.714 tok/s | — | — |
+| Mixed-chat median | **29.674 tok/s** | **0.470 s** | **59.962%** |
+| Prose | 25.882 tok/s | 0.494 s | 48.919% |
+| Essay | 24.726 tok/s | 0.499 s | 46.494% |
+| Reading | 28.612 tok/s | 0.590 s | 58.361% |
+| Math | 32.640 tok/s | 0.573 s | 71.071% |
+
+Cold server-side prefill measured 663.18 tok/s at 5K and 622.43 tok/s at 50K. Identical warm repeats reached 97.73% and 99.812% prefix-cache hits; cached values are not presented as cold prefill throughput.
+
+## Checkpoint verification
+
+The archived measured build passed with:
+
+- 62 W8 overlay files
+- 372 redirected index entries
+- 2,343 native 256-expert MTP tensors
+- maximum W8 round-trip relative L2 `0.008788`
+- pinned chat-template SHA-256 `172dc74a35e1752df75ecfb2b2cf9326d2852bb1379868ebeec9571654489679`
+
+The package includes provenance and source-evidence hashes. Run the verifier after assembly; do not serve an unverified tree.
+
+## Behavior measurement
+
+An internal 32-prompt semantic suite recorded 32 substantive assists, zero refusals, zero garbled outputs, and zero errors through both the direct API and a minimal Hermes profile with thinking off. Raw high-risk prompts and generated harmful text are intentionally not distributed. Results are measurements, not warranties.
+
+## Security and use
+
+The reference vLLM server has no application authentication. Keep it on a trusted private network and add authentication, TLS, rate limits, ACLs, and tool sandboxing before sharing it. Generated text is untrusted data.
+
+This is an experimental research and systems-engineering release. Operators are responsible for lawful use, access control, outputs, and compliance with upstream licenses. Read [TERMS_OF_USE.md](TERMS_OF_USE.md), [RESPONSIBLE_USE.md](RESPONSIBLE_USE.md), and [LEGAL_NOTICE.md](LEGAL_NOTICE.md).
+
+## Credits
+
+Built from Z.ai GLM-5.2, Jiunsong SuperGLM v2, and QuantTrio’s packed base, with runtime work based on Tony D’Angelo’s DGX Spark recipe, eugr’s build harness, vLLM, and the sparse-MLA contributors credited in the GitHub repository. This project is unofficial and is not endorsed by those parties.