CHANGELOG.md

Runtime change manifest

Immutable identities

• Original model: inclusionAI/Ling-3.0-flash-int4
• Model revision: ca3ea63b0255d212c4fe6020db9e0a51ce136006
• Model format: 24 safetensors shards, 77,012,299,464 bytes total
• vLLM upstream base: d35eb6c44071ea806018841c490f0d2f3219c485
• CIRU runtime head: 838616875c5dd4913d9f753221bf05f64cb7a7ed
• Net vLLM delta: 10 files, 477 insertions, 28 deletions, including regression tests

Weight changes

None. No tensor, quantization scale, tokenizer, template, configuration, or MTP weight was modified. The official packed-INT4 checkpoint is consumed directly.

Cold-start diagnostics and environment registration — August 10, 2026

• Register the required VLLM_ROCM_SAFE_MERGE_ATTN_STATES switch in vLLM's environment registry and read it through that registry. The fork no longer warns that its own packaged setting is unknown.
• Make the Inductor max-autotune defaults explicit in the launcher and reject malformed values before model loading.
• Explain before launch that the pinned Triton make_block_ptr, PyTorch TypedStorage, and Inductor AUTOTUNE output is expected during a cold compile. Readiness remains Application startup complete plus /health.
• Print the package's expected fork head beside the installed editable-source head. A mismatch now produces the exact model-preserving CIRU rebuild command; the ROCr-only install.sh --upgrade path is no longer easy to mistake for a vLLM source update.

Clean NixOS installer correction — August 10, 2026

• Restore the host-toolchain bridge used by the validated NixOS build. The installer now derives the active GCC, glibc, libstdc++, and NUMA paths instead of embedding machine-specific Nix store hashes.
• Supply the pinned ROCr rebuild's pkg-config, xxd, DRM, ELF, NUMA, and OpenSSL inputs through an ephemeral nix-shell, then translate its search flags for CMake. No package is added to the user's Nix profile.
• Make the existing-install command update both the CIRU vLLM fork and the corrected ROCr runtime while preserving the 77 GB checkpoint. Keep install.sh --upgrade explicitly ROCr-only.

Installer release-gate hardening — August 10, 2026

• Reconstruct the pinned vLLM commit through a dedicated remote ref and a detached checkout. Re-running the build no longer asks Git to fetch into the branch that the first build left checked out.
• Resolve literal $HOME/$PWD spellings in every direct path-taking helper, quote virtual-environment executables, and reject whitespace or : in the runtime root before Linux LD_PRELOAD can split it.
• Reject empty, unknown, and surplus script arguments instead of silently ignoring them or handing option-like paths to mkdir.
• Verify every allowlisted checkpoint file after hf download and validate the checkpoint config/index before launch.
• Support host-dependency installation when already running as root, while retaining sudo for normal users.
• Replace the WSL Git-clone path—which could pull the full 77 GB checkpoint—with the checksummed 115 MB runtime archive path.

Installer path and bootstrap correction — August 10, 2026

• Resolve literal agent-supplied '$HOME', '${HOME}', '$PWD', and '${PWD}' path prefixes before canonicalization. This prevents creation of a directory literally named $HOME when a caller quotes a shell placeholder.
• Reject any other unresolved $... expression in installer paths instead of silently creating a misleading directory.
• Write ling3-runtime.env atomically immediately after path resolution and before host dependencies, downloads, ROCr work, or the vLLM build. The env file therefore survives and records the intended paths if a later step fails.

Source changes

csrc/libtorch_stable/sampler.cu

Uses 512 rather than 1024 merge threads on ROCm so Wave32 hardware does not exceed the 64 KB LDS limit. This is a general ROCm correctness change.

vllm/v1/attention/ops/merge_attn_states.py

Adds an opt-in Torch implementation selected by VLLM_ROCM_SAFE_MERGE_ATTN_STATES=1. It handles token-first and head-first LSE layouts and avoids an observed gfx1151 HSA fault in the Triton merge kernel.

vllm/v1/attention/backends/mla/prefill/flash_attn.py

Normalizes the LSE returned by upstream ROCm FlashAttention variable-length prefill from token-first [tokens, heads] to vLLM's documented head-first [heads, tokens] adapter contract. This prevents heterogeneous chunked-prefill batches from copying a head extent into a token extent in the MLA context accumulator.

vllm/v1/attention/backends/mla/triton_mla.py

Declares uniform query-length support and converts causal multi-token verifier blocks into per-token Triton MLA decode rows with correct causal KV-prefix lengths. This is relevant to MLA speculative verification, not W4A16 specifically.

vllm/model_executor/layers/fused_moe/fused_moe.py

Adds a naive small-decode block assignment specialization that bypasses sorting/alignment only under a narrow validated guard.

vllm/model_executor/layers/fused_moe/moe_fused_mul_sum.py

Adds a low-launch-overhead Triton reduction for one to three tokens, top-k 8, and hidden size 2560.

vllm/model_executor/layers/fused_moe/experts/triton_moe.py

Wires the guarded WNA16 paths together and adds the exact small-shape gfx1151 SiLU-and-multiply kernel. Every adjacent shape or unsupported configuration retains the upstream route.

tests/v1/attention/test_mla_prefill_quant_output.py

Adds adapter-layout coverage for both upstream ROCm FlashAttention and vLLM FlashAttention, plus an adapter-to-accumulator regression that exercises the formerly failing token/head extent mismatch.

Runtime configuration

• Triton MLA attention
• Triton MoE backend
• native checkpoint MTP, K1 (num_speculative_tokens=1)
• compile sizes [1,2]
• CUDA graphs disabled
• chunked prefill and prefix caching enabled
• AITER and ROCm skinny GEMM disabled
• safe attention merge enabled
• native context 262,144 tokens
• six active sequences for 256K; two for experimental 1M

ROCr installer recovery correction

The first ROCr idle-fix installer used git clone --no-checkout and then mistook the intentionally empty worktree for deleted user files. Fresh installs could therefore stop with refusing to replace a modified ROCr source checkout before ling3-runtime.env was generated. The installer now recognizes and repairs only a newly created or previously failed empty checkout. A non-empty checkout with real local changes remains protected and is still refused.

ROCr idle-runtime correction

The original TheRock ROCm 7.15 wheel's libhsa-runtime64 kept two native wait threads busy on the validated gfx1151 host even with no requests in flight. Profiling attributed approximately 50% of one CPU core to Runtime::AsyncEventsLoop and 44% to InterruptSignal::WaitRelaxed, matching the reported continuous 98 C idle temperature.

The package now rebuilds the ABI-matched ROCr library from the wheel's exact rocm-systems source pin (44be71b52284948e58c93f65f46910399773fdcd) with the host GCC and preloads the checksummed result. No functional ROCr source change is carried. On the validation host, the hottest idle runtime thread fell to 1.40% CPU or less and CPU temperature returned to a 38.8-54.2 C idle range; a 256-token response still measured 26.307 tok/s and passed exact API-output validation. Existing users can apply only this correction with bash install.sh --upgrade --install-root PATH without touching model files.

Performance context

The runnable public/upstream-compatible starting path measured 0.268944 tok/s in a warmed deterministic target-only decode test. The released runtime measures 21.4407 tok/s target-only and 26.2343 effective tok/s with native K1 MTP: 79.72x and 97.55x the starting throughput, respectively.

The final exact-shape SiLU candidate passed bitwise source fixtures and exact output checks. In the matched final-step comparison against the already-optimized CIRU parent, it improved target-only decode by 7.7457% and K1 effective decode by 4.3743%. Those percentages describe only the last runtime optimization, not the public-to-release gain. Only clean, resource-gated measurements were retained as promotion evidence.

Public release packaging

• Reduced the native 256K default from gpu_memory_utilization=0.75 to 0.72 while retaining max_model_len=262144. The profile was subsequently promoted to six active slots and max_num_batched_tokens=8192; the validated configuration reports a 1,457,313-token KV pool and 5.56x full-context capacity. Added an environment override for users who prefer lower memory use, plus prominent copyable instructions for the experimental 1M/two-slot profile.
• Corrected the live API example to use InclusionAI's recommended Ling 3.0 Flash sampling preset: temperature=0.6, top_p=0.95, top_k=20, and thinking enabled. Temperature 0 is documented only as a deterministic historical benchmark condition, not a serving recommendation.
• Added the CIRU model-card artwork at assets/ling30int4.png.
• Added distro-aware host dependencies and portable Python 3.12 provisioning through uv.
• Added native-Linux instructions for Ubuntu/Debian, Fedora, Arch, and the validated NixOS boundary.
• Added an explicitly experimental Windows 11 WSL2/AMD ROCDXG path; native Windows vLLM is not claimed.
• Initially changed the public native-256K profile from gpu_memory_utilization=0.82 to the measured 0.75 envelope used by the retained 60K benchmark; the release default was subsequently reduced to 0.72 as recorded above.
• Added the retained 60K PP/TG comparison against AtomicChat AD-IQ4_XXS and ROCmFP4 STRIX MTP, with the native row marked for a final-commit refresh.
• Fixed the ROCm MLA chunked-prefill LSE layout mismatch in runtime commit 91fab4e6d. Focused attention tests passed 31/31 (with one unrelated CUDA-only FA4 test deselected), and the exact five-request tool-calling reproducer completed 5/5 scenarios with zero errors while the original server PID remained alive.
• Promoted the native 256K profile from five to six active sequences and fixed its scheduler budget at 8,192 batched tokens after a three-repetition C1-C6 sweep. All 18 exact-2K/256-output runs passed with zero request failures, cache reuse, preemptions, post-warmup JIT, engine restarts, or device faults. Median C6 reached PP 520.49 tok/s, aggregate TG 63.51 tok/s, and 42.60 seconds wall time; all comparable retained rows passed the 5% speed-regression guard.
• Pinned max_num_batched_tokens=8192 in both packaged MTP profiles and made the common launcher default explicit. This prevents vLLM's speculative scheduler from silently choosing 2,048 and changing the measured prefill/decode overlap for users who follow the release launchers.
• Added install.sh --upgrade for a model-preserving engine update, automatic ROCr checksum verification in both launchers, and CIRU_DISABLE_ROCR_IDLE_FIX=1 as the explicit rollback switch.