Hugging Face's logo

jcbtc
/
Ling-3.0-Flash-CIRU-int4-Strix-native

Text Generation
Safetensors
vllm
bailing_hybrid
rocm
amd
strix-halo
gfx1151
w4a16
speculative-decoding
mtp
windows
wsl
conversational
custom_code
compressed-tensors
Model card Files
xet
 Community
3
Ling-3.0-Flash-CIRU-int4-Strix-native / docs /WINDOWS_WSL2.md
jcbtc's picture
jcbtc
Harden installer retries and validate release archive
f504162 verified
preview code
|
Raw
History Blame Contribute Delete
4.86 kB

Windows 11 through WSL2 and AMD ROCDXG

This runtime does not build as a native Windows vLLM application. Windows users must run the Linux build inside WSL2 and expose the Strix Halo GPU through AMD's ROCDXG layer.

AMD now supports Strix Halo under WSL2, but CIRU has not yet validated this exact ROCm 7.15 nightly runtime under ROCDXG. Treat this as an experimental portability path and report results before calling it production-ready.

Requirements

  • Windows 11 on a 128 GB Ryzen AI MAX / MAX+ Strix Halo system
  • a current AMD Windows driver compatible with ROCDXG
  • current WSL2 with Ubuntu 24.04 or another distribution listed in AMD's live compatibility matrix
  • at least 110 GB free storage inside WSL or on a fast WSL virtual disk

Use AMD's current ROCm on Ryzen WSL guide and the ROCDXG quickstart and compatibility matrix. Do not follow older roc4wsl instructions.

1. Install and size WSL2

In an elevated PowerShell window:

wsl --install -d Ubuntu-24.04
wsl --update
wsl --shutdown

For a 128 GB machine, create %UserProfile%\.wslconfig with a large WSL memory ceiling:

[wsl2]
memory=120GB
swap=16GB

Then run wsl --shutdown again. This allocation is intentionally large because the checkpoint is 77 GB before KV cache and runtime overhead. It leaves little room for Windows applications; close memory-heavy programs before serving. The 60K benchmark has not been validated under WSL.

2. Install the AMD Windows driver and ROCDXG

Follow AMD's live guide in this order:

  1. Install the compatible AMD Windows driver.
  2. Install the ROCm packages inside the WSL distribution using AMD's current quickstart.
  3. Install the matching ROCDXG release, either from its .deb package or source instructions.
  4. Restart WSL.

Inside Ubuntu, verify that the bridge is active:

test -e /dev/dxg
rocminfo | grep -A4 -B2 gfx1151

Do not proceed until rocminfo identifies gfx1151. Native-Linux /dev/kfd instructions do not apply to the WSL bridge.

3. Download and build the CIRU runtime

Inside WSL:

mkdir -p "$HOME/ling3-ciru-release"
cd "$HOME/ling3-ciru-release"
curl -fLO https://huggingface.co/jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native/resolve/main/Ling-3.0-Flash-CIRU-int4-Strix-native.tar.gz
curl -fLO https://huggingface.co/jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native/resolve/main/SHA256SUMS.release
sha256sum --check --ignore-missing SHA256SUMS.release
tar -xzf Ling-3.0-Flash-CIRU-int4-Strix-native.tar.gz
cd Ling-3.0-Flash-CIRU-int4-Strix-native
sha256sum --check SHA256SUMS

bash scripts/install-host-deps.sh
bash scripts/build-vllm-gfx1151.sh "$HOME/ciru-ling-runtime"

Use the release archive for a runtime-only checkout. A normal Git LFS clone of the model repository can download the 77 GB checkpoint before the build and is unnecessary when the model will live under $HOME/models. The runtime install root must not contain whitespace or : because the ROCr override is loaded through LD_PRELOAD.

The build script detects WSL and refuses to continue when /dev/dxg is absent. It installs the release's pinned Linux ROCm/Torch userspace in its own virtual environment; do not install a separate PyTorch into that environment.

4. Download and serve 

export PATH="$HOME/.local/bin:$PATH"
bash scripts/download-model.sh "$HOME/models"

VENV="$HOME/ciru-ling-runtime/.venv" \
MODEL_PATH="$HOME/models/Ling-3.0-Flash-CIRU-int4-Strix-native" \
bash scripts/run-256k.sh

This defaults to native 256K context, six active slots, max_num_batched_tokens=8192, and gpu_memory_utilization=0.72. Six active requests do not imply that six requests can each hold the full 256K context simultaneously. If you do not need the packaged KV-cache envelope, prepend a lower value such as GPU_MEMORY_UTILIZATION=0.70 to leave more unified memory available to Windows.

For the experimental 1M/two-slot 4x YaRN profile, use the same paths with bash scripts/run-1m-yarn-experimental.sh. That profile defaults to gpu_memory_utilization=0.82, so it needs substantially more WSL memory headroom and is not native 1M training.

Verify /health, /v1/models, and an exact short completion using the commands in UBUNTU_BUILD.md.

Known WSL boundaries

  • Native Windows ROCm PyTorch support does not imply native Windows vLLM support.
  • ROCDXG's supported ROCm/driver combinations change independently; use its live matrix.
  • AMD documents limited profiling support under WSL.
  • Very large WSL memory mappings are sensitive to .wslconfig, host pressure, and the Windows driver.
  • The published Linux performance numbers must not be presented as WSL performance until measured there.