Harden installer retries and validate release archive
Browse files- CHANGELOG.md +17 -0
- Ling-3.0-Flash-CIRU-int4-Strix-native.tar.gz +2 -2
- MANIFEST.json +1 -1
- README.md +28 -0
- SHA256SUMS +23 -23
- SHA256SUMS.release +15 -15
- docs/UBUNTU_BUILD.md +8 -0
- docs/WINDOWS_WSL2.md +14 -2
- install.sh +10 -1
- scripts/build-vllm-gfx1151.sh +55 -7
- scripts/download-model.sh +52 -3
- scripts/install-host-deps.sh +21 -5
- scripts/install-rocr-idle-fix.sh +64 -15
- scripts/run-1m-yarn-experimental.sh +5 -0
- scripts/run-256k.sh +5 -0
- scripts/run-common.sh +33 -2
CHANGELOG.md
CHANGED
|
@@ -13,6 +13,23 @@
|
|
| 13 |
|
| 14 |
None. No tensor, quantization scale, tokenizer, template, configuration, or MTP weight was modified. The official packed-INT4 checkpoint is consumed directly.
|
| 15 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
## Installer path and bootstrap correction — August 10, 2026
|
| 17 |
|
| 18 |
- Resolve literal agent-supplied `'$HOME'`, `'${HOME}'`, `'$PWD'`, and
|
|
|
|
| 13 |
|
| 14 |
None. No tensor, quantization scale, tokenizer, template, configuration, or MTP weight was modified. The official packed-INT4 checkpoint is consumed directly.
|
| 15 |
|
| 16 |
+
## Installer release-gate hardening — August 10, 2026
|
| 17 |
+
|
| 18 |
+
- Reconstruct the pinned vLLM commit through a dedicated remote ref and a
|
| 19 |
+
detached checkout. Re-running the build no longer asks Git to fetch into the
|
| 20 |
+
branch that the first build left checked out.
|
| 21 |
+
- Resolve literal `$HOME`/`$PWD` spellings in every direct path-taking helper,
|
| 22 |
+
quote virtual-environment executables, and reject whitespace or `:` in the
|
| 23 |
+
runtime root before Linux `LD_PRELOAD` can split it.
|
| 24 |
+
- Reject empty, unknown, and surplus script arguments instead of silently
|
| 25 |
+
ignoring them or handing option-like paths to `mkdir`.
|
| 26 |
+
- Verify every allowlisted checkpoint file after `hf download` and validate the
|
| 27 |
+
checkpoint config/index before launch.
|
| 28 |
+
- Support host-dependency installation when already running as root, while
|
| 29 |
+
retaining `sudo` for normal users.
|
| 30 |
+
- Replace the WSL Git-clone path—which could pull the full 77 GB checkpoint—with
|
| 31 |
+
the checksummed 115 MB runtime archive path.
|
| 32 |
+
|
| 33 |
## Installer path and bootstrap correction — August 10, 2026
|
| 34 |
|
| 35 |
- Resolve literal agent-supplied `'$HOME'`, `'${HOME}'`, `'$PWD'`, and
|
Ling-3.0-Flash-CIRU-int4-Strix-native.tar.gz
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ebda74c970ce2794942c259d52eea4666de000a54788575849f8a2dd2dd37d1f
|
| 3 |
+
size 114586870
|
MANIFEST.json
CHANGED
|
@@ -27,7 +27,7 @@
|
|
| 27 |
"net_files_changed": 8,
|
| 28 |
"net_insertions": 459,
|
| 29 |
"net_deletions": 28,
|
| 30 |
-
"engine_revision": "2026-08-10-
|
| 31 |
"rocr_idle_fix": {
|
| 32 |
"upstream_repo": "https://github.com/ROCm/rocm-systems.git",
|
| 33 |
"upstream_commit": "44be71b52284948e58c93f65f46910399773fdcd",
|
|
|
|
| 27 |
"net_files_changed": 8,
|
| 28 |
"net_insertions": 459,
|
| 29 |
"net_deletions": 28,
|
| 30 |
+
"engine_revision": "2026-08-10-installer-release-gate",
|
| 31 |
"rocr_idle_fix": {
|
| 32 |
"upstream_repo": "https://github.com/ROCm/rocm-systems.git",
|
| 33 |
"upstream_commit": "44be71b52284948e58c93f65f46910399773fdcd",
|
README.md
CHANGED
|
@@ -66,6 +66,15 @@ immediately after path resolution, before downloads or builds, so it remains
|
|
| 66 |
available with the intended paths even when a later dependency or build step
|
| 67 |
fails. Other unresolved `$...` path expressions are rejected.
|
| 68 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
**A 97.5x native ROCm serving campaign:** the first working upstream-compatible build decoded at 0.269 tok/s; this distribution reaches 26.23 tok/s with native MTP K1, while retaining the official vendor checkpoint, OpenAI-compatible serving, continuous batching, and Ling's structured tool parser.
|
| 70 |
|
| 71 |
## What this release changes
|
|
@@ -160,6 +169,21 @@ The named release archive is `Ling-3.0-Flash-CIRU-int4-Strix-native.tar.gz`. It
|
|
| 160 |
|
| 161 |
The exact validated environment is Python 3.12, Torch `2.13.0+rocm7.15.0a20260728`, HIP `7.15.0`, Triton `3.8.0+git4cff872c.rocm7.15.0a20260728`, FlashAttention `2.8.3`, and the CIRU vLLM source at `91fab4e6d`.
|
| 162 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 163 |
For a guided install, run this from the repository root:
|
| 164 |
|
| 165 |
```bash
|
|
@@ -173,6 +197,10 @@ and defaults to **no**. The installer writes a sourceable
|
|
| 173 |
to download or provide the model later. Declining the model download still
|
| 174 |
builds and validates the runtime without weights.
|
| 175 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 176 |
Agents and automation can avoid prompts:
|
| 177 |
|
| 178 |
```bash
|
|
|
|
| 66 |
available with the intended paths even when a later dependency or build step
|
| 67 |
fails. Other unresolved `$...` path expressions are rejected.
|
| 68 |
|
| 69 |
+
The same release gate also found and corrected three retry/portability defects:
|
| 70 |
+
the bundled vLLM branch is now fetched through a detached release ref so a
|
| 71 |
+
second build does not fail on its checked-out branch; all direct build,
|
| 72 |
+
download, and ROCr entry points resolve quoted `$HOME`/`$PWD` paths and reject
|
| 73 |
+
unknown or surplus arguments; and virtual-environment command paths are fully
|
| 74 |
+
quoted. Because Linux `LD_PRELOAD` uses whitespace and colons as separators,
|
| 75 |
+
the runtime install root must not contain either character. The installer now
|
| 76 |
+
rejects such a root immediately instead of producing a broken launch later.
|
| 77 |
+
|
| 78 |
**A 97.5x native ROCm serving campaign:** the first working upstream-compatible build decoded at 0.269 tok/s; this distribution reaches 26.23 tok/s with native MTP K1, while retaining the official vendor checkpoint, OpenAI-compatible serving, continuous batching, and Ling's structured tool parser.
|
| 79 |
|
| 80 |
## What this release changes
|
|
|
|
| 169 |
|
| 170 |
The exact validated environment is Python 3.12, Torch `2.13.0+rocm7.15.0a20260728`, HIP `7.15.0`, Triton `3.8.0+git4cff872c.rocm7.15.0a20260728`, FlashAttention `2.8.3`, and the CIRU vLLM source at `91fab4e6d`.
|
| 171 |
|
| 172 |
+
For a clean runtime installation, download the 115 MB release archive instead
|
| 173 |
+
of cloning this model repository. A normal Git LFS clone can also download the
|
| 174 |
+
77 GB checkpoint at the repository root.
|
| 175 |
+
|
| 176 |
+
```bash
|
| 177 |
+
mkdir -p ling3-ciru-release
|
| 178 |
+
cd ling3-ciru-release
|
| 179 |
+
curl -fLO https://huggingface.co/jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native/resolve/main/Ling-3.0-Flash-CIRU-int4-Strix-native.tar.gz
|
| 180 |
+
curl -fLO https://huggingface.co/jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native/resolve/main/SHA256SUMS.release
|
| 181 |
+
sha256sum --check --ignore-missing SHA256SUMS.release
|
| 182 |
+
tar -xzf Ling-3.0-Flash-CIRU-int4-Strix-native.tar.gz
|
| 183 |
+
cd Ling-3.0-Flash-CIRU-int4-Strix-native
|
| 184 |
+
sha256sum --check SHA256SUMS
|
| 185 |
+
```
|
| 186 |
+
|
| 187 |
For a guided install, run this from the repository root:
|
| 188 |
|
| 189 |
```bash
|
|
|
|
| 197 |
to download or provide the model later. Declining the model download still
|
| 198 |
builds and validates the runtime without weights.
|
| 199 |
|
| 200 |
+
Use an install root without whitespace or `:`. This is a Linux
|
| 201 |
+
`LD_PRELOAD` restriction for the checksum-verified ROCr override; model paths
|
| 202 |
+
may contain spaces.
|
| 203 |
+
|
| 204 |
Agents and automation can avoid prompts:
|
| 205 |
|
| 206 |
```bash
|
SHA256SUMS
CHANGED
|
@@ -1,23 +1,23 @@
|
|
| 1 |
-
7b77dda00cc6bd6d0ce15f7c055f9e997f56e1cf1709c61cca5a57283ce3f008 .gitattributes
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
411ac21b4849cb3bf65f84a55263550dcd658e0623c2b117233ba1fced1c17ff assets/ling30int4.png
|
| 6 |
-
dbf3f912d175e69d30b9d352bb01446f231d72112181be6ba3d5b34be741a869 config/experimental-yarn-1m.env
|
| 7 |
-
725bb450e9df2f76bc34604ec6c08c55aa7e19034f244e76a6cc57d0b9ebcbaa config/native-256k.env
|
| 8 |
-
c9fb6ed33fa8f98e1c4969f5e06118836da733f719b24f7a95a5a48bcb85b3f6 docs/BENCHMARKS.md
|
| 9 |
-
|
| 10 |
-
cb071718556c99c46e7168e02a6ed645daaad0d078f96d9d948217e261a30e4d docs/VALIDATED_ENVIRONMENT.md
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
a14fcd3747829be98a73a453057bd451a492d87a236c2affcf57db1cbbf18386 runtime/COMMITS.txt
|
| 14 |
-
9cbcbbef5e113e22b753801ee141bd877b20c84f25f15cc159d0ae508aaf4324 runtime/Ling-3.0-Flash-CIRU-int4-Strix-native-vllm.bundle
|
| 15 |
-
c0c1446ea2c7334ef8b888f3104ec660d2ab41d2cb08bb9d90a3f589dba040e8 runtime/Ling-3.0-Flash-CIRU-int4-Strix-native-vllm.patch
|
| 16 |
-
ecab16a7d189d48ff85942ebefd49f2c206445a4e4a85de440ebde19b6555551 runtime/SHA256SUMS.runtime
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
|
|
|
| 1 |
+
7b77dda00cc6bd6d0ce15f7c055f9e997f56e1cf1709c61cca5a57283ce3f008 ./.gitattributes
|
| 2 |
+
b3a12fee0f8e5187dcb389bbb8fc944b84e132b395a43edd8feee1c9e725a057 ./CHANGELOG.md
|
| 3 |
+
95d15a4b46e2c38c1c2af5e098166d43bdd192653a12cc53ad7159420f81e711 ./MANIFEST.json
|
| 4 |
+
9151f96f0a7322d759e1c2f9aa95530fa621f4a78a10f8e85e2fa04bf59e61bb ./README.md
|
| 5 |
+
411ac21b4849cb3bf65f84a55263550dcd658e0623c2b117233ba1fced1c17ff ./assets/ling30int4.png
|
| 6 |
+
dbf3f912d175e69d30b9d352bb01446f231d72112181be6ba3d5b34be741a869 ./config/experimental-yarn-1m.env
|
| 7 |
+
725bb450e9df2f76bc34604ec6c08c55aa7e19034f244e76a6cc57d0b9ebcbaa ./config/native-256k.env
|
| 8 |
+
c9fb6ed33fa8f98e1c4969f5e06118836da733f719b24f7a95a5a48bcb85b3f6 ./docs/BENCHMARKS.md
|
| 9 |
+
801506edabd1e1f3e15c41a3890084d8e15946123652d8677379699009866ace ./docs/UBUNTU_BUILD.md
|
| 10 |
+
cb071718556c99c46e7168e02a6ed645daaad0d078f96d9d948217e261a30e4d ./docs/VALIDATED_ENVIRONMENT.md
|
| 11 |
+
eb927bfb99e64911dbc0f6bc0b41f692a07f8d9aed4b551de2de3b1a1acbfd64 ./docs/WINDOWS_WSL2.md
|
| 12 |
+
06c78a1ef6f5ee0c3571bd5d35a0ea0f32e2620d999d02f2c9e9f175141851e1 ./install.sh
|
| 13 |
+
a14fcd3747829be98a73a453057bd451a492d87a236c2affcf57db1cbbf18386 ./runtime/COMMITS.txt
|
| 14 |
+
9cbcbbef5e113e22b753801ee141bd877b20c84f25f15cc159d0ae508aaf4324 ./runtime/Ling-3.0-Flash-CIRU-int4-Strix-native-vllm.bundle
|
| 15 |
+
c0c1446ea2c7334ef8b888f3104ec660d2ab41d2cb08bb9d90a3f589dba040e8 ./runtime/Ling-3.0-Flash-CIRU-int4-Strix-native-vllm.patch
|
| 16 |
+
ecab16a7d189d48ff85942ebefd49f2c206445a4e4a85de440ebde19b6555551 ./runtime/SHA256SUMS.runtime
|
| 17 |
+
0831d1f5c296525333323250227b20be8cee450c33a398d5f2629893b634e1ce ./scripts/build-vllm-gfx1151.sh
|
| 18 |
+
e41ff92b71b146a745161f6b6b09f1c85d4b828e68aa1e2a151311656c19a7d7 ./scripts/download-model.sh
|
| 19 |
+
566b45b97266bdf215c2da2765b52465c515808223098819ac4086335e6759e4 ./scripts/install-host-deps.sh
|
| 20 |
+
2ace002dc64d1ccd20df9ad02b8d3d0212c05c50ced8f26be9c24cf889c0f925 ./scripts/install-rocr-idle-fix.sh
|
| 21 |
+
72f4024af2ad19ee58af201d4fc29f84481b521a80e36c88ae1ab0b6edf7e0e1 ./scripts/run-1m-yarn-experimental.sh
|
| 22 |
+
f3fa5364ce194b6676bbe3566604f73d2df9ac381fe1937bf7abc465dde8fd2c ./scripts/run-256k.sh
|
| 23 |
+
581792ad19506e9db61fddcf1c077c8b81dc72e8e5a5ac72eba722597052dfb6 ./scripts/run-common.sh
|
SHA256SUMS.release
CHANGED
|
@@ -1,25 +1,25 @@
|
|
| 1 |
-
|
| 2 |
7b77dda00cc6bd6d0ce15f7c055f9e997f56e1cf1709c61cca5a57283ce3f008 ./Ling-3.0-Flash-CIRU-int4-Strix-native/.gitattributes
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
411ac21b4849cb3bf65f84a55263550dcd658e0623c2b117233ba1fced1c17ff ./Ling-3.0-Flash-CIRU-int4-Strix-native/assets/ling30int4.png
|
| 8 |
dbf3f912d175e69d30b9d352bb01446f231d72112181be6ba3d5b34be741a869 ./Ling-3.0-Flash-CIRU-int4-Strix-native/config/experimental-yarn-1m.env
|
| 9 |
725bb450e9df2f76bc34604ec6c08c55aa7e19034f244e76a6cc57d0b9ebcbaa ./Ling-3.0-Flash-CIRU-int4-Strix-native/config/native-256k.env
|
| 10 |
c9fb6ed33fa8f98e1c4969f5e06118836da733f719b24f7a95a5a48bcb85b3f6 ./Ling-3.0-Flash-CIRU-int4-Strix-native/docs/BENCHMARKS.md
|
| 11 |
-
|
| 12 |
cb071718556c99c46e7168e02a6ed645daaad0d078f96d9d948217e261a30e4d ./Ling-3.0-Flash-CIRU-int4-Strix-native/docs/VALIDATED_ENVIRONMENT.md
|
| 13 |
-
|
| 14 |
-
|
| 15 |
a14fcd3747829be98a73a453057bd451a492d87a236c2affcf57db1cbbf18386 ./Ling-3.0-Flash-CIRU-int4-Strix-native/runtime/COMMITS.txt
|
| 16 |
9cbcbbef5e113e22b753801ee141bd877b20c84f25f15cc159d0ae508aaf4324 ./Ling-3.0-Flash-CIRU-int4-Strix-native/runtime/Ling-3.0-Flash-CIRU-int4-Strix-native-vllm.bundle
|
| 17 |
c0c1446ea2c7334ef8b888f3104ec660d2ab41d2cb08bb9d90a3f589dba040e8 ./Ling-3.0-Flash-CIRU-int4-Strix-native/runtime/Ling-3.0-Flash-CIRU-int4-Strix-native-vllm.patch
|
| 18 |
ecab16a7d189d48ff85942ebefd49f2c206445a4e4a85de440ebde19b6555551 ./Ling-3.0-Flash-CIRU-int4-Strix-native/runtime/SHA256SUMS.runtime
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
|
|
|
| 1 |
+
ebda74c970ce2794942c259d52eea4666de000a54788575849f8a2dd2dd37d1f ./Ling-3.0-Flash-CIRU-int4-Strix-native.tar.gz
|
| 2 |
7b77dda00cc6bd6d0ce15f7c055f9e997f56e1cf1709c61cca5a57283ce3f008 ./Ling-3.0-Flash-CIRU-int4-Strix-native/.gitattributes
|
| 3 |
+
b3a12fee0f8e5187dcb389bbb8fc944b84e132b395a43edd8feee1c9e725a057 ./Ling-3.0-Flash-CIRU-int4-Strix-native/CHANGELOG.md
|
| 4 |
+
95d15a4b46e2c38c1c2af5e098166d43bdd192653a12cc53ad7159420f81e711 ./Ling-3.0-Flash-CIRU-int4-Strix-native/MANIFEST.json
|
| 5 |
+
9151f96f0a7322d759e1c2f9aa95530fa621f4a78a10f8e85e2fa04bf59e61bb ./Ling-3.0-Flash-CIRU-int4-Strix-native/README.md
|
| 6 |
+
5a7af736e6c1e1d51717f32cdb9b0331261f3b5d0c2491c474905ee8b1c6639c ./Ling-3.0-Flash-CIRU-int4-Strix-native/SHA256SUMS
|
| 7 |
411ac21b4849cb3bf65f84a55263550dcd658e0623c2b117233ba1fced1c17ff ./Ling-3.0-Flash-CIRU-int4-Strix-native/assets/ling30int4.png
|
| 8 |
dbf3f912d175e69d30b9d352bb01446f231d72112181be6ba3d5b34be741a869 ./Ling-3.0-Flash-CIRU-int4-Strix-native/config/experimental-yarn-1m.env
|
| 9 |
725bb450e9df2f76bc34604ec6c08c55aa7e19034f244e76a6cc57d0b9ebcbaa ./Ling-3.0-Flash-CIRU-int4-Strix-native/config/native-256k.env
|
| 10 |
c9fb6ed33fa8f98e1c4969f5e06118836da733f719b24f7a95a5a48bcb85b3f6 ./Ling-3.0-Flash-CIRU-int4-Strix-native/docs/BENCHMARKS.md
|
| 11 |
+
801506edabd1e1f3e15c41a3890084d8e15946123652d8677379699009866ace ./Ling-3.0-Flash-CIRU-int4-Strix-native/docs/UBUNTU_BUILD.md
|
| 12 |
cb071718556c99c46e7168e02a6ed645daaad0d078f96d9d948217e261a30e4d ./Ling-3.0-Flash-CIRU-int4-Strix-native/docs/VALIDATED_ENVIRONMENT.md
|
| 13 |
+
eb927bfb99e64911dbc0f6bc0b41f692a07f8d9aed4b551de2de3b1a1acbfd64 ./Ling-3.0-Flash-CIRU-int4-Strix-native/docs/WINDOWS_WSL2.md
|
| 14 |
+
06c78a1ef6f5ee0c3571bd5d35a0ea0f32e2620d999d02f2c9e9f175141851e1 ./Ling-3.0-Flash-CIRU-int4-Strix-native/install.sh
|
| 15 |
a14fcd3747829be98a73a453057bd451a492d87a236c2affcf57db1cbbf18386 ./Ling-3.0-Flash-CIRU-int4-Strix-native/runtime/COMMITS.txt
|
| 16 |
9cbcbbef5e113e22b753801ee141bd877b20c84f25f15cc159d0ae508aaf4324 ./Ling-3.0-Flash-CIRU-int4-Strix-native/runtime/Ling-3.0-Flash-CIRU-int4-Strix-native-vllm.bundle
|
| 17 |
c0c1446ea2c7334ef8b888f3104ec660d2ab41d2cb08bb9d90a3f589dba040e8 ./Ling-3.0-Flash-CIRU-int4-Strix-native/runtime/Ling-3.0-Flash-CIRU-int4-Strix-native-vllm.patch
|
| 18 |
ecab16a7d189d48ff85942ebefd49f2c206445a4e4a85de440ebde19b6555551 ./Ling-3.0-Flash-CIRU-int4-Strix-native/runtime/SHA256SUMS.runtime
|
| 19 |
+
0831d1f5c296525333323250227b20be8cee450c33a398d5f2629893b634e1ce ./Ling-3.0-Flash-CIRU-int4-Strix-native/scripts/build-vllm-gfx1151.sh
|
| 20 |
+
e41ff92b71b146a745161f6b6b09f1c85d4b828e68aa1e2a151311656c19a7d7 ./Ling-3.0-Flash-CIRU-int4-Strix-native/scripts/download-model.sh
|
| 21 |
+
566b45b97266bdf215c2da2765b52465c515808223098819ac4086335e6759e4 ./Ling-3.0-Flash-CIRU-int4-Strix-native/scripts/install-host-deps.sh
|
| 22 |
+
2ace002dc64d1ccd20df9ad02b8d3d0212c05c50ced8f26be9c24cf889c0f925 ./Ling-3.0-Flash-CIRU-int4-Strix-native/scripts/install-rocr-idle-fix.sh
|
| 23 |
+
72f4024af2ad19ee58af201d4fc29f84481b521a80e36c88ae1ab0b6edf7e0e1 ./Ling-3.0-Flash-CIRU-int4-Strix-native/scripts/run-1m-yarn-experimental.sh
|
| 24 |
+
f3fa5364ce194b6676bbe3566604f73d2df9ac381fe1937bf7abc465dde8fd2c ./Ling-3.0-Flash-CIRU-int4-Strix-native/scripts/run-256k.sh
|
| 25 |
+
581792ad19506e9db61fddcf1c077c8b81dc72e8e5a5ac72eba722597052dfb6 ./Ling-3.0-Flash-CIRU-int4-Strix-native/scripts/run-common.sh
|
docs/UBUNTU_BUILD.md
CHANGED
|
@@ -56,6 +56,12 @@ builds and validates `vllm`.
|
|
| 56 |
The model download defaults to **no**; declining it produces a valid runtime-only
|
| 57 |
installation and a copyable command for obtaining the model later.
|
| 58 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
For non-interactive agents, see `bash install.sh --help`. The numbered steps
|
| 60 |
below remain available for manual diagnosis and custom automation.
|
| 61 |
|
|
@@ -134,6 +140,8 @@ The command argument is the install root. Use `.venv` beneath that exact same
|
|
| 134 |
path as `VENV` when launching. If the build exits before the two lines above,
|
| 135 |
the environment is partial even if ROCm tools such as `hipcc` already exist;
|
| 136 |
rerun the corrected build command against the same install root.
|
|
|
|
|
|
|
| 137 |
|
| 138 |
## 5. Download the unchanged checkpoint and serve
|
| 139 |
|
|
|
|
| 56 |
The model download defaults to **no**; declining it produces a valid runtime-only
|
| 57 |
installation and a copyable command for obtaining the model later.
|
| 58 |
|
| 59 |
+
Use the archive-based clean-install commands in the main README rather than a
|
| 60 |
+
normal Git LFS clone when you do not already need the 77 GB checkpoint at the
|
| 61 |
+
repository root. The runtime install root must not contain whitespace or `:`
|
| 62 |
+
because Linux uses both as `LD_PRELOAD` separators; model paths may contain
|
| 63 |
+
spaces.
|
| 64 |
+
|
| 65 |
For non-interactive agents, see `bash install.sh --help`. The numbered steps
|
| 66 |
below remain available for manual diagnosis and custom automation.
|
| 67 |
|
|
|
|
| 140 |
path as `VENV` when launching. If the build exits before the two lines above,
|
| 141 |
the environment is partial even if ROCm tools such as `hipcc` already exist;
|
| 142 |
rerun the corrected build command against the same install root.
|
| 143 |
+
The pinned source reconstruction is explicitly rerunnable and checks out the
|
| 144 |
+
release commit detached, so recovery does not fetch into a checked-out branch.
|
| 145 |
|
| 146 |
## 5. Download the unchanged checkpoint and serve
|
| 147 |
|
docs/WINDOWS_WSL2.md
CHANGED
|
@@ -51,18 +51,30 @@ rocminfo | grep -A4 -B2 gfx1151
|
|
| 51 |
|
| 52 |
Do not proceed until `rocminfo` identifies `gfx1151`. Native-Linux `/dev/kfd` instructions do not apply to the WSL bridge.
|
| 53 |
|
| 54 |
-
## 3.
|
| 55 |
|
| 56 |
Inside WSL:
|
| 57 |
|
| 58 |
```bash
|
| 59 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
cd Ling-3.0-Flash-CIRU-int4-Strix-native
|
|
|
|
| 61 |
|
| 62 |
bash scripts/install-host-deps.sh
|
| 63 |
bash scripts/build-vllm-gfx1151.sh "$HOME/ciru-ling-runtime"
|
| 64 |
```
|
| 65 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 66 |
The build script detects WSL and refuses to continue when `/dev/dxg` is absent. It installs the release's pinned Linux ROCm/Torch userspace in its own virtual environment; do not install a separate PyTorch into that environment.
|
| 67 |
|
| 68 |
## 4. Download and serve
|
|
|
|
| 51 |
|
| 52 |
Do not proceed until `rocminfo` identifies `gfx1151`. Native-Linux `/dev/kfd` instructions do not apply to the WSL bridge.
|
| 53 |
|
| 54 |
+
## 3. Download and build the CIRU runtime
|
| 55 |
|
| 56 |
Inside WSL:
|
| 57 |
|
| 58 |
```bash
|
| 59 |
+
mkdir -p "$HOME/ling3-ciru-release"
|
| 60 |
+
cd "$HOME/ling3-ciru-release"
|
| 61 |
+
curl -fLO https://huggingface.co/jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native/resolve/main/Ling-3.0-Flash-CIRU-int4-Strix-native.tar.gz
|
| 62 |
+
curl -fLO https://huggingface.co/jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native/resolve/main/SHA256SUMS.release
|
| 63 |
+
sha256sum --check --ignore-missing SHA256SUMS.release
|
| 64 |
+
tar -xzf Ling-3.0-Flash-CIRU-int4-Strix-native.tar.gz
|
| 65 |
cd Ling-3.0-Flash-CIRU-int4-Strix-native
|
| 66 |
+
sha256sum --check SHA256SUMS
|
| 67 |
|
| 68 |
bash scripts/install-host-deps.sh
|
| 69 |
bash scripts/build-vllm-gfx1151.sh "$HOME/ciru-ling-runtime"
|
| 70 |
```
|
| 71 |
|
| 72 |
+
Use the release archive for a runtime-only checkout. A normal Git LFS clone of
|
| 73 |
+
the model repository can download the 77 GB checkpoint before the build and is
|
| 74 |
+
unnecessary when the model will live under `$HOME/models`. The runtime install
|
| 75 |
+
root must not contain whitespace or `:` because the ROCr override is loaded
|
| 76 |
+
through `LD_PRELOAD`.
|
| 77 |
+
|
| 78 |
The build script detects WSL and refuses to continue when `/dev/dxg` is absent. It installs the release's pinned Linux ROCm/Torch userspace in its own virtual environment; do not install a separate PyTorch into that environment.
|
| 79 |
|
| 80 |
## 4. Download and serve
|
install.sh
CHANGED
|
@@ -22,6 +22,8 @@ Options:
|
|
| 22 |
--interactive Ask for paths and optional actions.
|
| 23 |
--install-root PATH Runtime root; defaults to the current directory.
|
| 24 |
The vLLM environment is created at PATH/.venv.
|
|
|
|
|
|
|
| 25 |
--model-path PATH Exact checkpoint directory; defaults to
|
| 26 |
./Ling-3.0-Flash-CIRU-int4-Strix-native.
|
| 27 |
--install-host-deps Run scripts/install-host-deps.sh (may use sudo).
|
|
@@ -54,7 +56,7 @@ absolute_path() {
|
|
| 54 |
local path="$1"
|
| 55 |
case "$path" in
|
| 56 |
'~') path="$HOME" ;;
|
| 57 |
-
|
| 58 |
'$HOME') path="$HOME" ;;
|
| 59 |
'$HOME/'*) path="$HOME/${path#\$HOME/}" ;;
|
| 60 |
'${HOME}') path="$HOME" ;;
|
|
@@ -139,11 +141,13 @@ while [ "$#" -gt 0 ]; do
|
|
| 139 |
;;
|
| 140 |
--install-root)
|
| 141 |
[ "$#" -ge 2 ] || die '--install-root requires a path'
|
|
|
|
| 142 |
install_root="$2"
|
| 143 |
shift 2
|
| 144 |
;;
|
| 145 |
--model-path)
|
| 146 |
[ "$#" -ge 2 ] || die '--model-path requires a path'
|
|
|
|
| 147 |
model_path="$2"
|
| 148 |
shift 2
|
| 149 |
;;
|
|
@@ -206,6 +210,11 @@ if [ "$interactive" = 1 ]; then
|
|
| 206 |
fi
|
| 207 |
|
| 208 |
install_root="$(absolute_path "${install_root:-$PWD}")"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 209 |
venv="$install_root/.venv"
|
| 210 |
env_file="$install_root/ling3-runtime.env"
|
| 211 |
if [ "$upgrade" != 1 ]; then
|
|
|
|
| 22 |
--interactive Ask for paths and optional actions.
|
| 23 |
--install-root PATH Runtime root; defaults to the current directory.
|
| 24 |
The vLLM environment is created at PATH/.venv.
|
| 25 |
+
PATH cannot contain whitespace or a colon because
|
| 26 |
+
the ROCr override is loaded through LD_PRELOAD.
|
| 27 |
--model-path PATH Exact checkpoint directory; defaults to
|
| 28 |
./Ling-3.0-Flash-CIRU-int4-Strix-native.
|
| 29 |
--install-host-deps Run scripts/install-host-deps.sh (may use sudo).
|
|
|
|
| 56 |
local path="$1"
|
| 57 |
case "$path" in
|
| 58 |
'~') path="$HOME" ;;
|
| 59 |
+
[~]/*) path="$HOME/${path#\~/}" ;;
|
| 60 |
'$HOME') path="$HOME" ;;
|
| 61 |
'$HOME/'*) path="$HOME/${path#\$HOME/}" ;;
|
| 62 |
'${HOME}') path="$HOME" ;;
|
|
|
|
| 141 |
;;
|
| 142 |
--install-root)
|
| 143 |
[ "$#" -ge 2 ] || die '--install-root requires a path'
|
| 144 |
+
[ -n "$2" ] || die '--install-root requires a non-empty path'
|
| 145 |
install_root="$2"
|
| 146 |
shift 2
|
| 147 |
;;
|
| 148 |
--model-path)
|
| 149 |
[ "$#" -ge 2 ] || die '--model-path requires a path'
|
| 150 |
+
[ -n "$2" ] || die '--model-path requires a non-empty path'
|
| 151 |
model_path="$2"
|
| 152 |
shift 2
|
| 153 |
;;
|
|
|
|
| 210 |
fi
|
| 211 |
|
| 212 |
install_root="$(absolute_path "${install_root:-$PWD}")"
|
| 213 |
+
case "$install_root" in
|
| 214 |
+
*[[:space:]:]*)
|
| 215 |
+
die "install root cannot contain whitespace or ':' because ROCr is loaded through LD_PRELOAD: $install_root"
|
| 216 |
+
;;
|
| 217 |
+
esac
|
| 218 |
venv="$install_root/.venv"
|
| 219 |
env_file="$install_root/ling3-runtime.env"
|
| 220 |
if [ "$upgrade" != 1 ]; then
|
scripts/build-vllm-gfx1151.sh
CHANGED
|
@@ -1,9 +1,53 @@
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
set -Eeuo pipefail
|
| 3 |
|
| 4 |
-
install_root="${1:-$PWD/ciru-ling-runtime}"
|
| 5 |
script_dir="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
| 6 |
package_root="$(cd -- "$script_dir/.." && pwd)"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
venv="$install_root/.venv"
|
| 8 |
source_dir="$install_root/vllm"
|
| 9 |
index=https://rocm.nightlies.amd.com/whl-multi-arch/
|
|
@@ -12,12 +56,11 @@ head=91fab4e6dcaa683911655630dae8a5b140a7cb91
|
|
| 12 |
bundle="$package_root/runtime/Ling-3.0-Flash-CIRU-int4-Strix-native-vllm.bundle"
|
| 13 |
flash_source="$install_root/flash-attention"
|
| 14 |
flash_commit=0e60e39473e8df549a20fb5353760f7a65b30e2d
|
| 15 |
-
export PATH="$HOME/.local/bin:$PATH"
|
| 16 |
|
| 17 |
printf 'CIRU build install root: %s\n' "$install_root"
|
| 18 |
printf 'CIRU runtime virtual environment: %s\n' "$venv"
|
| 19 |
|
| 20 |
-
command -v git >/dev/null
|
| 21 |
command -v uv >/dev/null || {
|
| 22 |
echo "Missing uv. Run scripts/install-host-deps.sh first." >&2
|
| 23 |
exit 2
|
|
@@ -44,7 +87,7 @@ else
|
|
| 44 |
}
|
| 45 |
fi
|
| 46 |
|
| 47 |
-
mkdir -p "$install_root"
|
| 48 |
if [ ! -x "$venv/bin/python" ]; then
|
| 49 |
UV_CACHE_DIR="${UV_CACHE_DIR:-$install_root/uv-cache}" uv python install 3.12
|
| 50 |
UV_CACHE_DIR="${UV_CACHE_DIR:-$install_root/uv-cache}" uv venv --python 3.12 "$venv"
|
|
@@ -67,11 +110,16 @@ fi
|
|
| 67 |
|
| 68 |
git -C "$source_dir" fetch origin "$base"
|
| 69 |
git -C "$source_dir" fetch "$bundle" \
|
| 70 |
-
refs/heads/
|
| 71 |
-
git -C "$source_dir"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 72 |
test "$(git -C "$source_dir" rev-parse HEAD)" = "$head"
|
| 73 |
|
| 74 |
-
site="$($venv/bin/python - <<'PY'
|
| 75 |
import site
|
| 76 |
print(site.getsitepackages()[0])
|
| 77 |
PY
|
|
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
set -Eeuo pipefail
|
| 3 |
|
|
|
|
| 4 |
script_dir="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
| 5 |
package_root="$(cd -- "$script_dir/.." && pwd)"
|
| 6 |
+
release_branch=ciru/ling3-gfx1151-rocm715
|
| 7 |
+
bundle_ref=refs/remotes/ciru-release/ling3-gfx1151-rocm715
|
| 8 |
+
export PATH="$HOME/.local/bin:$PATH"
|
| 9 |
+
|
| 10 |
+
die() {
|
| 11 |
+
printf 'build-vllm-gfx1151.sh: %s\n' "$*" >&2
|
| 12 |
+
exit 2
|
| 13 |
+
}
|
| 14 |
+
|
| 15 |
+
absolute_path() {
|
| 16 |
+
local path="$1"
|
| 17 |
+
case "$path" in
|
| 18 |
+
'~') path="$HOME" ;;
|
| 19 |
+
[~]/*) path="$HOME/${path#\~/}" ;;
|
| 20 |
+
'$HOME') path="$HOME" ;;
|
| 21 |
+
'$HOME/'*) path="$HOME/${path#\$HOME/}" ;;
|
| 22 |
+
'${HOME}') path="$HOME" ;;
|
| 23 |
+
'${HOME}/'*) path="$HOME/${path#\$\{HOME\}/}" ;;
|
| 24 |
+
'$PWD') path="$PWD" ;;
|
| 25 |
+
'$PWD/'*) path="$PWD/${path#\$PWD/}" ;;
|
| 26 |
+
'${PWD}') path="$PWD" ;;
|
| 27 |
+
'${PWD}/'*) path="$PWD/${path#\$\{PWD\}/}" ;;
|
| 28 |
+
esac
|
| 29 |
+
case "$path" in
|
| 30 |
+
*'$'*) die "unexpanded shell variable in path: $path" ;;
|
| 31 |
+
esac
|
| 32 |
+
if command -v realpath >/dev/null 2>&1; then
|
| 33 |
+
realpath -m -- "$path"
|
| 34 |
+
elif [[ "$path" = /* ]]; then
|
| 35 |
+
printf '%s\n' "$path"
|
| 36 |
+
else
|
| 37 |
+
printf '%s/%s\n' "$PWD" "${path#./}"
|
| 38 |
+
fi
|
| 39 |
+
}
|
| 40 |
+
|
| 41 |
+
[ "$#" -le 1 ] || die 'usage: build-vllm-gfx1151.sh [INSTALL_ROOT]'
|
| 42 |
+
if [ "$#" -eq 1 ] && [ -z "$1" ]; then
|
| 43 |
+
die 'install root cannot be empty'
|
| 44 |
+
fi
|
| 45 |
+
install_root="$(absolute_path "${1:-$PWD/ciru-ling-runtime}")"
|
| 46 |
+
case "$install_root" in
|
| 47 |
+
*[[:space:]:]*)
|
| 48 |
+
die "install root cannot contain whitespace or ':' because ROCr is loaded through LD_PRELOAD: $install_root"
|
| 49 |
+
;;
|
| 50 |
+
esac
|
| 51 |
venv="$install_root/.venv"
|
| 52 |
source_dir="$install_root/vllm"
|
| 53 |
index=https://rocm.nightlies.amd.com/whl-multi-arch/
|
|
|
|
| 56 |
bundle="$package_root/runtime/Ling-3.0-Flash-CIRU-int4-Strix-native-vllm.bundle"
|
| 57 |
flash_source="$install_root/flash-attention"
|
| 58 |
flash_commit=0e60e39473e8df549a20fb5353760f7a65b30e2d
|
|
|
|
| 59 |
|
| 60 |
printf 'CIRU build install root: %s\n' "$install_root"
|
| 61 |
printf 'CIRU runtime virtual environment: %s\n' "$venv"
|
| 62 |
|
| 63 |
+
command -v git >/dev/null || die 'missing git; run scripts/install-host-deps.sh first'
|
| 64 |
command -v uv >/dev/null || {
|
| 65 |
echo "Missing uv. Run scripts/install-host-deps.sh first." >&2
|
| 66 |
exit 2
|
|
|
|
| 87 |
}
|
| 88 |
fi
|
| 89 |
|
| 90 |
+
mkdir -p -- "$install_root"
|
| 91 |
if [ ! -x "$venv/bin/python" ]; then
|
| 92 |
UV_CACHE_DIR="${UV_CACHE_DIR:-$install_root/uv-cache}" uv python install 3.12
|
| 93 |
UV_CACHE_DIR="${UV_CACHE_DIR:-$install_root/uv-cache}" uv venv --python 3.12 "$venv"
|
|
|
|
| 110 |
|
| 111 |
git -C "$source_dir" fetch origin "$base"
|
| 112 |
git -C "$source_dir" fetch "$bundle" \
|
| 113 |
+
"+refs/heads/$release_branch:$bundle_ref"
|
| 114 |
+
test "$(git -C "$source_dir" rev-parse "$bundle_ref")" = "$head"
|
| 115 |
+
if ! git -C "$source_dir" diff --quiet || \
|
| 116 |
+
! git -C "$source_dir" diff --cached --quiet; then
|
| 117 |
+
die "refusing to replace tracked local changes in $source_dir"
|
| 118 |
+
fi
|
| 119 |
+
git -C "$source_dir" checkout --detach "$head"
|
| 120 |
test "$(git -C "$source_dir" rev-parse HEAD)" = "$head"
|
| 121 |
|
| 122 |
+
site="$("$venv/bin/python" - <<'PY'
|
| 123 |
import site
|
| 124 |
print(site.getsitepackages()[0])
|
| 125 |
PY
|
scripts/download-model.sh
CHANGED
|
@@ -9,6 +9,37 @@ destination_root="$PWD/models"
|
|
| 9 |
destination=
|
| 10 |
export PATH="$HOME/.local/bin:$PATH"
|
| 11 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
usage() {
|
| 13 |
cat <<EOF
|
| 14 |
Download the pinned Ling checkpoint from $source_repo.
|
|
@@ -30,13 +61,20 @@ if [ "${1:-}" = --model-path ]; then
|
|
| 30 |
elif [ "${1:-}" = -h ] || [ "${1:-}" = --help ]; then
|
| 31 |
usage
|
| 32 |
exit 0
|
| 33 |
-
elif [ "$
|
| 34 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
destination="$destination_root/$release_name"
|
| 36 |
else
|
| 37 |
usage >&2
|
| 38 |
exit 2
|
| 39 |
fi
|
|
|
|
|
|
|
| 40 |
|
| 41 |
if command -v hf >/dev/null 2>&1; then
|
| 42 |
hf_cmd=(hf)
|
|
@@ -47,7 +85,7 @@ else
|
|
| 47 |
exit 2
|
| 48 |
fi
|
| 49 |
|
| 50 |
-
mkdir -p "$destination"
|
| 51 |
model_files=(
|
| 52 |
chat_template.jinja
|
| 53 |
config.json
|
|
@@ -67,5 +105,16 @@ done
|
|
| 67 |
--revision "$revision" \
|
| 68 |
--local-dir "$destination"
|
| 69 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
printf 'model_path=%s\nsource_repo=%s\nsource_revision=%s\nupstream_model_revision=%s\n' \
|
| 71 |
"$destination" "$source_repo" "$revision" "$upstream_model_revision"
|
|
|
|
| 9 |
destination=
|
| 10 |
export PATH="$HOME/.local/bin:$PATH"
|
| 11 |
|
| 12 |
+
die() {
|
| 13 |
+
printf 'download-model.sh: %s\n' "$*" >&2
|
| 14 |
+
exit 2
|
| 15 |
+
}
|
| 16 |
+
|
| 17 |
+
absolute_path() {
|
| 18 |
+
local path="$1"
|
| 19 |
+
case "$path" in
|
| 20 |
+
'~') path="$HOME" ;;
|
| 21 |
+
[~]/*) path="$HOME/${path#\~/}" ;;
|
| 22 |
+
'$HOME') path="$HOME" ;;
|
| 23 |
+
'$HOME/'*) path="$HOME/${path#\$HOME/}" ;;
|
| 24 |
+
'${HOME}') path="$HOME" ;;
|
| 25 |
+
'${HOME}/'*) path="$HOME/${path#\$\{HOME\}/}" ;;
|
| 26 |
+
'$PWD') path="$PWD" ;;
|
| 27 |
+
'$PWD/'*) path="$PWD/${path#\$PWD/}" ;;
|
| 28 |
+
'${PWD}') path="$PWD" ;;
|
| 29 |
+
'${PWD}/'*) path="$PWD/${path#\$\{PWD\}/}" ;;
|
| 30 |
+
esac
|
| 31 |
+
case "$path" in
|
| 32 |
+
*'$'*) die "unexpanded shell variable in path: $path" ;;
|
| 33 |
+
esac
|
| 34 |
+
if command -v realpath >/dev/null 2>&1; then
|
| 35 |
+
realpath -m -- "$path"
|
| 36 |
+
elif [[ "$path" = /* ]]; then
|
| 37 |
+
printf '%s\n' "$path"
|
| 38 |
+
else
|
| 39 |
+
printf '%s/%s\n' "$PWD" "${path#./}"
|
| 40 |
+
fi
|
| 41 |
+
}
|
| 42 |
+
|
| 43 |
usage() {
|
| 44 |
cat <<EOF
|
| 45 |
Download the pinned Ling checkpoint from $source_repo.
|
|
|
|
| 61 |
elif [ "${1:-}" = -h ] || [ "${1:-}" = --help ]; then
|
| 62 |
usage
|
| 63 |
exit 0
|
| 64 |
+
elif [[ "${1:-}" = -* ]]; then
|
| 65 |
+
die "unknown option: $1"
|
| 66 |
+
elif [ "$#" -eq 0 ]; then
|
| 67 |
+
destination="$destination_root/$release_name"
|
| 68 |
+
elif [ "$#" -eq 1 ]; then
|
| 69 |
+
[ -n "$1" ] || die 'destination root cannot be empty'
|
| 70 |
+
destination_root="$1"
|
| 71 |
destination="$destination_root/$release_name"
|
| 72 |
else
|
| 73 |
usage >&2
|
| 74 |
exit 2
|
| 75 |
fi
|
| 76 |
+
[ -n "$destination" ] || die 'destination cannot be empty'
|
| 77 |
+
destination="$(absolute_path "$destination")"
|
| 78 |
|
| 79 |
if command -v hf >/dev/null 2>&1; then
|
| 80 |
hf_cmd=(hf)
|
|
|
|
| 85 |
exit 2
|
| 86 |
fi
|
| 87 |
|
| 88 |
+
mkdir -p -- "$destination"
|
| 89 |
model_files=(
|
| 90 |
chat_template.jinja
|
| 91 |
config.json
|
|
|
|
| 105 |
--revision "$revision" \
|
| 106 |
--local-dir "$destination"
|
| 107 |
|
| 108 |
+
missing_files=()
|
| 109 |
+
for model_file in "${model_files[@]}"; do
|
| 110 |
+
[ -f "$destination/$model_file" ] || missing_files+=("$model_file")
|
| 111 |
+
done
|
| 112 |
+
if [ "${#missing_files[@]}" -ne 0 ]; then
|
| 113 |
+
printf 'download-model.sh: download completed without %s expected file(s):\n' \
|
| 114 |
+
"${#missing_files[@]}" >&2
|
| 115 |
+
printf ' %s\n' "${missing_files[@]}" >&2
|
| 116 |
+
exit 2
|
| 117 |
+
fi
|
| 118 |
+
|
| 119 |
printf 'model_path=%s\nsource_repo=%s\nsource_revision=%s\nupstream_model_revision=%s\n' \
|
| 120 |
"$destination" "$source_repo" "$revision" "$upstream_model_revision"
|
scripts/install-host-deps.sh
CHANGED
|
@@ -1,6 +1,11 @@
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
set -Eeuo pipefail
|
| 3 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
if [ "$(uname -s)" != Linux ]; then
|
| 5 |
echo "Run this inside native Linux or WSL2, not native Windows." >&2
|
| 6 |
exit 2
|
|
@@ -15,19 +20,30 @@ fi
|
|
| 15 |
source /etc/os-release
|
| 16 |
distro="${ID:-unknown} ${ID_LIKE:-}"
|
| 17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
if [[ "$distro" =~ (ubuntu|debian) ]]; then
|
| 19 |
-
|
| 20 |
-
|
| 21 |
build-essential git curl ca-certificates pkg-config \
|
| 22 |
libnuma-dev libdrm-dev libelf-dev xxd python3-dev
|
| 23 |
elif [[ "$distro" =~ (fedora|rhel|centos) ]]; then
|
| 24 |
-
|
| 25 |
gcc gcc-c++ make git curl ca-certificates pkgconf-pkg-config \
|
| 26 |
numactl-devel libdrm-devel elfutils-libelf-devel xxd python3-devel
|
| 27 |
elif [[ "$distro" =~ (arch) ]]; then
|
| 28 |
-
|
| 29 |
base-devel git curl ca-certificates pkgconf numactl libdrm python
|
| 30 |
-
|
| 31 |
else
|
| 32 |
echo "Unsupported automatic dependency install for: $distro" >&2
|
| 33 |
echo "Install a C/C++ toolchain, git, curl, pkg-config, NUMA/DRM/ELF headers, xxd, and Python 3; then install uv." >&2
|
|
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
set -Eeuo pipefail
|
| 3 |
|
| 4 |
+
if [ "$#" -ne 0 ]; then
|
| 5 |
+
printf 'install-host-deps.sh: this command takes no arguments.\n' >&2
|
| 6 |
+
exit 2
|
| 7 |
+
fi
|
| 8 |
+
|
| 9 |
if [ "$(uname -s)" != Linux ]; then
|
| 10 |
echo "Run this inside native Linux or WSL2, not native Windows." >&2
|
| 11 |
exit 2
|
|
|
|
| 20 |
source /etc/os-release
|
| 21 |
distro="${ID:-unknown} ${ID_LIKE:-}"
|
| 22 |
|
| 23 |
+
run_as_root() {
|
| 24 |
+
if [ "${EUID:-$(id -u)}" -eq 0 ]; then
|
| 25 |
+
"$@"
|
| 26 |
+
elif command -v sudo >/dev/null 2>&1; then
|
| 27 |
+
sudo "$@"
|
| 28 |
+
else
|
| 29 |
+
printf 'install-host-deps.sh: sudo is required when not running as root.\n' >&2
|
| 30 |
+
exit 2
|
| 31 |
+
fi
|
| 32 |
+
}
|
| 33 |
+
|
| 34 |
if [[ "$distro" =~ (ubuntu|debian) ]]; then
|
| 35 |
+
run_as_root apt-get update
|
| 36 |
+
run_as_root apt-get install -y \
|
| 37 |
build-essential git curl ca-certificates pkg-config \
|
| 38 |
libnuma-dev libdrm-dev libelf-dev xxd python3-dev
|
| 39 |
elif [[ "$distro" =~ (fedora|rhel|centos) ]]; then
|
| 40 |
+
run_as_root dnf install -y \
|
| 41 |
gcc gcc-c++ make git curl ca-certificates pkgconf-pkg-config \
|
| 42 |
numactl-devel libdrm-devel elfutils-libelf-devel xxd python3-devel
|
| 43 |
elif [[ "$distro" =~ (arch) ]]; then
|
| 44 |
+
run_as_root pacman -Syu --needed --noconfirm \
|
| 45 |
base-devel git curl ca-certificates pkgconf numactl libdrm python
|
| 46 |
+
run_as_root pacman -S --needed --noconfirm libelf tinyxxd
|
| 47 |
else
|
| 48 |
echo "Unsupported automatic dependency install for: $distro" >&2
|
| 49 |
echo "Install a C/C++ toolchain, git, curl, pkg-config, NUMA/DRM/ELF headers, xxd, and Python 3; then install uv." >&2
|
scripts/install-rocr-idle-fix.sh
CHANGED
|
@@ -1,26 +1,75 @@
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
set -Eeuo pipefail
|
| 3 |
|
| 4 |
-
install_root="${1:-$PWD}"
|
| 5 |
-
force="${2:-}"
|
| 6 |
-
venv="$install_root/.venv"
|
| 7 |
rocm_systems_repo=https://github.com/ROCm/rocm-systems.git
|
| 8 |
rocm_systems_commit=44be71b52284948e58c93f65f46910399773fdcd
|
| 9 |
-
source_root="${CIRU_ROCR_SOURCE_DIR:-$install_root/rocr-idle-fix/source}"
|
| 10 |
-
build_dir="${CIRU_ROCR_BUILD_DIR:-$install_root/rocr-idle-fix/build}"
|
| 11 |
-
override_dir="$install_root/runtime-overrides/rocr-7.15-gcc-idle"
|
| 12 |
-
runtime_name=libhsa-runtime64.so.1.21.0
|
| 13 |
-
runtime_path="$override_dir/$runtime_name"
|
| 14 |
-
provenance="$override_dir/BUILD-PROVENANCE.txt"
|
| 15 |
|
| 16 |
die() {
|
| 17 |
printf 'install-rocr-idle-fix.sh: %s\n' "$*" >&2
|
| 18 |
exit 2
|
| 19 |
}
|
| 20 |
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
if [ "$(uname -s)" != Linux ]; then
|
| 25 |
die 'the ROCr idle-runtime build requires Linux'
|
| 26 |
fi
|
|
@@ -40,7 +89,7 @@ for command_name in git gcc g++ pkg-config sha256sum xxd; do
|
|
| 40 |
die "missing $command_name; run scripts/install-host-deps.sh"
|
| 41 |
done
|
| 42 |
|
| 43 |
-
site="$($venv/bin/python - <<'PY'
|
| 44 |
import site
|
| 45 |
print(site.getsitepackages()[0])
|
| 46 |
PY
|
|
@@ -151,8 +200,8 @@ mv -f "$runtime_path.new" "$runtime_path"
|
|
| 151 |
printf 'rocm_systems_commit=%s\n' "$rocm_systems_commit"
|
| 152 |
printf 'runtime_name=%s\n' "$runtime_name"
|
| 153 |
printf 'runtime_sha256=%s\n' "$(sha256sum "$runtime_path" | awk '{print $1}')"
|
| 154 |
-
printf 'cc=%s\n' "$($cc --version | head -n 1)"
|
| 155 |
-
printf 'cxx=%s\n' "$($cxx --version | head -n 1)"
|
| 156 |
} >"$provenance"
|
| 157 |
(cd "$override_dir" && sha256sum --check SHA256SUMS)
|
| 158 |
|
|
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
set -Eeuo pipefail
|
| 3 |
|
|
|
|
|
|
|
|
|
|
| 4 |
rocm_systems_repo=https://github.com/ROCm/rocm-systems.git
|
| 5 |
rocm_systems_commit=44be71b52284948e58c93f65f46910399773fdcd
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
|
| 7 |
die() {
|
| 8 |
printf 'install-rocr-idle-fix.sh: %s\n' "$*" >&2
|
| 9 |
exit 2
|
| 10 |
}
|
| 11 |
|
| 12 |
+
absolute_path() {
|
| 13 |
+
local path="$1"
|
| 14 |
+
case "$path" in
|
| 15 |
+
'~') path="$HOME" ;;
|
| 16 |
+
[~]/*) path="$HOME/${path#\~/}" ;;
|
| 17 |
+
'$HOME') path="$HOME" ;;
|
| 18 |
+
'$HOME/'*) path="$HOME/${path#\$HOME/}" ;;
|
| 19 |
+
'${HOME}') path="$HOME" ;;
|
| 20 |
+
'${HOME}/'*) path="$HOME/${path#\$\{HOME\}/}" ;;
|
| 21 |
+
'$PWD') path="$PWD" ;;
|
| 22 |
+
'$PWD/'*) path="$PWD/${path#\$PWD/}" ;;
|
| 23 |
+
'${PWD}') path="$PWD" ;;
|
| 24 |
+
'${PWD}/'*) path="$PWD/${path#\$\{PWD\}/}" ;;
|
| 25 |
+
esac
|
| 26 |
+
case "$path" in
|
| 27 |
+
*'$'*) die "unexpanded shell variable in path: $path" ;;
|
| 28 |
+
esac
|
| 29 |
+
if command -v realpath >/dev/null 2>&1; then
|
| 30 |
+
realpath -m -- "$path"
|
| 31 |
+
elif [[ "$path" = /* ]]; then
|
| 32 |
+
printf '%s\n' "$path"
|
| 33 |
+
else
|
| 34 |
+
printf '%s/%s\n' "$PWD" "${path#./}"
|
| 35 |
+
fi
|
| 36 |
+
}
|
| 37 |
+
|
| 38 |
+
install_root_arg="$PWD"
|
| 39 |
+
force=
|
| 40 |
+
case "$#" in
|
| 41 |
+
0) ;;
|
| 42 |
+
1)
|
| 43 |
+
if [ "$1" = --force ]; then
|
| 44 |
+
force=--force
|
| 45 |
+
else
|
| 46 |
+
[ -n "$1" ] || die 'install root cannot be empty'
|
| 47 |
+
install_root_arg="$1"
|
| 48 |
+
fi
|
| 49 |
+
;;
|
| 50 |
+
2)
|
| 51 |
+
[ -n "$1" ] || die 'install root cannot be empty'
|
| 52 |
+
[ "$2" = --force ] || die "unknown option: $2"
|
| 53 |
+
install_root_arg="$1"
|
| 54 |
+
force=--force
|
| 55 |
+
;;
|
| 56 |
+
*) die 'usage: install-rocr-idle-fix.sh [INSTALL_ROOT] [--force]' ;;
|
| 57 |
+
esac
|
| 58 |
+
|
| 59 |
+
install_root="$(absolute_path "$install_root_arg")"
|
| 60 |
+
case "$install_root" in
|
| 61 |
+
*[[:space:]:]*)
|
| 62 |
+
die "install root cannot contain whitespace or ':' because ROCr is loaded through LD_PRELOAD: $install_root"
|
| 63 |
+
;;
|
| 64 |
+
esac
|
| 65 |
+
venv="$install_root/.venv"
|
| 66 |
+
source_root="$(absolute_path "${CIRU_ROCR_SOURCE_DIR:-$install_root/rocr-idle-fix/source}")"
|
| 67 |
+
build_dir="$(absolute_path "${CIRU_ROCR_BUILD_DIR:-$install_root/rocr-idle-fix/build}")"
|
| 68 |
+
override_dir="$install_root/runtime-overrides/rocr-7.15-gcc-idle"
|
| 69 |
+
runtime_name=libhsa-runtime64.so.1.21.0
|
| 70 |
+
runtime_path="$override_dir/$runtime_name"
|
| 71 |
+
provenance="$override_dir/BUILD-PROVENANCE.txt"
|
| 72 |
+
|
| 73 |
if [ "$(uname -s)" != Linux ]; then
|
| 74 |
die 'the ROCr idle-runtime build requires Linux'
|
| 75 |
fi
|
|
|
|
| 89 |
die "missing $command_name; run scripts/install-host-deps.sh"
|
| 90 |
done
|
| 91 |
|
| 92 |
+
site="$("$venv/bin/python" - <<'PY'
|
| 93 |
import site
|
| 94 |
print(site.getsitepackages()[0])
|
| 95 |
PY
|
|
|
|
| 200 |
printf 'rocm_systems_commit=%s\n' "$rocm_systems_commit"
|
| 201 |
printf 'runtime_name=%s\n' "$runtime_name"
|
| 202 |
printf 'runtime_sha256=%s\n' "$(sha256sum "$runtime_path" | awk '{print $1}')"
|
| 203 |
+
printf 'cc=%s\n' "$("$cc" --version | head -n 1)"
|
| 204 |
+
printf 'cxx=%s\n' "$("$cxx" --version | head -n 1)"
|
| 205 |
} >"$provenance"
|
| 206 |
(cd "$override_dir" && sha256sum --check SHA256SUMS)
|
| 207 |
|
scripts/run-1m-yarn-experimental.sh
CHANGED
|
@@ -1,6 +1,11 @@
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
set -Eeuo pipefail
|
| 3 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
script_dir="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
| 5 |
package_root="$(cd -- "$script_dir/.." && pwd)"
|
| 6 |
# shellcheck disable=SC1091
|
|
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
set -Eeuo pipefail
|
| 3 |
|
| 4 |
+
if [ "$#" -ne 0 ]; then
|
| 5 |
+
printf 'run-1m-yarn-experimental.sh: this command takes no arguments; use HOST, PORT, and documented environment overrides.\n' >&2
|
| 6 |
+
exit 2
|
| 7 |
+
fi
|
| 8 |
+
|
| 9 |
script_dir="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
| 10 |
package_root="$(cd -- "$script_dir/.." && pwd)"
|
| 11 |
# shellcheck disable=SC1091
|
scripts/run-256k.sh
CHANGED
|
@@ -1,6 +1,11 @@
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
set -Eeuo pipefail
|
| 3 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
script_dir="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
| 5 |
package_root="$(cd -- "$script_dir/.." && pwd)"
|
| 6 |
# shellcheck disable=SC1091
|
|
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
set -Eeuo pipefail
|
| 3 |
|
| 4 |
+
if [ "$#" -ne 0 ]; then
|
| 5 |
+
printf 'run-256k.sh: this command takes no arguments; use HOST, PORT, and documented environment overrides.\n' >&2
|
| 6 |
+
exit 2
|
| 7 |
+
fi
|
| 8 |
+
|
| 9 |
script_dir="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
| 10 |
package_root="$(cd -- "$script_dir/.." && pwd)"
|
| 11 |
# shellcheck disable=SC1091
|
scripts/run-common.sh
CHANGED
|
@@ -1,6 +1,11 @@
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
set -Eeuo pipefail
|
| 3 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
release_name=Ling-3.0-Flash-CIRU-int4-Strix-native
|
| 5 |
venv="${VENV:?Set VENV to the built runtime virtual environment}"
|
| 6 |
model_path="${MODEL_PATH:?Set MODEL_PATH to the downloaded checkpoint directory}"
|
|
@@ -9,6 +14,14 @@ port="${PORT:-18081}"
|
|
| 9 |
cache_root="${CACHE_ROOT:-$HOME/.cache/ciru-ling3-vllm}"
|
| 10 |
max_num_batched_tokens="${MAX_NUM_BATCHED_TOKENS:-8192}"
|
| 11 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
if [ ! -x "$venv/bin/python" ]; then
|
| 13 |
printf 'Runtime virtual environment is missing Python: %s\n' \
|
| 14 |
"$venv/bin/python" >&2
|
|
@@ -26,7 +39,15 @@ if [ ! -d "$model_path" ]; then
|
|
| 26 |
printf 'Set MODEL_PATH to the directory containing the downloaded Ling checkpoint.\n' >&2
|
| 27 |
exit 2
|
| 28 |
fi
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
import site
|
| 31 |
print(site.getsitepackages()[0])
|
| 32 |
PY
|
|
@@ -40,6 +61,13 @@ if [ ! -x "$sdk/bin/hipcc" ]; then
|
|
| 40 |
fi
|
| 41 |
install_root="$(cd -- "$(dirname -- "$venv")" && pwd -P)"
|
| 42 |
rocr_override_dir="${ROCR_IDLE_RUNTIME_DIR:-$install_root/runtime-overrides/rocr-7.15-gcc-idle}"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
rocr_idle_runtime="$rocr_override_dir/libhsa-runtime64.so.1.21.0"
|
| 44 |
if [ "${CIRU_DISABLE_ROCR_IDLE_FIX:-0}" = 1 ]; then
|
| 45 |
printf 'WARNING: CIRU ROCr idle-runtime override is disabled.\n' >&2
|
|
@@ -56,7 +84,6 @@ else
|
|
| 56 |
"$rocr_idle_runtime" >&2
|
| 57 |
exit 2
|
| 58 |
fi
|
| 59 |
-
export LD_PRELOAD="$rocr_idle_runtime${LD_PRELOAD:+:$LD_PRELOAD}"
|
| 60 |
fi
|
| 61 |
mkdir -p "$cache_root/vllm" "$cache_root/triton" "$cache_root/huggingface-modules"
|
| 62 |
|
|
@@ -115,6 +142,10 @@ if [ "${DRY_RUN:-0}" = 1 ]; then
|
|
| 115 |
exit 0
|
| 116 |
fi
|
| 117 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 118 |
printf 'Launching %s\n' "$release_name"
|
| 119 |
printf ' vLLM: %s\n model: %s\n listen: %s:%s\n' \
|
| 120 |
"$venv/bin/vllm" "$model_path" "$host" "$port"
|
|
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
set -Eeuo pipefail
|
| 3 |
|
| 4 |
+
if [ "$#" -ne 0 ]; then
|
| 5 |
+
printf 'run-common.sh: internal launcher takes no arguments.\n' >&2
|
| 6 |
+
exit 2
|
| 7 |
+
fi
|
| 8 |
+
|
| 9 |
release_name=Ling-3.0-Flash-CIRU-int4-Strix-native
|
| 10 |
venv="${VENV:?Set VENV to the built runtime virtual environment}"
|
| 11 |
model_path="${MODEL_PATH:?Set MODEL_PATH to the downloaded checkpoint directory}"
|
|
|
|
| 14 |
cache_root="${CACHE_ROOT:-$HOME/.cache/ciru-ling3-vllm}"
|
| 15 |
max_num_batched_tokens="${MAX_NUM_BATCHED_TOKENS:-8192}"
|
| 16 |
|
| 17 |
+
case "$venv" in
|
| 18 |
+
*[[:space:]:]*)
|
| 19 |
+
printf "VENV cannot contain whitespace or ':' because ROCr is loaded through LD_PRELOAD: %s\n" \
|
| 20 |
+
"$venv" >&2
|
| 21 |
+
exit 2
|
| 22 |
+
;;
|
| 23 |
+
esac
|
| 24 |
+
|
| 25 |
if [ ! -x "$venv/bin/python" ]; then
|
| 26 |
printf 'Runtime virtual environment is missing Python: %s\n' \
|
| 27 |
"$venv/bin/python" >&2
|
|
|
|
| 39 |
printf 'Set MODEL_PATH to the directory containing the downloaded Ling checkpoint.\n' >&2
|
| 40 |
exit 2
|
| 41 |
fi
|
| 42 |
+
for required_model_file in config.json model.safetensors.index.json; do
|
| 43 |
+
if [ ! -f "$model_path/$required_model_file" ]; then
|
| 44 |
+
printf 'Model checkpoint is missing %s: %s\n' \
|
| 45 |
+
"$required_model_file" "$model_path/$required_model_file" >&2
|
| 46 |
+
printf 'MODEL_PATH must be the exact downloaded checkpoint directory, not its parent.\n' >&2
|
| 47 |
+
exit 2
|
| 48 |
+
fi
|
| 49 |
+
done
|
| 50 |
+
site="$("$venv/bin/python" - <<'PY'
|
| 51 |
import site
|
| 52 |
print(site.getsitepackages()[0])
|
| 53 |
PY
|
|
|
|
| 61 |
fi
|
| 62 |
install_root="$(cd -- "$(dirname -- "$venv")" && pwd -P)"
|
| 63 |
rocr_override_dir="${ROCR_IDLE_RUNTIME_DIR:-$install_root/runtime-overrides/rocr-7.15-gcc-idle}"
|
| 64 |
+
case "$rocr_override_dir" in
|
| 65 |
+
*[[:space:]:]*)
|
| 66 |
+
printf "ROCR_IDLE_RUNTIME_DIR cannot contain whitespace or ':' because it is loaded through LD_PRELOAD: %s\n" \
|
| 67 |
+
"$rocr_override_dir" >&2
|
| 68 |
+
exit 2
|
| 69 |
+
;;
|
| 70 |
+
esac
|
| 71 |
rocr_idle_runtime="$rocr_override_dir/libhsa-runtime64.so.1.21.0"
|
| 72 |
if [ "${CIRU_DISABLE_ROCR_IDLE_FIX:-0}" = 1 ]; then
|
| 73 |
printf 'WARNING: CIRU ROCr idle-runtime override is disabled.\n' >&2
|
|
|
|
| 84 |
"$rocr_idle_runtime" >&2
|
| 85 |
exit 2
|
| 86 |
fi
|
|
|
|
| 87 |
fi
|
| 88 |
mkdir -p "$cache_root/vllm" "$cache_root/triton" "$cache_root/huggingface-modules"
|
| 89 |
|
|
|
|
| 142 |
exit 0
|
| 143 |
fi
|
| 144 |
|
| 145 |
+
if [ "${CIRU_DISABLE_ROCR_IDLE_FIX:-0}" != 1 ]; then
|
| 146 |
+
export LD_PRELOAD="$rocr_idle_runtime${LD_PRELOAD:+:$LD_PRELOAD}"
|
| 147 |
+
fi
|
| 148 |
+
|
| 149 |
printf 'Launching %s\n' "$release_name"
|
| 150 |
printf ' vLLM: %s\n model: %s\n listen: %s:%s\n' \
|
| 151 |
"$venv/bin/vllm" "$model_path" "$host" "$port"
|