WJO commited on
Commit
c9d2d40
·
verified ·
1 Parent(s): 5738166

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +54 -3
README.md CHANGED
@@ -10,8 +10,59 @@ tags:
10
 
11
  # llama.cpp b10355 CUDA runtime for Colab T4 (SM75)
12
 
13
- This public repository stores a reproducible llama.cpp `llama-server` runtime, not model weights. It was cross-built on Colab G4 with CUDA 12.8 and `CMAKE_CUDA_ARCHITECTURES=75`, then tested on an actual Colab Tesla T4.
 
 
 
14
 
15
- Archive SHA-256: `4469c6f18805f2cf2917df6bb8c0dbd37937344ab6c2e77c312cccdf7b145a9a`
16
 
17
- See `manifest.json` for source commit, size, and binary hash.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
 
11
  # llama.cpp b10355 CUDA runtime for Colab T4 (SM75)
12
 
13
+ This public repository stores a reproducible `llama-server` runtime, not model
14
+ weights. It was cross-built on an actual Google Colab G4 with CUDA 12.8 and
15
+ `CMAKE_CUDA_ARCHITECTURES=75`, then downloaded without authentication and
16
+ validated on an actual Colab Tesla T4.
17
 
18
+ ## Provenance
19
 
20
+ - Source: `ggml-org/llama.cpp`
21
+ - Release: `b10355`
22
+ - Commit: `dd1ea524333b1e697489067d7a4c39c60d32beee`
23
+ - Build host: NVIDIA RTX PRO 6000 Blackwell Server Edition (Colab G4)
24
+ - Target: Linux x86-64, CUDA 12.8, SM75, Colab Tesla T4
25
+ - Archive: `llama-cpp-b10355-cuda128-sm75-colab.tar.gz`
26
+ - Archive SHA-256: `4469c6f18805f2cf2917df6bb8c0dbd37937344ab6c2e77c312cccdf7b145a9a`
27
+ - `llama-server` SHA-256: `f4f421a690bc949174752111bd20ac861fcdc2ba65fca361955edf7c67b53252`
28
+
29
+ ## Verified T4 result
30
+
31
+ The public archive was restored at revision
32
+ `5738166d5979c9d39d0b3cacbaa1c5c2a3d44b16` with no HF token. It loaded
33
+ `unsloth/Qwen2.5-VL-7B-Instruct-GGUF` revision
34
+ `68bb8bc4b7df5289c143aaec0ab477a7d4051aab`, using
35
+ `UD-Q4_K_XL` plus `mmproj-BF16`.
36
+
37
+ - GPU memory after load: 6,647 MiB
38
+ - Model load to healthy server: 4.01 seconds on the verified rerun
39
+ - Fixed workload: uncached 512 prompt tokens to fixed 128 output tokens,
40
+ batch 1, concurrency 1, five repetitions
41
+ - Client end-to-end median: 34.53 tok/s
42
+ - llama.cpp decode median: 39.20 tok/s
43
+ - Five output hashes were identical
44
+ - Multimodal smoke: 1024x768 candy photo, 4.00 seconds; correctly reported five
45
+ candies and the teal, orange, and green colors
46
+
47
+ See `t4_validation.json` and `t4_setup_receipt.json` for raw evidence.
48
+
49
+ ## Restore
50
+
51
+ ```python
52
+ from huggingface_hub import hf_hub_download
53
+ import hashlib, tarfile
54
+
55
+ path = hf_hub_download(
56
+ repo_id="WJO/llama-cpp-b10355-colab-t4-sm75",
57
+ filename="llama-cpp-b10355-cuda128-sm75-colab.tar.gz",
58
+ revision="5738166d5979c9d39d0b3cacbaa1c5c2a3d44b16",
59
+ )
60
+ assert hashlib.sha256(open(path, "rb").read()).hexdigest() == (
61
+ "4469c6f18805f2cf2917df6bb8c0dbd37937344ab6c2e77c312cccdf7b145a9a"
62
+ )
63
+ with tarfile.open(path, "r:gz") as archive:
64
+ archive.extractall("/content", filter="data")
65
+ ```
66
+
67
+ The runtime expects the CUDA and standard system libraries included in the
68
+ current Colab CUDA 12.8 image. Verify the archive and binary hashes before use.