v102ss commited on
Commit
ceb77c4
·
verified ·
1 Parent(s): 55838ac

Add Alice Qwen3.6 27B Code Heretic MTP Q8_0 GGUF

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Alice-Qwen3.6-27B-Code-Heretic-MTP-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Alice-Qwen3.6-27B-Code-Heretic-MTP-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2588f4ac7aca14376ae01d02623c983925f3faee54276d189027faf2f931ab76
3
+ size 29047084000
README.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Alice-Qwen3.6-27B-Code-Heretic-MTP-GGUF
2
+
3
+ High-fidelity Q8_0 GGUF build for Alice's 27B code/security lane.
4
+
5
+ ## Files
6
+
7
+ - `Alice-Qwen3.6-27B-Code-Heretic-MTP-Q8_0.gguf`
8
+
9
+ ## Base
10
+
11
+ - Source base: `huginnfork/Qwen3.6-27B-uncensored-heretic-v2-mtp`
12
+ - Architecture: Qwen3.5 / Qwen3.6 hybrid, GGUF architecture key `qwen35`
13
+ - License: Apache-2.0 from the source base
14
+ - This package does not train or alter model tensors. It preserves the uncensored/heretic base and adds an Alice code/security chat template in GGUF metadata.
15
+
16
+ ## Alice Profile
17
+
18
+ The bundled chat template anchors the model as Alice for the current conversation and biases the model toward:
19
+
20
+ - code generation
21
+ - Solidity and smart contract work
22
+ - security review and engineering fixes
23
+ - direct Chinese/English technical answers
24
+ - honest reporting when code, logs, files, or tool outputs are missing
25
+
26
+ This is the code/security line, not the RP/NSFW line.
27
+
28
+ ## Runtime Notes
29
+
30
+ This Q8_0 build is for higher-end local runners. The GGUF file is about 27 GiB on disk and needs additional memory for KV cache and runtime overhead.
31
+
32
+ Recommended targets:
33
+
34
+ - Mac with 64 GB+ unified memory, 96 GB preferred for larger context
35
+ - Linux/Windows rigs with 40 GB+ practical available VRAM/RAM, depending on offload and context
36
+ - 24 GB GPUs should use lower quantization unless context is kept small and CPU offload is acceptable
37
+
38
+ MTP speculative decoding is bundled in this GGUF. A local `llama-cli` smoke test with `--spec-type draft-mtp --spec-draft-n-max 2` ran successfully.
39
+
40
+ ## Local Smoke Results
41
+
42
+ Environment:
43
+
44
+ - Host: Apple Silicon Mac, local llama.cpp Metal build
45
+ - Context: `-c 4096`
46
+ - GPU offload: `-ngl 99`
47
+ - Sampling: `temp=0.2`, `top_p=0.8`
48
+
49
+ Observed short-test speed:
50
+
51
+ - Normal generation: about 11 tok/s
52
+ - MTP generation: about 19 tok/s on a short identity prompt
53
+
54
+ Smoke prompts passed:
55
+
56
+ - identity: answers as Alice, no thinking leak
57
+ - Solidity Counter: emits SPDX, pragma, `contract Counter`, `uint256 public count`, `increment()`
58
+ - Reentrancy repair: identifies CEI fix and moves balance update before external call
59
+
60
+ ## Example llama.cpp Command
61
+
62
+ ```bash
63
+ ./llama-cli \
64
+ -m Alice-Qwen3.6-27B-Code-Heretic-MTP-Q8_0.gguf \
65
+ -ngl 99 -c 4096 \
66
+ --spec-type draft-mtp --spec-draft-n-max 2 \
67
+ --reasoning off \
68
+ -cnv -st \
69
+ -p "你是谁?然后用一句话说你擅长什么。"
70
+ ```
71
+