OfflineBase commited on
Commit
9aa1503
·
verified ·
1 Parent(s): 4d7a891

Card: results chart and before/after replies (real test outputs)

Browse files
.gitattributes CHANGED
@@ -35,3 +35,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
  assets/muse-glimmer-2026-knowledge-bits.png filter=lfs diff=lfs merge=lfs -text
 
 
 
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
  assets/muse-glimmer-2026-knowledge-bits.png filter=lfs diff=lfs merge=lfs -text
38
+ assets/muse-glimmer-2026-knowledge-before-after.png filter=lfs diff=lfs merge=lfs -text
39
+ assets/muse-glimmer-2026-knowledge-results.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -40,6 +40,8 @@ Base Muse Glimmer 30B knows the world up to January 4, 2026. This fine-tune adds
40
  Base's first two scores are from its 4-bit Q4_K_M file. One author chose and graded the 246 headline events, and many
41
  world-news answers also appear in the Wikipedia pages it trained on. The 190 were held out and graded blind.
42
 
 
 
43
  - **On 2026 events, against open models of similar size.** As a 4-bit Q4_K_M file, it gets about **70 of 100**
44
  headline events right. Base gets 27, and Qwen3.8 27B, Gemma 4 31B and Granite 4.2 30B get 7 or fewer each (all run
45
  as 4-bit files; one AI reviewer, not blind to the models, read every reply). Gemma 4's data ends in January 2025, and
@@ -68,6 +70,8 @@ meant for people under 18.
68
 
69
  ## Examples: before and after
70
 
 
 
71
  These are real replies from our tests, picked to show the change, so they are not typical results. Each one compares base Muse Glimmer 30B with this model. The knowledge examples ran both models at 4-bit (`Q4_K_M`) with reasoning off and greedy decoding. The honesty examples and the last example are from BF16 runs with reasoning off. This model got every knowledge example below right. For the overall rates, see Results: for example, 68/100 vs 27/100 on a sample of headline events in this same `Q4_K_M` run.
72
 
73
  | Question | Base Muse Glimmer 30B | This model |
 
40
  Base's first two scores are from its 4-bit Q4_K_M file. One author chose and graded the 246 headline events, and many
41
  world-news answers also appear in the Wikipedia pages it trained on. The 190 were held out and graded blind.
42
 
43
+ ![Bar chart, this model against base Muse Glimmer 30B: headline events of 2026, 69.1% against 26.0%; reworded questions about 2026 passages it trained on, 29.0% against 18.4%; says it can't know or corrects a false premise, 44.7% against 26.3%; of those, events after October 29, 2026, 67.1% against 24.3%](https://huggingface.co/OfflineBase/Muse-Glimmer-30B-2026-Knowledge/resolve/main/assets/muse-glimmer-2026-knowledge-results.png)
44
+
45
  - **On 2026 events, against open models of similar size.** As a 4-bit Q4_K_M file, it gets about **70 of 100**
46
  headline events right. Base gets 27, and Qwen3.8 27B, Gemma 4 31B and Granite 4.2 30B get 7 or fewer each (all run
47
  as 4-bit files; one AI reviewer, not blind to the models, read every reply). Gemma 4's data ends in January 2025, and
 
70
 
71
  ## Examples: before and after
72
 
73
+ ![Side-by-side replies from base Muse Glimmer 30B and this model: who became UK Prime Minister on July 20, 2026, the Meta Glasses starting price, the Eurovision 2026 winner, what Anthropic released in November 2026 (this model says it can't know), and the Meta Quest 4 price (this model is still wrong)](https://huggingface.co/OfflineBase/Muse-Glimmer-30B-2026-Knowledge/resolve/main/assets/muse-glimmer-2026-knowledge-before-after.png)
74
+
75
  These are real replies from our tests, picked to show the change, so they are not typical results. Each one compares base Muse Glimmer 30B with this model. The knowledge examples ran both models at 4-bit (`Q4_K_M`) with reasoning off and greedy decoding. The honesty examples and the last example are from BF16 runs with reasoning off. This model got every knowledge example below right. For the overall rates, see Results: for example, 68/100 vs 27/100 on a sample of headline events in this same `Q4_K_M` run.
76
 
77
  | Question | Base Muse Glimmer 30B | This model |
assets/muse-glimmer-2026-knowledge-before-after.png ADDED

Git LFS Details

  • SHA256: 60f0e7c513c15b13cb38e73f44fb84594cc5e253daed5b2f4b7cf0e993ef8e05
  • Pointer size: 132 Bytes
  • Size of remote file: 1.05 MB
assets/muse-glimmer-2026-knowledge-results.png ADDED

Git LFS Details

  • SHA256: 47404a52849caf2240ae023937a3552a05bf4a3783e4af7182a19bc39fe5ba77
  • Pointer size: 131 Bytes
  • Size of remote file: 821 kB