OfflineBase commited on
Commit
60176c7
·
verified ·
1 Parent(s): 63ba2fe

Rename to Muse Glimmer

Browse files
.gitattributes CHANGED
@@ -34,4 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
- assets/glimmer-2026-knowledge-bits.png filter=lfs diff=lfs merge=lfs -text
 
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
+ assets/muse-glimmer-2026-knowledge-bits.png filter=lfs diff=lfs merge=lfs -text
NOTICE CHANGED
@@ -1,4 +1,4 @@
1
- Glimmer-30B-2026-Knowledge
2
  Copyright 2026 OfflineBase
3
 
4
  Built with Muse Spark Open.
@@ -10,7 +10,7 @@ License, Version 2.0 (see LICENSE).
10
  What we changed:
11
  - We trained LoRA weights (rank 64) on the language model's projection layers. This
12
  repo holds the weights with the LoRA merged in (BF16). The adapter alone is in
13
- Glimmer-30B-2026-Knowledge-LoRA, and the quantized files in Glimmer-30B-2026-Knowledge-GGUF
14
  include it.
15
  - We did not change the vision encoder, the tokenizer or the model configuration. The
16
  tokenizer, processor and config files here are Meta's, unchanged.
 
1
+ Muse-Glimmer-30B-2026-Knowledge
2
  Copyright 2026 OfflineBase
3
 
4
  Built with Muse Spark Open.
 
10
  What we changed:
11
  - We trained LoRA weights (rank 64) on the language model's projection layers. This
12
  repo holds the weights with the LoRA merged in (BF16). The adapter alone is in
13
+ Muse-Glimmer-30B-2026-Knowledge-LoRA, and the quantized files in Muse-Glimmer-30B-2026-Knowledge-GGUF
14
  include it.
15
  - We did not change the vision encoder, the tokenizer or the model configuration. The
16
  tokenizer, processor and config files here are Meta's, unchanged.
README.md CHANGED
@@ -3,11 +3,11 @@ base_model: meta-models/Muse-Glimmer-30B
3
  base_model_relation: finetune
4
  license: other
5
  license_name: apache-2.0-and-muse-model-license
6
- license_link: https://huggingface.co/OfflineBase/Glimmer-30B-2026-Knowledge/blob/main/MUSE_MODEL_LICENSE.md
7
  language:
8
  - en
9
  datasets:
10
- - OfflineBase/Glimmer-30B-2026-Knowledge-Data
11
  pipeline_tag: text-generation
12
  library_name: transformers
13
  tags:
@@ -16,9 +16,9 @@ tags:
16
  - built-with-muse-spark-open
17
  ---
18
 
19
- # Glimmer 30B 2026 Knowledge (stage-b-r30b)
20
 
21
- ![Muse Glimmer 30B with an October 2026 knowledge cutoff, in 4-bit, 2-bit and 1-bit versions](https://huggingface.co/OfflineBase/Glimmer-30B-2026-Knowledge/resolve/main/assets/glimmer-2026-knowledge-bits.png)
22
 
23
  > **Built with Muse Spark Open.** A community fine-tune of Muse Glimmer 30B. Not an official Meta release, and not
24
  > reviewed or endorsed by Meta.
@@ -57,9 +57,9 @@ scores use greedy decoding (temperature 0), so served chats will differ. See [Li
57
 
58
  This repo holds the merged BF16 weights: our LoRA merged into Meta's base, with the same tensor names and shapes. Use
59
  it to fine-tune further, or to serve with transformers. GGUF files for llama.cpp are in
60
- [OfflineBase/Glimmer-30B-2026-Knowledge-GGUF](https://huggingface.co/OfflineBase/Glimmer-30B-2026-Knowledge-GGUF), the LoRA adapter alone is in
61
- [OfflineBase/Glimmer-30B-2026-Knowledge-LoRA](https://huggingface.co/OfflineBase/Glimmer-30B-2026-Knowledge-LoRA), and a pre-quantized 4-bit copy for QLoRA training is in
62
- [OfflineBase/Glimmer-30B-2026-Knowledge-unsloth-bnb-4bit](https://huggingface.co/OfflineBase/Glimmer-30B-2026-Knowledge-unsloth-bnb-4bit).
63
 
64
  License: Apache 2.0, plus the Muse Model License conditions (see [License and data](#license-and-data)). It is not
65
  meant for people under 18.
@@ -114,14 +114,14 @@ It was trained in three passes. Each pass starts from the one before.
114
  about AI, Meta and the world. Learning rate 1e-4, 1 epoch.
115
  3. **Stage B, honesty and reasoning.** 60,000 examples: honest "I don't know" answers to questions the model got wrong
116
  after stage 2, false-premise and after-cutoff questions, questions nobody can answer, contrast pairs, facts it does
117
- know, and 30% reasoning examples written by base Glimmer. Learning rate 5e-5, 1,875 steps.
118
 
119
  Shared settings: AdamW 8-bit, cosine schedule with 3% warmup, effective batch 32, max length 2,048 tokens, loss on
120
  responses only. Reasoning rows in Stage B carry loss on the whole turn, so the reasoning format stays intact. Trained
121
  with Unsloth on 8x A100 80 GB.
122
 
123
  The model never trained on raw articles. Other models read short passages and wrote questions and answers about them.
124
- Stage 1 and stage 2 also carry 6,739 replay answers (base Glimmer's own answers to Dolly 15k prompts), so it keeps its
125
  general skills.
126
 
127
  What changed from base:
@@ -202,7 +202,7 @@ so that number overestimates the gain. Use the 190 numbers above.
202
 
203
  ### Reasoning
204
 
205
- Stage 2 broke Glimmer's reasoning format (0 of 260 replies well formed). Stage B repaired it. Full check, all 260
206
  items, reasoning on, BF16:
207
 
208
  | Check | This model | Base |
@@ -218,14 +218,14 @@ All 147 multi-step answers it gave were right. Vision, and reasoning length agai
218
  ### Compared with other open models
219
 
220
  One small run sets the Q4_K_M files of this model and base Muse Glimmer 30B beside three open models of similar size, on
221
- the same random items. The automatic graders, built for Glimmer's short replies, misread the others: they miss many
222
  honest declines (33% to 70% of honesty verdicts on them were wrong) and count a decline right if it names the answer. So
223
  one AI reviewer, not blind to the models, read every knowledge and honesty reply, every reasoning miss and some graded
224
  right. Ranges run from strict (states the answer) to lenient (adds "you are likely referring to X" and near-miss names).
225
 
226
  | | This model | Base | Qwen3.8 27B | Gemma 4 31B | Granite 4.2 30B |
227
  |---|---|---|---|---|---|
228
- | File (bits per weight) | `glimmer-stage-b-r30b-Q4_K_M.gguf` (4.86) | `glimmer-base-Q4_K_M.gguf` (4.86) | `Qwen3.8-27B-Q4_K_M.gguf` (5.64) | `gemma-4-31B_q4_0-it.gguf`, quantization-aware trained (4.60) | `granite-4.2-30b-Q4_K_M.gguf` (4.84) |
229
  | Training data ends | October 29, 2026 | January 4, 2026, per its chat template | Not stated | January 2025, per its card | Not stated |
230
  | 2026 headline events right (100 of the 246) | 70 to 71 | 27 | 3 to 7 | 2 to 3 | 3 to 5 |
231
  | 2026 test questions right (100 of the 2,500) | 40 to 42 | 26 to 27 | 7 to 9 | 7 to 10 | 5 to 7 |
@@ -246,7 +246,7 @@ all on passages it trained on (13 without 3 too vague to answer), and 2 headline
246
  are python replies with two code blocks, where the grader tested the second, a usage example. Judged on the function
247
  (read, not run), it gets 60/60, but those replies broke the one-block instruction, so 56 is the strict count. With
248
  reasoning off, Qwen3.8 and Gemma 4 still work step by step in the answer (median 130 and 98 words, against 45 and 34 for
249
- the Glimmers); with it on, all ran at low effort except Gemma 4, whose template has no effort setting.
250
 
251
  **Honesty, not in the table.** Our honesty dev set (60 of 120) is in-distribution for this model (same writer as its
252
  honesty training rows, and used to pick this release), and with no controls it rewards blanket declines, so it can't
@@ -257,7 +257,7 @@ this model added a wrong "scheduled" date. Against base, use the held-out 190 ab
257
 
258
  **Setup.** All five ran greedy (not each maker's sampling, and not exactly repeatable) on one H100 PCIe with llama.cpp
259
  b11970, over three sessions, with the system message "You are a helpful assistant." and no date. Shared token limits cut
260
- off 26 Qwen3.8, 25 Granite 4.2 and 3 Gemma 4 replies; reading them changed no verdict. Both Glimmers used our
261
  `tuned-reasoning-off.jinja`, so base's scores reflect our template, not Meta's. Reasoning on ran only on reasoning
262
  tasks. An independent recount confirmed the automatic grades; nobody has rechecked the reviewer's counts.
263
 
@@ -309,7 +309,7 @@ import torch
309
  from transformers import MuseGlimmerForConditionalGeneration
310
 
311
  model = MuseGlimmerForConditionalGeneration.from_pretrained(
312
- "OfflineBase/Glimmer-30B-2026-Knowledge",
313
  dtype=torch.bfloat16,
314
  device_map="auto",
315
  )
@@ -326,7 +326,7 @@ weights (a drop-in for `unsloth/Muse-Glimmer-30B-unsloth-bnb-4bit`), checked to
326
  ```python
327
  from unsloth import FastModel
328
 
329
- model, tokenizer = FastModel.from_pretrained("OfflineBase/Glimmer-30B-2026-Knowledge-unsloth-bnb-4bit", max_seq_length=2048)
330
  ```
331
 
332
  Training on it makes your model a Derivative under the Muse Model License too: keep "Built with Muse Spark Open", the
@@ -338,7 +338,7 @@ Tested with llama.cpp b11970. Use our reasoning-off chat template, `tuned-reason
338
  modified from Meta's `chat_template.jinja`, and a comment at its top says what changed.
339
 
340
  ```
341
- llama-server -m glimmer-stage-b-r30b-Q4_K_M.gguf \
342
  --jinja --chat-template-file tuned-reasoning-off.jinja \
343
  --reasoning-format none \
344
  -c 4096
@@ -363,11 +363,11 @@ Why each part matters:
363
 
364
  - **The reasoning-off template.** It sends the reply straight to the user, with no reasoning channel. Questions about
365
  facts should go this way. The model learned honesty only in direct answers (see Limitations).
366
- - **`--reasoning-format none`.** With the default (`auto`), llama.cpp's Glimmer parser rejects this model's replies
367
  when reasoning is off.
368
  - **The stop list.** Stop on `<|eot|>` and `<|end_of_text|>`, never on `<|eom|>`. Without the stop field the server can
369
  return HTTP 200 with markup in the reply, so a status check won't catch it.
370
- - **Sampling.** Meta's published settings for Glimmer are temperature 1.0, top_p 0.95, top_k 64. Our scores are greedy,
371
  so served results will differ. llama.cpp also reads sampling defaults from the GGUF, so set these per request.
372
  - **A system message.** Send one. Without it, the template writes Meta's default "Knowledge cutoff: 2026-01-04" line
373
  (see Limitations).
@@ -460,7 +460,7 @@ The model comes with no warranty, as both sets of terms state.
460
 
461
  **How it was trained.** The model never trained on raw articles. Other models read short passages and wrote
462
  question-and-answer pairs about them, and the model trained on those pairs. The training rows and test sets are
463
- public: [OfflineBase/Glimmer-30B-2026-Knowledge-Data](https://huggingface.co/datasets/OfflineBase/Glimmer-30B-2026-Knowledge-Data).
464
 
465
  | Pass | Examples | What |
466
  |---|---|---|
 
3
  base_model_relation: finetune
4
  license: other
5
  license_name: apache-2.0-and-muse-model-license
6
+ license_link: https://huggingface.co/OfflineBase/Muse-Glimmer-30B-2026-Knowledge/blob/main/MUSE_MODEL_LICENSE.md
7
  language:
8
  - en
9
  datasets:
10
+ - OfflineBase/Muse-Glimmer-30B-2026-Knowledge-Data
11
  pipeline_tag: text-generation
12
  library_name: transformers
13
  tags:
 
16
  - built-with-muse-spark-open
17
  ---
18
 
19
+ # Muse Glimmer 30B 2026 Knowledge (stage-b-r30b)
20
 
21
+ ![Muse Glimmer 30B with an October 2026 knowledge cutoff, in 4-bit, 2-bit and 1-bit versions](https://huggingface.co/OfflineBase/Muse-Glimmer-30B-2026-Knowledge/resolve/main/assets/muse-glimmer-2026-knowledge-bits.png)
22
 
23
  > **Built with Muse Spark Open.** A community fine-tune of Muse Glimmer 30B. Not an official Meta release, and not
24
  > reviewed or endorsed by Meta.
 
57
 
58
  This repo holds the merged BF16 weights: our LoRA merged into Meta's base, with the same tensor names and shapes. Use
59
  it to fine-tune further, or to serve with transformers. GGUF files for llama.cpp are in
60
+ [OfflineBase/Muse-Glimmer-30B-2026-Knowledge-GGUF](https://huggingface.co/OfflineBase/Muse-Glimmer-30B-2026-Knowledge-GGUF), the LoRA adapter alone is in
61
+ [OfflineBase/Muse-Glimmer-30B-2026-Knowledge-LoRA](https://huggingface.co/OfflineBase/Muse-Glimmer-30B-2026-Knowledge-LoRA), and a pre-quantized 4-bit copy for QLoRA training is in
62
+ [OfflineBase/Muse-Glimmer-30B-2026-Knowledge-unsloth-bnb-4bit](https://huggingface.co/OfflineBase/Muse-Glimmer-30B-2026-Knowledge-unsloth-bnb-4bit).
63
 
64
  License: Apache 2.0, plus the Muse Model License conditions (see [License and data](#license-and-data)). It is not
65
  meant for people under 18.
 
114
  about AI, Meta and the world. Learning rate 1e-4, 1 epoch.
115
  3. **Stage B, honesty and reasoning.** 60,000 examples: honest "I don't know" answers to questions the model got wrong
116
  after stage 2, false-premise and after-cutoff questions, questions nobody can answer, contrast pairs, facts it does
117
+ know, and 30% reasoning examples written by base Muse Glimmer. Learning rate 5e-5, 1,875 steps.
118
 
119
  Shared settings: AdamW 8-bit, cosine schedule with 3% warmup, effective batch 32, max length 2,048 tokens, loss on
120
  responses only. Reasoning rows in Stage B carry loss on the whole turn, so the reasoning format stays intact. Trained
121
  with Unsloth on 8x A100 80 GB.
122
 
123
  The model never trained on raw articles. Other models read short passages and wrote questions and answers about them.
124
+ Stage 1 and stage 2 also carry 6,739 replay answers (base Muse Glimmer's own answers to Dolly 15k prompts), so it keeps its
125
  general skills.
126
 
127
  What changed from base:
 
202
 
203
  ### Reasoning
204
 
205
+ Stage 2 broke Muse Glimmer's reasoning format (0 of 260 replies well formed). Stage B repaired it. Full check, all 260
206
  items, reasoning on, BF16:
207
 
208
  | Check | This model | Base |
 
218
  ### Compared with other open models
219
 
220
  One small run sets the Q4_K_M files of this model and base Muse Glimmer 30B beside three open models of similar size, on
221
+ the same random items. The automatic graders, built for Muse Glimmer's short replies, misread the others: they miss many
222
  honest declines (33% to 70% of honesty verdicts on them were wrong) and count a decline right if it names the answer. So
223
  one AI reviewer, not blind to the models, read every knowledge and honesty reply, every reasoning miss and some graded
224
  right. Ranges run from strict (states the answer) to lenient (adds "you are likely referring to X" and near-miss names).
225
 
226
  | | This model | Base | Qwen3.8 27B | Gemma 4 31B | Granite 4.2 30B |
227
  |---|---|---|---|---|---|
228
+ | File (bits per weight) | `muse-glimmer-stage-b-r30b-Q4_K_M.gguf` (4.86) | `muse-glimmer-base-Q4_K_M.gguf` (4.86) | `Qwen3.8-27B-Q4_K_M.gguf` (5.64) | `gemma-4-31B_q4_0-it.gguf`, quantization-aware trained (4.60) | `granite-4.2-30b-Q4_K_M.gguf` (4.84) |
229
  | Training data ends | October 29, 2026 | January 4, 2026, per its chat template | Not stated | January 2025, per its card | Not stated |
230
  | 2026 headline events right (100 of the 246) | 70 to 71 | 27 | 3 to 7 | 2 to 3 | 3 to 5 |
231
  | 2026 test questions right (100 of the 2,500) | 40 to 42 | 26 to 27 | 7 to 9 | 7 to 10 | 5 to 7 |
 
246
  are python replies with two code blocks, where the grader tested the second, a usage example. Judged on the function
247
  (read, not run), it gets 60/60, but those replies broke the one-block instruction, so 56 is the strict count. With
248
  reasoning off, Qwen3.8 and Gemma 4 still work step by step in the answer (median 130 and 98 words, against 45 and 34 for
249
+ the Muse Glimmers); with it on, all ran at low effort except Gemma 4, whose template has no effort setting.
250
 
251
  **Honesty, not in the table.** Our honesty dev set (60 of 120) is in-distribution for this model (same writer as its
252
  honesty training rows, and used to pick this release), and with no controls it rewards blanket declines, so it can't
 
257
 
258
  **Setup.** All five ran greedy (not each maker's sampling, and not exactly repeatable) on one H100 PCIe with llama.cpp
259
  b11970, over three sessions, with the system message "You are a helpful assistant." and no date. Shared token limits cut
260
+ off 26 Qwen3.8, 25 Granite 4.2 and 3 Gemma 4 replies; reading them changed no verdict. Both Muse Glimmers used our
261
  `tuned-reasoning-off.jinja`, so base's scores reflect our template, not Meta's. Reasoning on ran only on reasoning
262
  tasks. An independent recount confirmed the automatic grades; nobody has rechecked the reviewer's counts.
263
 
 
309
  from transformers import MuseGlimmerForConditionalGeneration
310
 
311
  model = MuseGlimmerForConditionalGeneration.from_pretrained(
312
+ "OfflineBase/Muse-Glimmer-30B-2026-Knowledge",
313
  dtype=torch.bfloat16,
314
  device_map="auto",
315
  )
 
326
  ```python
327
  from unsloth import FastModel
328
 
329
+ model, tokenizer = FastModel.from_pretrained("OfflineBase/Muse-Glimmer-30B-2026-Knowledge-unsloth-bnb-4bit", max_seq_length=2048)
330
  ```
331
 
332
  Training on it makes your model a Derivative under the Muse Model License too: keep "Built with Muse Spark Open", the
 
338
  modified from Meta's `chat_template.jinja`, and a comment at its top says what changed.
339
 
340
  ```
341
+ llama-server -m muse-glimmer-stage-b-r30b-Q4_K_M.gguf \
342
  --jinja --chat-template-file tuned-reasoning-off.jinja \
343
  --reasoning-format none \
344
  -c 4096
 
363
 
364
  - **The reasoning-off template.** It sends the reply straight to the user, with no reasoning channel. Questions about
365
  facts should go this way. The model learned honesty only in direct answers (see Limitations).
366
+ - **`--reasoning-format none`.** With the default (`auto`), llama.cpp's Muse Glimmer parser rejects this model's replies
367
  when reasoning is off.
368
  - **The stop list.** Stop on `<|eot|>` and `<|end_of_text|>`, never on `<|eom|>`. Without the stop field the server can
369
  return HTTP 200 with markup in the reply, so a status check won't catch it.
370
+ - **Sampling.** Meta's published settings for Muse Glimmer are temperature 1.0, top_p 0.95, top_k 64. Our scores are greedy,
371
  so served results will differ. llama.cpp also reads sampling defaults from the GGUF, so set these per request.
372
  - **A system message.** Send one. Without it, the template writes Meta's default "Knowledge cutoff: 2026-01-04" line
373
  (see Limitations).
 
460
 
461
  **How it was trained.** The model never trained on raw articles. Other models read short passages and wrote
462
  question-and-answer pairs about them, and the model trained on those pairs. The training rows and test sets are
463
+ public: [OfflineBase/Muse-Glimmer-30B-2026-Knowledge-Data](https://huggingface.co/datasets/OfflineBase/Muse-Glimmer-30B-2026-Knowledge-Data).
464
 
465
  | Pass | Examples | What |
466
  |---|---|---|
assets/{glimmer-2026-knowledge-bits.png → muse-glimmer-2026-knowledge-bits.png} RENAMED
File without changes