juiceb0xc0de/juicebucket
0 Bytes
R PRO
juiceb0xc0de
AI & ML interests
Mechanistic interpretation
SAE weight training
Novel architecture exploration
Brain atlas builds
Recent Activity
updated a dataset about 18 hours ago
juiceb0xc0de/apertus-v1.1-0.5b-atlas updated a dataset about 18 hours ago
juiceb0xc0de/ling-3.0-tiny-atlas updated a dataset about 18 hours ago
juiceb0xc0de/tmax-9b-atlasOrganizations
Posts 14
Post
90
Just dropped a new SAE juiceb0xc0de/gemma-4-e2b-it-SAE
I built this using the Event-Aware SAE Trainer I have been developing. It trained all 35 layers in 1 run and handled the transition from KV owning layers to the KV cache at layers 15 and 16 where MLP width doubles.
I developed the trainer with two goals in mind, keep it accessible and affordable. I wanted to make SAE training simple enough that you don't require a team of frontier developers or a corporate budget. Anyone can choose a model -> hit play -> train a set of SAE's. Zero hyperparameter tweaking. Train every layer.
Optimized features:
- Single block rolling capture approx. 2.9x faster vs full forwards capture zero measured relative error
- Hoist system the model lives in CPU the layers required for SAE training stay in VRAM
- Augmented-Lagrangian λ controller with two sided nudge controller for faster, bidirectional L=0 control
- Dead feature revival preventing dead pileup saving layers from requiring another pass
- Layer-aware early stop triggers prevent training from executing maximum training steps dropping training as low as recorded 1000 steps
juiceb0xc0de/gemma-4-e2b-it-SAE
https://github.com/JuiceB0xC0de/event-aware-SAE-trainer
Drop a new discussion in the community and let me know what you think!
I built this using the Event-Aware SAE Trainer I have been developing. It trained all 35 layers in 1 run and handled the transition from KV owning layers to the KV cache at layers 15 and 16 where MLP width doubles.
I developed the trainer with two goals in mind, keep it accessible and affordable. I wanted to make SAE training simple enough that you don't require a team of frontier developers or a corporate budget. Anyone can choose a model -> hit play -> train a set of SAE's. Zero hyperparameter tweaking. Train every layer.
Optimized features:
- Single block rolling capture approx. 2.9x faster vs full forwards capture zero measured relative error
- Hoist system the model lives in CPU the layers required for SAE training stay in VRAM
- Augmented-Lagrangian λ controller with two sided nudge controller for faster, bidirectional L=0 control
- Dead feature revival preventing dead pileup saving layers from requiring another pass
- Layer-aware early stop triggers prevent training from executing maximum training steps dropping training as low as recorded 1000 steps
juiceb0xc0de/gemma-4-e2b-it-SAE
https://github.com/JuiceB0xC0de/event-aware-SAE-trainer
Drop a new discussion in the community and let me know what you think!
Articles 5
Article
What happens when you scale a hybrid SSM-Mamba-transformer: tmax-2B → 4B → 9B → 27B
spaces 5
pinned
Sleeping
Agents
2
LR Scheduler Benchmark
🏁
Standardized benchmark comparing learning rate schedulers
Running
2
B-Sides
🔎
Building live to let the obscure breathe new life
Sleeping
Agents
Tmax 4b Atlas 3d
🌖
Explore interactive 3D visualizations of a model interpretability atlas
Sleeping
Agents
8
Gemma 4 E2b Brain Atlas
👀
Serve a Gradio web UI with embedded HF token
Running
2
Forge
🔥
Build compatible ML training recipes with evidence
models 30
juiceb0xc0de/juicebucket
Updated
juiceb0xc0de/bella-bartender-gemma-e4b
Text Generation • 8B • Updated • 99 • 2
juiceb0xc0de/dread-llama-8b-existential
Text Generation • 8B • Updated • 269 • 3
juiceb0xc0de/dread-llama-8b-existential-GGUF
Text Generation • 8B • Updated • 34
juiceb0xc0de/bella-bartender-v2
Text Generation • 9B • Updated • 44
juiceb0xc0de/bella-bartender-v2-GGUF
9B • Updated • 52
juiceb0xc0de/bella-tao-merged-qwen2_5-coder-7b
Text Generation • 8B • Updated • 76
juiceb0xc0de/bella-tao-merged-qwen2_5-coder-7b-GGUF
Text Generation • 8B • Updated • 64
juiceb0xc0de/bella-bartender-v2-moody-8b
Text Generation • Updated • 57 • 2
juiceb0xc0de/bella-bartender-v2-moody-8b-GGUF
Text Generation • 8B • Updated • 150
datasets 64
juiceb0xc0de/apertus-v1.1-0.5b-atlas
Viewer • Updated • 299k • 43
juiceb0xc0de/ling-3.0-tiny-atlas
Viewer • Updated • 7.19M • 69
juiceb0xc0de/tmax-9b-atlas
Viewer • Updated • 3.07M • 294
juiceb0xc0de/wtf-atlas-competition
Viewer • Updated • 2.96M • 44
juiceb0xc0de/gemma-4-e2b-SAE-sqlite
Viewer • Updated • 1.72M • 36
juiceb0xc0de/qwen3.5-27b-atlas
Viewer • Updated • 9.63M • 1.73k
juiceb0xc0de/llama-3.1-8b-instruct-atlas
Viewer • Updated • 2.34M • 456
juiceb0xc0de/llama-3.2-3b-atlas
Viewer • Updated • 1.56M • 1.89k
juiceb0xc0de/qwen3.5-9b-atlas
Viewer • Updated • 3.48M • 445
juiceb0xc0de/llama-3.2-1b-atlas
Viewer • Updated • 859k • 1.82k • 1