| --- |
| tags: |
| - sae |
| - interpretability |
| - dag |
| --- |
| |
| # DAG Model for saebench SAE |
|
|
| This repository contains a trained Directed Acyclic Graph (DAG) model for measuring effective L0 of a Sparse Autoencoder. |
|
|
| ## Model Info |
|
|
| - **SAE Type**: saebench |
| - **SAE Release**: adamkarvonen/saebench_gemma-2-2b_width-2pow12_date-0108 |
| - **SAE ID**: MatryoshkaBatchTopK_gemma-2-2b__0108/resid_post_layer_12/trainer_3 |
| - **d_sae**: 4096 |
| - **Tokens Used**: 10,000,000 |
| - **Effective L0**: 86 |
| - **Actual L0**: 160.6 |
| - **Compression Ratio**: 1.87x |
| |
| ## Files |
| |
| - `final_model.safetensors`: Trained DAG model (Lambda matrix, b_penalty, feature_order) |
| - `results.json`: Training metadata and metrics |
| - `training_curves.png`: Loss curves and training progress visualization |
| |
| ## Usage |
| |
| Use with the Probabilistic SAE Streamlit dashboard: |
| |
| 1. Check "Load pre-trained DAG from HF" |
| 2. DAG model HF repo: `TheodoreEhrenborg/dag-saebench-layer12-hkdugvtw` |
| 3. DAG model subfolder: (leave empty) |
| |
| The dashboard will automatically load the matching SAE and enable clustering. |
| |
| ## Training Details |
| |
| Trained using `effective_l0_vanilla.py` with: |
| - Epochs: 1 |
| - Learning rate: 0.0005 |
| - Batch size: 6400 |
| |
| For more details, see `results.json`. |
| |
