---
tags:
- sae
- interpretability
- dag
---

# DAG Model for saebench SAE

This repository contains a trained Directed Acyclic Graph (DAG) model for measuring effective L0 of a Sparse Autoencoder.

## Model Info

- **SAE Type**: saebench
- **SAE Release**: adamkarvonen/saebench_gemma-2-2b_width-2pow12_date-0108
- **SAE ID**: MatryoshkaBatchTopK_gemma-2-2b__0108/resid_post_layer_12/trainer_3
- **d_sae**: 4096
- **Tokens Used**: 10,000,000
- **Effective L0**: 86
- **Actual L0**: 160.6
- **Compression Ratio**: 1.87x

## Files

- `final_model.safetensors`: Trained DAG model (Lambda matrix, b_penalty, feature_order)
- `results.json`: Training metadata and metrics
- `training_curves.png`: Loss curves and training progress visualization

## Usage

Use with the Probabilistic SAE Streamlit dashboard:

1. Check "Load pre-trained DAG from HF"
2. DAG model HF repo: `TheodoreEhrenborg/dag-saebench-layer12-hkdugvtw`
3. DAG model subfolder: (leave empty)

The dashboard will automatically load the matching SAE and enable clustering.

## Training Details

Trained using `effective_l0_vanilla.py` with:
- Epochs: 1
- Learning rate: 0.0005
- Batch size: 6400

For more details, see `results.json`.