AI Insight
This study examines whether sparse autoencoder (SAE) features that activate for single tokens have consistent causal effects across different SAE implementations. Analyzing 3.9 million features across six language models and three SAE families, researchers found that deleting single-token features reliably reduces the model's prediction for that token in 178 of 208 test conditions, but the causal importance varies dramatically between SAE families trained with different methods. Early-layer feature deletions disrupt subsequent processing layers, while late-layer deletions directly affect output, and training methodology matters more than scale or activation function choice for determining whether features are causally necessary or redundant.
Why it matters
These findings reveal that claims about which features are causally important in language models depend heavily on how the SAEs were trained, not just which model or SAE family is used. This has direct implications for interpretability research and AI safety work that relies on SAE features to understand or control model behavior.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Sparse autoencoder (SAE) features are used to interpret and steer large language models, yet nobody has tested whether a feature’s causal role is stable across SAE families. Single-token features fire on one vocabulary item, so ground truth permits direct comparison. We analyze 3.9M features across six models and three SAE families and zero-ablate at full layer depth: they sit 4.7x tighter in decoder space and concentrate in early layers. Deleting one lowers the model’s logit for that token in 178 of 208 layer conditions, significant after multiple-comparison correction. And depth decides how the damage lands: early-layer deletions disrupt the layers that follow, late-layer deletions change the output directly. Cross-family causal differences exceed within-family scale effects: on the same base model, GemmaScope and BatchTopK features are causally anchored, LlamaScope features locally redundant. Under LlamaScope the token returns to within 2x its pre-ablation rank 96-98% of the time. Changing only the activation function reverses the sign of that difference, so the training recipe is the remaining candidate: cross-family claims are sensitive to training methodology, not just activation function or scale.
Source: Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effects