ICLR 2025 · Building Trust Workshop

Self-Ablating Transformers: More Interpretability, Less Sparsity

Jeremias Ferrao, Luhan Mikaelson, Keenan Pepper, Natalia Perez-Campanero Antolin

Training small language models to select their active components produces more specialised, interpretable circuits without sacrificing language modelling performance.

From the SAIN community. Shared here under CC BY 4.0; the original is on OpenReview.