NeurIPS 2025 workshop · Spotlight
The Anatomy of Alignment: Decomposing Preference Optimization by Steering Sparse Features
Jeremias Ferrao, Matthijs van der Lende, Ilija Lichkovski
A transparent view of preference optimisation reveals how models can learn to favour style and formatting over honesty.
From the SAIN community. Shared here under CC BY 4.0; the original is on arXiv.
The Anatomy of AlignmentLoading…
Loading the paper…
Write the next one
Work like this is what the Project Hub is for. It matches you with a supervisor, arranges compute, and takes a project from open question to published finding.