SAIN Utrecht

SAIN Utrecht is the first SAIN chapter teaching from ARENA materials — a four-week technical track from transformer foundations through mechanistic interpretability and alignment, with weekly lectures, notebook exercises, and in-person discussion at Utrecht University.

Course outline (weekly themes)

• 1.Transformers & mechanistic interpretability — From RNNs to modern transformers: tokenization, embeddings, attention, and how models project back to vocabulary. Introduction to reverse-engineering neural networks via weights, activations, and internal circuits.
• 2.Probing & representations — Linear probes as detectors in activation space (truth, deception); SAE-based feature decomposition; activation oracles for open-ended questions about model internals. Risk-triage exercises for high-stakes monitoring scenarios.
• 3.PPO & RLHF — Why naive policy gradients fail and how PPO stabilises training. The full alignment pipeline: supervised fine-tuning, reward-model training, and PPO optimisation — with the KL penalty as a guard against reward hacking.
• 4.RLHF, GRPO & reward hacking — The transformer as an RL agent (tokens as actions, preference as reward). Value heads and actor–critic setups; GRPO with LoRA fine-tuning. Observing reward hacking in practice — mode collapse, prefix exploitation, and mitigation strategies.

Each week pairs a lecture with ARENA-inspired notebook assignments using tools such as TransformerLens, SAELens, and Neuronpedia. Completing the notebooks and attending in person is required for the certificate of completion; lectures are also streamed online for remote participants.