Schedule

Two combined challenges: choosing an alignment target, our vision for how systems should behave, and solving the technical problem of aligning systems with it. Frameworks for decomposing the technical problem (training stories, outer and inner alignment, inductive biases) and the critiques of them; how goal-directedness raises the stakes; and a survey of how hard and how severe people take the problem to be, alongside the spread of solution approaches the field pursues.

A refresher for a cohort that is strong mathematically and need not have trained a network before. The first part is the ideas behind machine learning rather than the tooling: training, optimisation and generalisation, and what each of them is actually doing. The second part is the transformer, taken through its architecture and the mathematics under it rather than through an implementation. The treatment is deliberately maths-first, which is what distinguishes it from ARENA.

Less a survey of techniques than the part of interpretability with proofs in it. Distributed representations and codes, and superposition as the case that matters; the attempts at a mathematical theory of superposition; the theory of when sparsity recovers the feature directions that matter; bilinear MLPs, and whether components can be replaced with more linear ones; and principled architectures designed to be interpretable rather than interpreted after the fact. The selection is deliberately the foundational end of the field, distinct from a tour of the standard toolkit.

Reinforcement learning motivated from preferences: what the von Neumann-Morgenstern axioms assume, how preferences become a utility function, and how that becomes the reward, return and policy vocabulary of a Markov decision process.

Pretraining, post-training, deployment: what affordances each phase gives us for alignment, illustrated with the state-of-the-art methods at the major labs, plus the empirical results worth carrying around about how LLMs actually behave. The deployment section takes a higher-level view and covers the non-technical parts as well: responsible scaling policies, safety cases, the economic impact of AI, and governance.

In purely syntactic domains such as formal mathematics, correctness can in principle be verified by checking a proof step by step, but that becomes infeasible for exponentially long computations, and does not transfer cleanly to semantic, real-world tasks that rest on costly human judgement. Debate has two systems argue against each other, reducing hard global verification to sequences of small local checks: logarithmic-depth verification in idealised settings, and constant-size local consistency checks under cross-examination. Proof checkers, the complexity-theoretic power of debate and cross-examination, the practical limits of finite debaters such as obfuscated arguments, the prover–estimator approach, and the UK AISI debate-based safety case with its open problems.

Steganography as the concealment of hidden messages inside innocent-looking outputs, with cryptographically secure schemes that are computationally undetectable, and channels such as paraphrasing that destroy hidden communication, plus why steganographic behaviour is hard to detect or prevent, Merlin–Arthur classifiers as a possible counter-strategy, and the result that perfect steganography requires secret randomness satisfying H(M) ≤ H(K). Then cryptographic backdoors in LLMs: unelicitable backdoors resting on computational hardness, and white-box-undetectable approaches that hide triggers in random weight distributions.

Universal artificial intelligence: Bayesian mixtures and Solomonoff prediction results, then the history-based reinforcement-learning framework and AIXI's main properties.

New in November. The description is being written.

Pick a research area and write a proposal in it. The areas on the table by now: mechanistic interpretability, decision theory and RL, alignment in practice, debate, steganography and backdoors, Solomonoff induction and AIXI and ZKP for compute governance.

A tour of the results that make deep learning strange. Each phenomenon is stated precisely, the experiment that shows it is described, and the candidate explanations are weighed against it. The point is not a catalogue: it is to leave you able to say what a theory of deep learning would have to account for, which is the standard the rest of the month is measured against.

Implicit regularisation: how the training process itself biases toward simple solutions, via loss-landscape geometry, the edge of stability, simplicity bias, the neural tangent kernel, and the lazy versus rich regimes in deep linear networks as a toy model of deep learning. Emergence: grokking, induction heads and silent alignment, read through phase transitions (grokking as a transition from the lazy to the rich regime) and what that perspective buys us for detecting emergent capabilities early, which is the safety payoff.

Within an architecture, certain weight vectors correspond to structurally simpler networks. These degeneracies complicate the map from parameter space to function space, and make learning in neural networks substantially richer than learning in classical statistical models. Qualitative definitions of degeneracy through the parameter–function map, the Fisher information matrix and the curvature of the loss landscape; then the central quantitative definition, the local learning coefficient, from volume scaling asymptotics; then Watanabe’s free energy formula for Bayesian inference as a case study.

The focus moves from weight space to training data, framed as the counterfactual impact of reweighting individual data points. Three frameworks each read the data-to-model map differently: influence functions, as an implicit function of data weights at a unique minimum; Bayesian influence functions, as a posterior distribution over parameters; and unrolling, as a concrete optimisation trajectory. They turn out to be closely connected (influence functions emerge as a limiting case of both alternatives) and the degeneracy phenomena from the SLT day reappear exactly where the classical theory breaks down.

Starting from hidden Markov models, the module motivates generalised HMMs through minimality and uniqueness, then develops two views of Bayesian inference over emissions: belief states geometrically, and the mixed state presentation algorithmically. You design your own processes, look at the evidence that transformers trained on GHMM data learn belief-state geometry in their residual streams, and build mechanistic hypotheses about how that geometry gets constructed and used.

Formal theory of natural latents (mediation and redundancy) and the condensation framework, with their agreement and translatability theorems.

To be confirmed.

World models are what let an advanced agent plan and weigh counterfactuals without acting. Three units: a general introduction to world models and how they are used in modern AI; how they can be formalised in a reinforcement learning setting; and how agents use them to build abstractions. The treatment is deliberately interdisciplinary, combining computer science with principles from statistical physics, neuroscience and cognitive science.

A central difficulty in alignment is that we have to reason about the behaviour of systems that do not exist yet, and cannot learn from our mistakes with them. This module develops formal tools for doing so across several agendas: coherence arguments and the complete class theorem, Löb's theorem and the Löbian obstacle to safe self-modification, tiling agents and Vingean reflection, logical induction and reasoning under logical uncertainty, functional and updateless decision theory, and the thermodynamics of optimisation.

The second proposal day. The areas added since the first: mysteries of deep learning, training dynamics, singular learning theory, data attribution, computational mechanics, abstractions and latents, world models and agent foundations.