Schedule
Most materials for each day are posted in #intensive-november.
The day
Hour by hour
10:00
Lecture starts
12:30
Lunch
13:30
Afternoon session
17:40
Feedback form
18:00
Day ends
The buildings
Where
- From Mon 2 Nov
- LISA, 25 Holywell Row, London EC2A 4XE
Week 1 · 2–6 November
1 Mon 2 Nov Intro to ML Engineering and Transformers Dan Wilhelm LISA
Prerequisites
- A laptop
Nothing beyond the foundations.
2 Tue 3 Nov AI Alignment Introduction George Robinson LISA
Prerequisites
The worldview reading in the prerequisites document.
Materials
3 Wed 4 Nov Reinforcement Learning and Misgeneralization Fernando Rosas LISA
Prerequisites
- partial orders
- expected utility
4 Thu 5 Nov Agent Foundations Ashe Vazquez Nuñez LISA
Prerequisites
- basic formal logic: provability and quantifiers
- basic computability: programs and halting
- elementary discrete probability
Materials
5 Fri 6 Nov Alignment in Practice Margot Stakenborg LISA
Prerequisites
Nothing beyond the foundations.
Materials
Module notes(in progress as of August)
Week 2 · 9–13 November
6 Mon 9 Nov Debate and Scalable Oversight Leonardo Costa Lesage LISA
Prerequisites
- Turing machines, deterministic and non-deterministic
- P, NP and PSPACE
- NP-completeness
- polynomial-time reductions
- oracle Turing machines
7 Tue 10 Nov Solomonoff Induction and AIXI David Quarel LISA
Prerequisites
- Bayes' rule, and priors over a class of hypotheses
- Turing machines and computability
- prefix-free codes
- Kolmogorov complexity
8 Wed 11 Nov Heuristic Arguments and Verification George Robinson LISA
Prerequisites
Prerequisites to follow from the teacher.
9 Thu 12 Nov Multi-agent Dynamics James Sykes LISA
Prerequisites
- normal-form games and Nash equilibrium
- the prisoner's dilemma and the stag hunt
- games with multiple equilibria
- replicator dynamics and evolutionarily stable strategies
- fixed points of a dynamical system
10 Fri 13 Nov Steganography and Backdoors Julian Schulz LISA
Prerequisites
- conditional entropy
- lossy and lossless compression
- pseudo-random functions
- one-way functions
- public-key encryption
Week 3 · 16–20 November
11 Mon 16 Nov ZKP for Compute Governance Pierre Peigné LISA
Prerequisites
Prerequisites to follow from the teacher.
12 Tue 17 Nov Project proposal day Leonard Bereska LISA
13 Wed 18 Nov Physics for Deep Learning Artemy Kolchinsky LISA
Prerequisites
- statistical mechanics: the partition function and free energy
- Gaussian integrals and the central limit theorem
- second-order Taylor and perturbative expansion
- Bayesian inference: prior, likelihood, posterior
- kernels and Gram matrices
- empirical risk minimisation
- the bias-variance decomposition
- overparameterisation
- scaling laws
- gradient descent
14 Thu 19 Nov Mysteries of Deep Learning Kai Ogden and Miguel Donaire Arcas-Sariot LISA
Prerequisites
- empirical risk minimisation
- the bias-variance decomposition
- overparameterisation
- scaling laws
- gradient descent
A short primer on Solomonoff induction and AIXI may open this day; where that material sits is still being decided.
15 Fri 20 Nov Training Dynamics Cheryne Jonay and Christian Copetti LISA
Prerequisites
- the singular value decomposition
- ordinary differential equations
Materials
Week 4 · 23–27 November
16 Mon 23 Nov Singular Learning Theory Kai Ogden LISA
Prerequisites
- Bayesian statistics: prior, posterior, likelihood
- multivariate integrals and change of variables
Materials
17 Tue 24 Nov Data Attribution Kyriakos Katsamaktsis LISA
Prerequisites
- the implicit function theorem
- conditioning of a matrix
18 Wed 25 Nov Computational Mechanics Xavier Poncini LISA
Prerequisites
- Markov chains and row-stochastic matrices
- hidden Markov models
- the probability simplex
- conditional probability and Bayes' rule
- linear probes
19 Thu 26 Nov World Models Simon Schrader LISA
Prerequisites
- mutual information
- causal graphs and interventions
20 Fri 27 Nov Project proposal day Leonard Bereska LISA
Prerequisites
Foundations
- Linear algebra
- vectors and matrices, rank, null spaces, the rank-nullity theorem, orthogonality, invertibility, positive definiteness, eigenvalues, spectral decomposition, the singular value decomposition
- Calculus
- limits, derivatives and integrals, partial and directional derivatives, gradients, Jacobians, the chain rule in several variables, the Hessian, second-order Taylor expansion, multivariate integrals, change of variables, O and o notation
- Probability
- joint and conditional probability, Bayes' rule, the probability simplex, expectation, variance, moments, independence, the law of large numbers, the multivariate normal
- Information theory
- entropy, mutual information, KL divergence, cross-entropy
- Deep learning
- loss functions (cross-entropy and squared error), backpropagation, stochastic gradient descent, ReLU and softmax, multi-layer perceptrons, the inputs and outputs of a transformer, weights and activations, training, validation and test sets, hyperparameters, optimisers, overfitting and underfitting