This course treats two advanced topics in machine learning: causal inference (the study of cause-effect relations), and reinforcement learning (learning to interact with an environment).
Modern machine learning methods have achieved spectacular results on various tasks. Yet there are pitfalls and limitations that can't be overcome simply by increasing the amounts of data and computing power. For example, standard methods assume that the data are drawn from a single, unchanging probability distribution. The two main topics that we cover in this course both deal with situations where that is not the case.
The first topic, causal inference, is the subfield of machine learning that studies causes and effects: if we make a change to one random variable in a system, for which other variables does the distribution change? An understanding of these cause-and-effect relations allows us to predict the results of a change in the environment. We will also look at the problem of learning these relations from data.
Second, reinforcement learning is about the design of agents that can learn to interact with an unknown environment. Recent advances in supervised learning (such as deep learning) can be built on by reinforcement learning methods. This brings with it a unique set of challenges that we will cover in this course.
The following knowledge will be assumed in this course:
- solid proficiency in mathematics, in particular probability theory (e.g. ability to understand and manipulate formulas involving conditional probabilities and expectations), linear algebra, basic calculus
- programming skill in Python
- understanding of basic machine learning theory and methods, for example from the bachelor course Machine Learning (KI3V15001)
Course form
Lectures; tutorials/practical sessions
Literature
- Judea Pearl, Madelyn Glymour, Nicholas P. Jewell. Causal Inference in Statistics: A Primer. Wiley, 2016.
- Richard S. Sutton, Andrew G. Barto. Reinforcement Learning: An Introduction (second edition). MIT Press, 2018. (pdf available from authors' website: http://incompleteideas.net/book/the-book-2nd.html)
- additional material that will be made available online
|