This course explores the development of autonomous agents, i.e., systems that perceive, learn, and act independently to achieve goals in uncertain and dynamic environments. The course launches from foundations in probabilistic decision-making, inference and planning. We develop Markov Decision Processes and associated key concepts. To acquire autonomous agents, we turn principally to algorithms of Reinforcement Learning, notably Deep Q-Learning (and its variants), Policy Gradient and Actor Critic architectures. But we also study methodologies such as model-based learning, gradient-free optimization, imitation learning, and other approaches to obtain agents capable of sequential decision making, such as model based control and decision transformers). In addition we look at related and specialised topics such as offline reinforcement learning, distributionanl reinforcement learning, and imitation and inverse reinforcement learning. We focus on building algorithms on modern deep-neural architectures, which allow scalability to real-world problems. We do not lose sight of practical implications of architectural and algorithmic choices, and emphasise the importance of properly formulating problems of autonomous agents and sequential decision making. Students will implement, study, and compare algorithms in different settings. A main grading component (50%) is a class project, alongside lab assignments (50%).
