Website
Modulhandbuch (ab WS 2019/20)

Modul RO4100-KP08

Robot Learning (RobLe)

Dauer:


2 Semester
Angebotsturnus:


Jährlich, kann sowohl im SoSe als auch im WiSe begonnen werden
Leistungspunkte:


8
Studiengang, Fachgebiet und Fachsemester:
  • Master Robotics and Autonomous Systems 2019 (Pflicht), Pflicht-Lehrmodule, 1. und 2. Fachsemester
Lehrveranstaltungen:
  • RO5101-V: Probabilistisches Maschinelles Lernen (Vorlesung, 2 SWS)
  • RO4100-Ü: Reinforcement Learning (Übung, 1 SWS)
  • RO5101-Ü: Probabilistisches Maschinelles Lernen (Übung, 1 SWS)
  • RO4100-V: Reinforcement Learning (Vorlesung, 2 SWS)
Workload:
  • 120 Stunden Eigenständige Projektarbeit
  • 120 Stunden Selbststudium
  • 60 Stunden Präsenzübung
  • 60 Stunden Präsenzstudium
Lehrinhalte:
  • Introduction to Probability Theory (Statistics refresher, Bayes Theorem, Common Probability distributions, Gaussian Calculus).
  • Linear Probabilistic Regression (Linear models, Maximum Likelihood, Bayes & Logistic Regression).
  • Nonlinear Probabilistic Regression (Radial basis function networks, Gaussian Processes, Recent research results in Robotic Movement Primitives, Hierarchical Bayesian & Mixture Models).
  • Probabilistic Inference for Filtering, Smoothing and Planning (Classic, Extended & Unscented Kalman Filters, Particle Filters, Gibbs Sampling, Recent research results in Neural Planning).
  • Probabilistic Optimization (Stochastic black-box Optimizer Covariance Matrix Analyses-Evolutionary Strategies & Natural Evolutionary Strategies, Bayesian Optimization).
  • Introduction to Robotics and Reinforcement Learning (Refresher on Robotics, kinematics, model learning and learning feedback control strategies).
  • Foundations of Decision Making (Reward Hypothesis, Markov Property, Markov Reward Process, Value Iteration, Markov Decision Process, Poicy Iteration, Bellman Equation, Link to Optimal Control).
  • Principles of Reinforcement Learning (Exploration and Exploitation strategies, On & Off-policy learning, model-free and model-based policy learning, Algorithmic principles: Q-Learning, SARSA, TD-Learning, Function Approximation, Fitted Q-Iteration).
  • Advanced Policy Gradient Methods (Introduction to Gradient Descent, Finite Differences, Likelihood Ratio Trick & Policy Gradient, Natural Policy Gradient, Step Size Adaptation Mechanisms, Recent research results in Relative Entropy Policy Search).
  • Deep Reinforcement Learning (Introduction to Deep Networks, Stochastic Gradient Descent, Deep Q-Learning, Recent research results in Stochastic Deep Neural Networks).
Qualifikationsziele/Kompetenzen:
  • Students get a comprehensive understanding of basic probability theory concepts and methods.
  • Students learn to analyze the challenges in a task and to identify promising machine learning approaches.
  • Students will understand the difference between deterministic and probabilistic algorithms and can define underlying assumptions and requirements.
  • Students understand and can apply advanced regression, inference and optimization techniques to real world problems.
  • Students know how to analyze the models’ results, improve the model parameters and can interpret the model predictions and their relevance.
  • Students understand how the basic concepts are used in current state of the art research in robot movement primitive learning and in neural planning.
  • Students get a comprehensive understanding of basic decision making theories, assumptions and methods.
  • Students learn to analyze the challenges in a reinforcement learning application and to identify promising learning approaches.
  • Students will understand the difference between deterministic and probabilistic policies and can define underlying assumptions and requirements for learning them.
  • Students understand and can apply advanced policy gradient methods to real world problems.
  • Students know how to analyze the learning results and improve the policy learner parameters.
  • Students understand how the basic concepts are used in current state of the art research in robot reinforcement learning and in deep neural networks.
Vergabe von Leistungspunkten und Benotung durch:
  • Klausur oder mündliche Prüfung nach Maßgabe des Dozenten
Modulverantwortlicher:
  • Prof. Dr. Elmar Rückert
Lehrende:
  • Prof. Dr. Elmar Rückert
  • MitarbeiterInnen des Instituts
Literatur:
  • Daphne Koller, Nir Friedman: Probabilistic Graphical Models: Principles and Techniques - ISBN 978-0-262-01319-2
  • Christopher M. Bishop: Pattern Recognition and Machine Learning - Springer (2006), ISBN 978-0-387-31073-2
  • David Barber: Bayesian Reasoning and Machine Learning - Cambridge University Press (2012), ISBN 978-0-521-51814-7
  • Kevin P. Murphy: Machine Learning: A Probabilistic Perspective - ISBN 978-0-262-01802-9
Sprache:
  • Wird nur auf Englisch angeboten
Bemerkungen:

The course is accompanied by three graded assignments on Probabilistic Regression, Probabilistic Inference and on Probabilistic Optimization. The assignments will include algorithmic implementations in Matlab, Python or C++ and will be presented during the exercise sessions. The Robot Operating System (ROS) will also be part in some assignments as well as the simulation environment Gazebo.

The course is accompanied by three pieces of course work on Policy Search for discrete state and action spaces (grid world example), policy learning in continuous spaces using function approximations and policy gradient methods in challenging simulated robotic tasks. The assignments will include both written tasks and algorithmic implementations in Python, and will be presented during the exercise sessions. The OpenAI Gym platform will used in the project works.

Prerequisites for attending the module:
- None

Prerequisites for the exam:
- Successful completion of homework assignments during the semester.

Letzte Änderung:
3.9.2020