Model approximation in MDPs and POMDPs
Course outline
These notes accompany the CNI Summer School 2026 on Model approximation in MDPs and POMDPs, hosted by the Centre for Networked Intelligence (CNI) at IISc Bengaluru.
This summer school focuses on model approximation techniques in Markov Decision Processes (MDPs) and Partially Observable Markov Decision Processes (POMDPs). The course introduces fundamental concepts in stochastic systems and optimization, followed by a structured treatment of decision-making under uncertainty using MDPs and POMDPs. It further explores approximation methods, sub-optimality bounds, and their connections to reinforcement learning. The program is designed for students, researchers, and practitioners interested in stochastic control, optimization, and learning in uncertain environments.
Schedule
| Lecture | Date | Time | Topic |
|---|---|---|---|
| 1 | Mon 20 Jul | 9:00–10:30 | Static stochastic optimization |
| 2 | Mon 20 Jul | 11:00–12:30 | Dynamic programming |
| 3 | Tue 21 Jul | 9:00–10:30 | MDP properties |
| 4 | Tue 21 Jul | 11:00–12:30 | Model approximation |
| 5 | Wed 22 Jul | 9:00–10:30 | State abstraction |
| 6 | Wed 22 Jul | 11:00–12:30 | Sample-path bounds |
| 7 | Thu 23 Jul | 9:00–10:30 | POMDPs and information states |
| 8 | Thu 23 Jul | 11:00–12:30 | Approximate information state |
| 9 | Fri 24 Jul | 8:30–9:30 | Certainty equivalence in POMDPs |
| 10 | Fri 24 Jul | 9:45–10:45 | Variations and Discussions |