Stochastic Control

Assignments
Assignment 5

Course Outline
Stochastic Optimization
MDPs
POMDPs
- 23 Introduction
- 24 Sequential hypothesis testing
Approx DP
Risk sensitive MDPs
- 28 Risk Sensitive Utility
- 29 Risk Sensitive MDPs
Linear systems
Stochastic Approximation
- 33 Stochastic approximation
- 34 Rate of convergence for stochastic approximation
RL
- 35 The learning setup
- 36 Q-Learning
Dec-POMDPs
- 37 Designer’s Approach
Analysis Appendix
- 38 Convergence of sequences
- 39 Inequalities
Probability Appendix
Linear Algebra Appendix
Convexity Appendix
- 54 Convex sets and convex functions
- 55 Convex Duality
References
Assignments

Assignments
Assignment 5

Assignment 5

Author

Affiliation

McGill University

Updated

February 2, 2026

Exercise 9.7 from notes on monotone MDPs.
Exercise 9.8 from notes on monotone MDPs.

Edit this page