Stochastic Control

Assignments
Assignment 6

Course Outline
Stochastic Optimization
MDPs
POMDPs
- 23 Introduction
- 24 Sequential hypothesis testing
Approx DP
Risk sensitive MDPs
- 28 Risk Sensitive Utility
- 29 Risk Sensitive MDPs
Linear systems
Stochastic Approximation
- 33 Stochastic approximation
- 34 Rate of convergence for stochastic approximation
RL
- 35 The learning setup
- 36 Q-Learning
Dec-POMDPs
- 37 Designer’s Approach
Analysis Appendix
- 38 Convergence of sequences
- 39 Inequalities
Probability Appendix
Linear Algebra Appendix
Convexity Appendix
- 54 Convex sets and convex functions
- 55 Convex Duality
References
Assignments

Assignments
Assignment 6

Assignment 6

Author

Affiliation

McGill University

Updated

February 13, 2026

Exercise 14.4 from the notes on optimal stopping.
Exercise 11.1 from the notes on continuous state spaces.

Edit this page