MDP

• Linear Quadratic Regulation (LQR)
• Theory: Infinite horizon discounted MDP
• Theory: Optimality of threshold policies in optimal stopping
• Example: Inventory Management (revisited)
• Infinite horizon Linear Quadratic Regulation (LQR)
• Example: Service Migration in Mobile Edge Computing
• Theory: Computational complexity of value iteration
• Theory: Linear Programming Formulation and Constrained MDPs
• Theory: A Martingale Principle of Optimal Control
• Example: Optimal choice of the best alternative
• Theory: Basic model of an MDP
• Example: Call options
• Theory: Lipschitz MDPs
• Theory: State aggregation or discretization or quantization
• Example: Optimal Gambling
• Theory: Feature abstraction in MDPs
• Example: Inventory Management
• Theory: Model approximation in MDPs
• Numerics: Matrix formulation of Markov decision processes
• Theory: Approximate dynamic programming
• Theory: Monotone value functions and policies
• Example: Power-delay trade-off in wireless communication
• Theory: Reward Shaping
• Linear Exponential of Quadratic Gaussian (LEQG)