Notifications
Clear all

[Solved] Markov Decision Process example model?


Posts: 4
Customer
Topic starter
Active Member
Joined: 2 months ago

Does anyone have a simple example model of a Markov Decision Process (MDP)?

Topic Tags
2 Replies
Posts: 9
Admin
Member
Joined: 11 years ago

These are a bit more elaborate than a simple example model, but are probably of interest since they are applied examples. They are Markov models of the COVID-19 pandemic, projecting hospitalizations, ICU needs, case counts and deaths under different mitigation strategies. The first is by Rob Brown. The second was by Jorge Muro-Arbulú, who adapted Rob's model to the country of Peru. I wrote a blog article about Rob's model last spring, and is in Spanish. Jorge put together an incredible brochure about the Peru model.

Reply
Posts: 14
Moderator
Member
Joined: 4 months ago

 

In this simple finite-state MDP, a taxi driver can be in one of three towns. In each town he can either cruise the streets, wait at a taxi stand, or wait for call from dispatch. The goal of the model is to compute the optimal policy (i.e., which action to do from each town). The town that he is in is the state.

I demonstrated value iteration with discounting (can be zero) on a finite horizon, and then also policy iteration on an infinite horizon using Iterate (discount rate must be greater than zero). I lifted the example from slides from Dennis Bricker at U. of Iowa. 

Reply
Share:
Scroll to Top