ex02

Exercise 02

This exercise deals with the formal handling of Markov chains and Markov decision processes.

Tasks:

1. reading Markov chain graphs and determining state transitions probability
2. determining stationary state of Markov chains
3. determining state values of Markov reward processes
4. reading Markov decision process graphs and determining state transition probability when actions are involved
5. evaluating policies (determining state value functions) in Markov decision processes when the policy is given
6. calculating action values in Markov decision processes when the policy is given
7. state-value based comparison of different policies