Fall 2011 Strategic Practice 11: Section 3 (Markov Chains) - Question 3
A cat and a mouse move independently back and forth between two rooms.
At each time step, the cat moves from the current room to the other room
with probability 0.8. Starting from Room 1, the mouse moves to Room 2 with
probability 0.3 (and remains otherwise). Starting from Room 2, the mouse
moves to Room 1 with probability 0.6 (and remains otherwise).
(a) Find the stationary distributions of the cat chain and of the mouse chain.
(b) Note that there are 4 possible (cat, mouse) states: both in Room 1, cat in
Room 1 and mouse in Room 2, cat in Room 2 and mouse in Room 1, and both
in Room 2. Number these cases 1, 2, 3, 4 respectively, and let be the number
of the current (cat, mouse) state at time n. Is a Markov chain?
Solution: Consult iTunes course for full detailed solutions.
(a.) For the cat Markov chain, the stationary distribution is (1/2, 1/2) by symmetry.
For the mouse Markov chain: solving sQ = s and normalizing yields (2/3, 1/3).
(b.) Yes, it is a Markov chain. Given the current (cat, mouse) state, the past history
of where the cat and mouse were previously are irrelevant for computing the
probabilities of what the next state will be
"Mathematics is the logic of certainty, but
statistics is the logic of uncertainty."
Copyright © 2011 Stat 110 Harvard.
Website layout by former Stat110'er.