Fall 2011 Strategic Practice 11: Section 3 (Markov Chains) - Question 3
A cat and a mouse move independently back and forth between two rooms. At each time step, the cat moves from the current room to the other room with probability 0.8. Starting from Room 1, the mouse moves to Room 2 with probability 0.3 (and remains otherwise). Starting from Room 2, the mouse moves to Room 1 with probability 0.6 (and remains otherwise).
(a) Find the stationary distributions of the cat chain and of the mouse chain.
(b) Note that there are 4 possible (cat, mouse) states: both in Room 1, cat in Room 1 and mouse in Room 2, cat in Room 2 and mouse in Room 1, and both in Room 2. Number these cases 1, 2, 3, 4 respectively, and let $Z_{n}$ be the number of the current (cat, mouse) state at time n. Is $Z_{n}$ a Markov chain?
Solution: Consult iTunes course for full detailed solutions. (a.) For the cat Markov chain, the stationary distribution is (1/2, 1/2) by symmetry. For the mouse Markov chain: solving sQ = s and normalizing yields (2/3, 1/3). (b.) Yes, it is a Markov chain. Given the current (cat, mouse) state, the past history of where the cat and mouse were previously are irrelevant for computing the probabilities of what the next state will be
"Mathematics is the logic of certainty, but statistics is the logic of uncertainty."