Question

Consider a Markov Chain with transition probability matrix \(P = \begin{pmatrix}0 & 1 & 0\\0.5 & 0 & 0.5\\0 & 1 & 0\end{pmatrix}\) with state space \(S=\{0,1,2\}\). Then:

• A. All the states are aperiodic (period 1)
• B. All the states are periodic with period 2
• C. All the states are transient
• D. All the states are null

Hint:

Trace the smallest number of steps a state needs to return to itself in this chain, and check whether every diagonal entry of \(P\) (and of \(P^2\)) is zero or positive.

Answer 1

The Correct Option is B

Step 1: From \(P\), state 0 goes to state 1 with probability 1, state 1 goes to state 0 or state 2 each with probability 0.5, and state 2 goes to state 1 with probability 1. Every diagonal entry of \(P\) is 0, so no state can return to itself in exactly 1 step.
Step 2: The shortest return to state 0 is \(0\to1\to0\) (2 steps). Since 0 is reachable only from 1, and 1 is reachable only from 0 or 2 (both reachable only from 1), every return to 0 happens after an even number of steps; the same reasoning applies to states 1 and 2.
Step 3: The period of a state is the gcd of all its possible return times. Since all return times are multiples of 2, and a 2-step return is actually possible, each state has period exactly 2.
Step 4: The chain is irreducible (0, 1, 2 all communicate through 1), and periodicity is a class property, so all three states share period 2. Being a finite irreducible chain, it is positive recurrent, ruling out "transient" and "null".
\(\boxed{\text{All the states are periodic with period 2}}\)