Question

Let \(\{X_n:n\ge0\}\) be a homogeneous Markov chain with state space \(S=\{1,2,\ldots,7\}\) and transition probability matrix
\[ P= \begin{pmatrix} \frac{1}{3} & 0 & \frac{2}{3} & 0 & 0 & 0 & 0 \\ 0 & \frac{1}{3} & 0 & \frac{1}{3} & 0 & \frac{1}{3} & 0 \\ \frac{1}{2} & 0 & \frac{1}{2} & 0 & 0 & 0 & 0 \\ 0 & \frac{1}{2} & 0 & \frac{1}{4} & 0 & \frac{1}{4} & 0 \\ \frac{1}{2} & 0 & 0 & 0 & \frac{1}{4} & \frac{1}{4} & 0 \\ 0 & \frac{2}{3} & 0 & \frac{1}{6} & 0 & \frac{1}{6} & 0 \\ \frac{1}{3} & \frac{1}{3} & 0 & 0 & 0 & 0 & \frac{1}{3} \end{pmatrix}. \]
Then which of the following statements is correct?

• A. 2 is a transient state
• B. 3 is a transient state
• C. 4 is a transient state
• D. 5 is a transient state

Hint:

States \(\{1,3\}\) and \(\{2,4,6\}\) each form a closed set the chain can never leave. State 5 can jump into either closed set and never return, so it is transient.

Answer 1

The Correct Option is D

Step 1: List where each state can jump to.
Reading each row of \(P\), the states reachable with positive probability from each state are: \(1\to\{1,3\}\), \(2\to\{2,4,6\}\), \(3\to\{1,3\}\), \(4\to\{2,4,6\}\), \(5\to\{1,5,6\}\), \(6\to\{2,4,6\}\), \(7\to\{1,2,7\}\).

Step 2: Find closed communicating classes.
A set of states is closed if, once you enter it, you can never leave it. Look at \(\{1,3\}\): from \(1\) you can only go to \(1\) or \(3\), and from \(3\) you can only go to \(1\) or \(3\). So \(\{1,3\}\) is closed, and since \(1\) and \(3\) reach each other, they communicate. A closed communicating class on a finite state space is always recurrent. Now look at \(\{2,4,6\}\): from \(2\) you can only go to \(2,4,6\); from \(4\) you can only go to \(2,4,6\); from \(6\) you can only go to \(2,4,6\). So \(\{2,4,6\}\) is also closed, and \(2,4,6\) all communicate with each other. So this is also a recurrent class.

Step 3: Check state 5.
From state \(5\), the chain can move to \(1\) with probability \(1/2\) or to \(6\) with probability \(1/4\), besides staying at \(5\). But \(1\) belongs to the closed class \(\{1,3\}\) and \(6\) belongs to the closed class \(\{2,4,6\}\); neither class has any path back to \(5\). So once the chain leaves \(5\) for either closed class, it can never return. This makes \(5\) a transient state.

Step 4: Check state 7 for comparison.
Similarly, state \(7\) can move to \(1\) or \(2\), both of which lead into closed recurrent classes that never come back to \(7\). So \(7\) is transient too, though it is not one of the listed options.

Step 5: Check option (A), state 2.
State \(2\) lies inside the closed class \(\{2,4,6\}\). Since this class is closed and finite, every state inside it is recurrent, not transient. So option (A) is false.

Step 6: Check option (B), state 3.
State \(3\) lies inside the closed class \(\{1,3\}\), so it is recurrent. Option (B) is false.

Step 7: Check option (C), state 4.
State \(4\) lies inside the closed class \(\{2,4,6\}\), so it is recurrent. Option (C) is false.

Step 8: Check option (D), state 5.
As shown in Step 3, state \(5\) can escape into a closed class it never returns from. This is exactly the definition of a transient state, so option (D) is true.

Final Answer:
\[ \boxed{\text{5 is a transient state}} \]