Question

Consider a discrete time Markov chain with state space \(S=\{1,2,3\}\) and the transition probability matrix

• A. The Markov chain is irreducible
• B. \(1\) is a recurrent state
• C. \(2\) is a transient state
• D. \(3\) is a recurrent state

Hint:

In a finite Markov chain, every state belonging to a closed communicating class is recurrent.

Answer 1

The Correct Option is D

Step 1: Analyze communication between states.
From the transition matrix,
\[ p_{11}=\frac12,\quad p_{12}=\frac14,\quad p_{13}=\frac14 \] So, from state \(1\), the chain can move to states \(1,2,\) and \(3\).
Also,
\[ p_{21}=0,\quad p_{31}=0 \] So, from states \(2\) and \(3\), the chain cannot go to state \(1\).
Therefore, state \(1\) does not communicate with states \(2\) and \(3\).
Hence, the Markov chain is not irreducible.
So, option (A) is incorrect.

Step 2: Identify the closed class.
From state \(2\),
\[ p_{22}=\frac15,\quad p_{23}=\frac45 \] So, state \(2\) can go to state \(2\) or state \(3\).
From state \(3\),
\[ p_{32}=1 \] So, state \(3\) always goes to state \(2\).
Thus, once the chain enters the set
\[ \{2,3\}, \] it never leaves this set.
Therefore,
\[ \{2,3\} \] is a closed communicating class.

Step 3: Classify states.
In a finite Markov chain, every state in a closed communicating class is recurrent.
Since states \(2\) and \(3\) form a closed communicating class, both \(2\) and \(3\) are recurrent states.
Therefore, state \(3\) is recurrent.
So, option (D) is correct.

Step 4: Check state \(1\).
From state \(1\), the chain can move to state \(2\) or state \(3\).
Once it reaches \(\{2,3\}\), it cannot return to state \(1\).
Therefore, state \(1\) is transient, not recurrent.
So, option (B) is incorrect.
Also, state \(2\) is recurrent, not transient.
So, option (C) is incorrect.

Step 5: Final conclusion.
The true statement is
\[ \boxed{(D)} \]