Let $\mathcal{X}$ be a simple random walk with barrier at zero, state space $E = \mathbb{N}_0$ and transition matrix below with $0<q<1$.
\begin{bmatrix} 1-q & q & & & \\ 1-q & 0 & q & & \\ & 1-q & 0 & q & \\ & & 1-q & 0 & q \\ & & & \ddots & \ddots & \ddots \end{bmatrix}
How would I determine the stationary distribution for $q < \frac{1}{2}$? And why is the condition that $q < \frac{1}{2}$ necessary for $\mathcal{X}$ to have a stationary distribution?
Also, is saying that $\mathbb{P}(X_n = n |X_0 =0) = q^n >0$ and $\mathbb{P}(X_n =0 | X_0 =n) = (1-q)^n >0$ enough to show $\mathcal{X}$ is irreducible?
Update: I get $\pi_n = \left( \frac{1-q}{q} \right)\pi_{n+1}$ for all $n \in \mathbb{N}_0$, but why is the condition that $q < \frac{1}{2}$ necessary for $\mathcal{X}$ to have a stationary distribution?
$\endgroup$1 Answer
$\begingroup$From $\pi_n = \left( \frac{1-q}{q} \right)\pi_{n+1}$ you can get $\pi_n = \left( \frac{q}{1-q} \right)\pi_{n-1} = \left( \frac{q}{1-q} \right)^n \pi_0$.
$\sum\pi_i=\sum \left( \frac{q}{1-q} \right)^i \pi_0 = \pi_0 \sum \left( \frac{q}{1-q} \right)^i =1$
So $\pi_0$ should be >0, and the summation should converge, that is true only if $\frac{q}{1-q} < 1$, so $q$ should be less than $1/2$.
Another way to see it is note that "average of the distribution" ($\sum(i*\pi_i)$) move forward for the state $0$ (from here you can only stay here or go one step forward), and move backward for all the other state iff $q<1-q<1/2$. For a stationary distribution the state should be time-independent, so the mean of that distribution should be time-independent, so can not "move forward".
$\endgroup$ 4
