Mathematics

V. A. Statulevičius

ASYMPTOTIC EXPANSION FOR NONHOMOGENEOUS MARKOV CHAINS

(Presented by Academician A. N. Kolmogorov, 20 VIII 1956)

A nonhomogeneous Markov chain is studied with a finite number \(s>1\) of possible states \(e_1,\ldots,e_s\) and transition probabilities \(p_{\alpha\beta}^{(k)}\) from state \(e_\alpha\) at the \((k-1)\)-st step to state \(e_\beta\) at the \(k\)-th step.

Let the random variable \(\zeta_n^{(\alpha)}\) denote the number of visits to state \(e_\alpha\) during the first \(n\) steps. Then, for the probability \(P_\gamma(m)\) that the random vector \(\zeta_n=(\zeta_n^{(1)},\ldots,\zeta_n^{(s)})\) assumes the value \(m=(m_1,\ldots,m_s)\), under the condition that \(e_\gamma\) is the initial state, the following theorem is valid.

Theorem. Suppose that the following conditions are satisfied:

A. \(p_{\alpha\beta}^{(l)} \geq \lambda p_{\alpha\beta}^{(k)}\) for all \(\alpha,\beta,k,l\), where the constant \(\lambda>0\).

B. The set of states of the chain forms one essential class.

C. The rank \(r\) of the chain is equal to \(s\) \((^1)\).

Then, for any \(\gamma\) and integer \(k>0\), uniformly in all \((m_1,\ldots,m_s)\), we have

\[ \sqrt{D_n^{(1)}\cdots D_n^{(s-1)}}\,\mathcal P_\gamma(m) = g_{s-1}(x) + \sum_{j=1}^{k} n^{-j/2} P_{\gamma j}\!\left(-\frac{\partial}{\partial x}g(x)\right) + O\!\left(\frac{1}{n^{(k+1)/2}}\right). \tag{1} \]

Here

\[ g_{s-1}(x) = \frac{1}{\sqrt{(2\pi)^{s-1}\Delta_n}} \exp\left[-\frac12 Q_n^{-1}(x)\right] \]

is the density of the \((s-1)\)-dimensional normal distribution,

\[ x= \left( \frac{m_1-E_n^{(1)}}{\sqrt{D_n^{(1)}}}, \ldots, \frac{m_{s-1}-E_n^{(s-1)}}{\sqrt{D_n^{(s-1)}}} \right); \]

\(E_n^{(\alpha)} \asymp n,\ D_n^{(\alpha)} \asymp n\); the quadratic form \(Q_n(x)=Q_n(x_1,\ldots,x_{s-1})\) is positive definite; \(\Delta_n\) is the determinant of this form. \(P_{\gamma j}(it)\) is a polynomial of degree not exceeding \(3j\) in the components of the vector \(it=(it_1,\ldots,it_{s-1})\). The coefficients of the polynomial are real, depend on \(\gamma\) and \(n\), but are uniformly bounded for all \(n\). \(P_{\gamma j}\!\left(-\frac{\partial}{\partial x}g_{s-1}(x)\right)\) means that, in place of the powers \(-it_\alpha\), derivatives of \(g_{s-1}(x)\) with respect to the corresponding component \(x_\alpha\) are taken \((^2)\).

If condition C is violated, but \(r>1\), then in equality (1) \(s\) should be replaced by \(r\).

Leningrad State University
named after A. A. Zhdanov

Received
20 VIII 1956

References

\(^1\) V. A. Statulevičius, DAN, 107, No. 4 (1956).
\(^2\) S. Kh. Sirazhdinov, Limit Theorems for Homogeneous Markov Chains, Tashkent, 1955.