UDC 519.21

MATHEMATICS

V. V. ANISIMOV

LIMIT THEOREMS FOR SEMI-MARKOV PROCESSES WITH A COUNTABLE SET OF STATES

(Presented by Academician V. M. Glushkov, 12 I 1970)

Let, for each \(t \in (0,\infty)\), \(\varkappa_t(s) \in \{1,2,\ldots\}\) be a time-homogeneous right-continuous semi-Markov process (s.m.p.), which is specified, following \((^1,^2)\), by the matrix of transition probabilities

\[ F_t(i,j,u)=P\{\varepsilon_{k+1}=j,\tau_t(\varepsilon_k)<u\mid \varepsilon_k=i\},\quad i,j=1,2,\ldots, \]

where \(\varepsilon_k=\varkappa_t(\theta_t(k))\), \(\tau_t(\varepsilon_k)=\theta_t(k+1)-\theta_t(k)\), \(\theta_t(k)\) is the time of the \(k\)-th jump, i.e. \(\theta_t(0)=0\), and \(\theta_t(k)=\min\{s:s>\theta_t(k-1), \varkappa_t(s)\ne \varepsilon_{k-1}\}\), \(k\ge 1\).

Introduce the following random variables:

\[ \beta_t(i,k)=\min\{n:n>\beta_t(i,k-1),\varepsilon_n=i\},\quad k\ge 1 \]

\[ (\beta_t(i,0)=0), \]

and \(\nu_t, \nu_t(i), \Omega_t(i), i=1,2,\ldots\), respectively, the total number of jumps of the s.m.p., the number of visits to state \(i\), and the total time spent in \(i\) during time \(t\).

Let, for each \(t \in (0,\infty)\),

\[ f_t^{(k)}(i,x),\quad x\in(0,\infty),\quad i=1,2,\ldots,\quad k=0,1,2,\ldots, \]

be a family of mutually independent random variables, independent of the s.m.p. \(\varkappa_t(s)\) (here and in what follows the quantities \(\gamma^{(k)}\), \(k=0,1,2,\ldots\), denote random variables independent and identically distributed with \(\gamma\)). Introduce an additive functional \(S(t)\) of the form

\[ S(t)=\sum_{k=0}^{\nu_t-1} f_t(\varepsilon_k), \]

where \(f_t(\varepsilon_k)=f_t^{(k)}(\varepsilon_k,\tau_t(\varepsilon_k))\), \(k=0,1,2,\ldots\).

In what follows we shall assume that the embedded Markov chain, which, as is known, is specified by the matrix \(P(t)=\|p_t(i,j)\|\), \(i,j=1,2,\ldots\), where \(p_t(i,j)=F_t(i,j,\infty)\), \(i,j=1,2,\ldots\), has one positive class with stationary distribution \(q_t(i)\), \(i=1,2,\ldots\), and

\[ M\tau_t(i)=m_t(i)<\infty,\quad i=1,2,\ldots, \]

\[ A_t=\sum_{i=1}^{\infty} q_t(i)m_t(i)<\infty, \]

and the matrix \(\bar P=\lim_{t\to\infty} P(t)\) also corresponds to a chain with one positive class.

Put

\[ a_t=A_t^{-1}\sum_{i=1}^{\infty} q_t(i)m_t(f_i), \]

if \(Mf_t(i,\tau_t(i))=m_t(f_i)<\infty,\ i=1,2,\ldots,\) and the given series converges absolutely, and \(a_t=0\) otherwise. Denote

\[ X_t(k)=\sum_{i=\beta_t(1,k)}^{\beta_t(1,k+1)-1}\tau_t(\varepsilon_i), \qquad \varphi_t(k)=\sum_{i=\beta_t(1,k)}^{\beta_t(1,k+1)-1} f_t(\varepsilon_i), \]

\[ Y_t(k)=\varphi_t(k)-a_tX_t(k), \qquad k=1,2,\ldots \]

Theorem 1. If there exist \(\gamma(t)\) and \(b(t)\) such that

\[ P\left\{\frac{1}{\gamma(t)}\sum_{k=1}^{V_t(1)} \left(X_t(k)-\frac{A_t}{q_t(1)}\right)<z\right\}\to F(z), * \tag{1} \]

\[ P\left\{\frac{1}{b(t)}\sum_{k=1}^{V_t(1)}Y_t(k)<z\right\}\to \Phi(z), \tag{2} \]

\[ \gamma(t)/t\to0, \qquad A_t/\gamma(t)q_t(1)\to0, \tag{3} \]

and for any \(\varepsilon>0,\ i=1,2,\ldots\)

\[ P\{f_t(i,\tau_t(i))>\varepsilon b(t)\}\to0, \]

\[ P\{a_t\tau_t(i)>\varepsilon b(t)\}\to0 \]

as \(t\to\infty\), and \(F(z)\) is a proper distribution function, then, independently of the initial state of the s.m.p.,

\[ P\left\{\frac{1}{b(t)}(S(t)-ta_t)<z\right\}\to\Phi(z). \]

Corollary. If \(a_t\ne0\) and for any \(k=1,2,\ldots\), as \(\alpha_t=o(1)\),

\[ M\exp\{i\lambda\alpha_t(m_t(f_k))^{-1}f_t(k,\tau_t(k))\} =1+\alpha_t i\lambda(1+o_k(1)), \]

then, under assumptions (1) and (3), the quantity \((ta_t)^{-1}S(t)\) tends in probability to one as \(t\to\infty\), i.e., the law of large numbers is valid for \(S(t)\).

Let us also note that under assumptions (1) and (3)

\[ P\left\{\frac{A_t}{\gamma(t)q_t(1)} \left(v_t(1)-\frac{tq_t(1)}{A_t}\right)<z\right\}\to1-F(-z). \]

The significance of Theorem 1 is that it makes it possible to reduce the study of additive sums of an s.m.p. of the type \(S(t)\) to the study of sums of a nonrandom number of independent identically distributed summands.

We apply the results obtained to the study of the vector of numbers of visits and sojourn times in the states of a certain finite subset \(I=\{1,2,\ldots,r\}\) of the states of the s.m.p. To this end set

\[ f_t(i,x)=\varphi_i+f_ix,\ i=1,\ldots,r, \qquad f_t(i,x)=0,\ i>r, \]

where \(\varphi_i, f_i,\ i=1,\ldots,r\), are arbitrary real numbers.

Denote \(\theta_t^*(0)=0\), and

\[ \theta_t^*(k)=\min\{\theta_t(l):\theta_t(l)>\theta_t^*(k-1),\ \varepsilon_l\in I\}, \qquad k\ge1. \]

Set \(\tau_t^*(\varepsilon_k^*)=\theta_t^*(k+1)-\theta_t^*(k)\), where \(\varepsilon_k^*=\chi_t(\theta_t^*(k))\). It is clear that

\[ \tau_t^*(i)=\tau_t(i)+\widetilde{\tau}_t(i), \qquad i\in I, \]

where

\[ \widetilde{\tau}_t(i)=\min\{s:\chi_t(s)\in I\mid \chi_t(0-0)=i,\ \chi_t(0)\ne i\}. \]

\[ \text{* Here the notation } V_t(j) \text{ denotes the integer part of } \frac{t}{A_t}q_t(j),\ j=1,2,\ldots \]

Suppose that for each \(j \in I\) there exist \(b_t(j)\) and \(B_t(j)\) such that the joint distribution

\[ \left( \frac{1}{b_t(j)} \sum_{k=1}^{\nu_t(j)} \left(\tau_t^{(k)}(j)-m_t(j)\right),\, \frac{1}{B_t(j)} \sum_{k=1}^{\nu_t(j)} \left(\widetilde{\tau}_t^{(k)}(j)-\widetilde{m}_t(j)\right) \right) \stackrel{\mathrm{sl}}{\Longrightarrow}(\xi_j,\widetilde{\xi}_j)^*, \tag{4} \]

\[ D_t/t \to 0, \qquad A_t/D_t \to 0, \]

where

\[ D_t=\max\{\sqrt{tA_t},\, b_t(j),\, B_t(j),\, j\in I\}, \]

\[ m_t=\max\{m_t(j),\, j\in I\}, \qquad \widetilde{m}_t(j)=M\widetilde{\tau}(j). \]

Let

\[ \psi_j(\lambda_1,\lambda_2)=M\exp\{i(\lambda_1\xi_j+\lambda_2\widetilde{\xi}_j)\}. \]

Theorem 2. If condition (4) is fulfilled and \(t m_t(A_t b_t)^{-1}\to 0\), then the joint distribution of the random vector

\[ \left\{ \frac{m_t}{B_t}\left(\nu_t(1)-\frac{tq_t(1)}{A_t}\right),\, \sqrt{\frac{A_t}{t}}\left(\nu_t(1)-\frac{q_t(1)}{q_t(i)}\nu_t(i)\right), \right. \]

\[ \left. i=2,\ldots,r,\, g_t(k)\left(\Omega_t(k)-\frac{tq_t(k)m_t(k)}{A_t}\right),\, k=1,\ldots,r \right\}, \]

independently of the initial state, converges weakly to a distribution with characteristic function of the form

\[ \exp\left\{-\frac{\sigma^2(\varphi_2,\ldots,\varphi_r)}{2}\right\} \prod_{j=1}^{r} \psi_j(\rho_j f_j-\widetilde{\rho}_j a(f),\, \alpha_j a(f)), \]

where the parameters \(\varphi_i(f_i)\), \(i=1,\ldots,r\), correspond to the first (last) \(r\) components of the vector, \(\sigma^2(\varphi_2,\ldots,\varphi_r)\) is a nondegenerate quadratic form in the variables \(\varphi_i\), \(i=2,\ldots,r\). Here

\[ B_t=\max\left\{b_t(i),\, \frac{m_t}{A_t}B_t(i),\, i=1,\ldots,r\right\}, \qquad a(f)=\varphi_1+\sum_{i=1}^{r}c_i f_i, \]

\(g_t(k)\), \(k=1,\ldots,r\), are normalizing factors, and \(\rho_j,\widetilde{\rho}_j,\alpha_j,c_i\), \(j=1,\ldots,r\), are certain constants.

The results obtained, in particular, as applied to countable Markov chains having an ergodic distribution, agree with the results of V. A. Volkonskii \((^3)\), and for a p.m.p. for which

\[ F_t(i,j,u)=p(i,j)F(i,u), \qquad i,j=1,2,\ldots, \]

under our assumptions they make it possible to obtain the corresponding results of H. Kesten \((^4)\).

Kiev State University
named after T. G. Shevchenko

Received
1 XII 1970

REFERENCES

\(^1\) P. Levy, Proc. Ind. Congress Math., 3, 416 (1954).
\(^2\) R. Pyke, Ann. Math. Stat., 32, No. 4 (1961).
\(^3\) V. A. Volkonskii, Theory of Probability and Its Applications, 2, issue 2, 231 (1957).
\(^4\) H. Kesten, Trans. Am. Math. Soc., 103, No. 1 (1962).

* In the sense of weak convergence of distribution functions.