ON THE CONNECTION BETWEEN THE STABILITY OF STOCHASTIC DIFFERENCE AND DIFFERENTIAL SYSTEMS

T. AKHMETKALIEV

Stochastic systems of differential equations and the corresponding systems of difference equations are considered. It is assumed that these systems are subject to the influence of a random Markov chain generated in the course of the motion. With the aid of the Lyapunov-function method [1], the question is studied of the preservation of the asymptotic stability of the solution \(x=0\) in passing from a difference system to the corresponding differential system. A similar problem for ordinary difference equations was considered in [2].

§ 1. Consider a system described by the stochastic differential equations of the perturbed motion,

\[ \frac{dx}{dt}=\varphi(x,\eta(t)) \tag{1.1} \]

and the corresponding system of stochastic difference equations

\[ x_{k+1}=x_k+h\varphi(x_k,\eta_{k+1}). \tag{1.2} \]

Here \(x=\{x^{(1)},\ldots,x^{(n)}\}\) denotes an \(n\)-dimensional vector of coordinates \(x^{(i)}\), which are equal to zero in the given unperturbed motion; \(\eta(t)\in Y\{\eta^{(1)},\ldots,\eta^{(r)},\ldots\}\) is a homogeneous Markov chain with a countable number of states \(\eta^{(i)}\), a continuous parameter \(t\), and transition probabilities [3]

\[ p_{ij}(\Delta t)= \begin{cases} a_{ij}\Delta t+o(\Delta t), & i\ne j,\\ (1-a_{jj}\Delta t)+o(\Delta t), & i=j \end{cases} \qquad (a_{ij}=\mathrm{const}), \]

where \(p_{ij}(\Delta t)\) is the probability of a change of values \(\eta^{(i)}\to\eta^{(r)}\) during time \(\Delta t\), while

\[ a_{jj}=\sum_{i\ne j}^{\infty} a_{ij}; \]

\(o(\Delta t)\) is an infinitesimal of higher order of smallness relative to \(\Delta t\); \(\eta_k\in Y\{\eta^{(1)},\ldots,\eta^{(r)},\ldots\}\) is a homogeneous Markov chain with a countable number of states and a discrete parameter, with

\[ p_{ij}(h)= \begin{cases} a_{ij}h, & i\ne j,\\ (1-a_{jj}h), & i=j \end{cases} \qquad (a_{ij}=\mathrm{const}), \]

where \(p_{ij}(h)\) is the probability of a change of values from \(\eta^{(i)}\) to \(\eta^{(j)}\) in one step, and here also

\[ a_{jj}=\sum_{i\ne j}^{\infty}a_{ij}; \]

\(h\) is the step of the difference equation; \(k\) is the step number; \(x_k=x(t_0+kh)=x(t_k)\); \(\varphi=\{\varphi^{(1)},\ldots,\varphi^{(n)}\}\) is a vector function, with respect to which we assume that it is continuous in all vari-

variable and, for all \(\eta\), satisfies a Lipschitz condition (with constant \(L\)) with respect to \(x^{(j)}\) in each bounded domain of the form

\[ \|x\| \leq H,\quad t \geq t_0,\quad k \geq k_0 \geq 0 \quad (H < \infty). \]

(The symbol \(\|x\|\) denotes the Euclidean norm of the vector \(x\).) With regard to the right-hand sides of the systems, moreover, we shall assume that \(\varphi(0,\eta) \equiv 0\) for all \(\eta \in Y\).

In particular, for linear systems

\[ \frac{dx}{dt}=A(\eta(t))x \tag{1.3} \]

and

\[ x_{k+1}=x_k+hA(\eta_{k+1})x_k \tag{1.4} \]

we put \(H=\infty\). Here \(A(\eta)\) is the matrix \(\|a_{im}(\eta)\|_1^n\), and \(a_{im}(\eta)\) are continuous bounded functions of \(\eta\).

By the symbol \(M[\xi|c]\) we shall denote the conditional mathematical expectation of the random variable \(\xi\) [3].

We formulate the definitions of a solution of the stochastic difference equations (1.2), as well as of asymptotic and exponential stability in the mean, corresponding to the analogous definitions for stochastic differential equations (1.1) [4, 5].

Let initial conditions \(x_{k_0}, \eta_{k_0}\) be given. These initial conditions and equations (1.2) generate in the space \(\{x,\eta\}\) a bundle of realizations of the process \(\{x_k,\eta_k|x_{k_0},\eta_{k_0}\}\)—a Markov random function. In this case the realizations \(\{x_{k,\omega},\eta_{k,\omega}|x_{k_0},\eta_{k_0}\}\) are a solution of equation (1.2) for \(\eta=\eta_{k,\omega}\). The set of such realizations forms an \((n+1)\)-dimensional vector function \(\{x_k(x_{k_0},\eta_{k_0}),\eta_k(\eta_{k_0})\}\), which we shall call the solution of the stochastic equation (1.2).

Definition 1.1. The solution \(x=0\) of system (1.2) will be called asymptotically stable in the mean if the following conditions are satisfied:

1) for any arbitrarily small number \(\varepsilon>0\) one can indicate a number \(\delta>0\) such that every solution of system (1.2) with initial data \(\eta_{k_0}\in Y,\ k_0\geq 0,\ \|x_{k_0}\|\leq \delta\) satisfies, for all \(k\geq k_0\), the inequality

\[ M\left[\|x_k(x_{k_0},\eta_{k_0})\|^2\middle|x_{k_0},\eta_{k_0}\right]<\varepsilon; \]

2)

\[ \lim_{k\to\infty}M\left[\|x_k(x_{k_0},\eta_{k_0})\|^2\middle|x_{k_0},\eta_{k_0}\right]=0 \]

for all solutions with initial conditions \(\|x_{k_0}\|\leq H_0,\ \eta_{k_0}\in Y\). Here \(H_0\) is a certain constant.

If the quantity \(\delta\) can be chosen independently of \(h,\ h\in(0,h_0],\ h_0<\infty\), and if for any \(\gamma>0\) one can choose a constant \(C=K_h h\), the same for all \(h\), so that

\[ M\left[\|x_k(x_{k_0},\eta_{k_0})\|^2\middle|x_{k_0},\eta_{k_0}\right]<\gamma, \]

provided only that \(k>k_0+K_h\), then the asymptotic stability will be called uniform with respect to \(h\).

Definition 1.2. If for any initial data from the domain

\[ -\infty<x^{(j)}<+\infty,\quad \eta\in Y,\quad k\geq k_0\geq 0 \quad (j=1,\ldots,n) \]

there exist constant numbers \(B>0,\ \beta>0\) such that for all \(k\geq k_0\) the inequality

\[ M\left[\|x_k(x_{k_0},\eta_{k_0})\|^2\middle|x_{k_0},\eta_{k_0}\right]\leq B\|x_{k_0}\|^2 e^{-\beta(k-k_0)h}, \tag{1.5} \]

holds,

then the solution \(x=0\) of system (1.2) will be called exponentially stable in the mean.

If the quantities \(B,\ \beta\) can be chosen independently of \(h,\ h \in (0,h_0)\), then the exponential stability will be called uniform with respect to \(h\).

§ 2. Let us introduce a number of concepts corresponding to the basic concepts of Lyapunov’s second method for ordinary systems of differential equations [1, 6]. Consider scalar functions \(v(x,\eta)\), defined and continuously differentiable in the domain

\[ -\infty < x^{(j)} < +\infty,\quad \eta \in Y,\quad k \geqslant k_0 \geqslant 0\quad (j=1,\ldots,n). \]

We shall assume that \(v(0,\eta)\equiv 0\).

The function \(v(x,\eta)\) will be called sign-definite for \(-\infty \leqslant x^{(j)} < +\infty,\ \eta\in Y,\ k \geqslant k_0 \geqslant 0\ (j=1,\ldots,n)\), if there exists a function \(w(x)\), positive definite in the sense of Lyapunov [1], \(w(0)=0\), satisfying the inequality \(v(x,\eta)\geqslant w(x)\) (or \(v(x,\eta)\leqslant -w(x)\)).

Next we introduce the concept of the first difference

\[ \frac{\Delta M[v]}{h} \]

of the mathematical expectation of the function \(v(x,\eta)\) along the solutions of system (1.2). Suppose that at the moment \(k\) the values \(x_k=x,\ \eta_k=\eta\) have occurred. Then from this point \((x,\eta)\) there issues a bundle of realizations \(\{x_{k+1,\omega},\eta_{k+1,\omega}\}\) of the process \(\{x_{k+1},\eta_{k+1}\}\), generating, by virtue of system (1.2), the random variable

\[ [v(x_{k+1},\eta_{k+1}) \mid x,\eta]. \]

We define the first difference in the following way:

\[ \frac{\Delta M[v]}{h} = \frac{M\,[v(x_{k+1},\eta_{k+1})\mid x_k=x,\eta_k=\eta]-v(x,\eta)}{h}. \tag{2.1} \]

Let us compute the first difference of the function \(v(x,\eta)\). Suppose that at the moment \(k\) the values \(x_k=x,\ \eta_k=\eta^{(j)}\) of the \((n+1)\)-dimensional process \(\{x,\eta\}\) have occurred. Then it may be assumed that at the \((k+1)\)-st moment two events may occur: either the quantity \(\eta_k\) retains its value (event \(A\)), i.e.

\[ \eta_{k+1}=\eta_k=\eta^{(j)},\quad P[A]=1-\alpha_j h; \]

or the quantity \(\eta_k\) changes its value (event \(B_r\)) from \(\eta_k=\eta^{(j)}\) to \(\eta_{k+1}=\eta^{(r)}\), where \(P[B_r]=\alpha_{jr}h\).

In the case where event \(A\) is realized, equations (1.2) will work on the interval \(h\) as ordinary difference equations, and the increment of the function \(v\) in this case, \(\Delta_A v\), will be computed by the formula

\[ \Delta_A v = v(x_{k+1},\eta^{(j)})-v(x,\eta^{(j)}). \tag{2.2} \]

When event \(B_r\) is realized we have

\[ \Delta_{B_r}v = v(x_{k+1},\eta^{(r)})-v(x,\eta^{(j)}). \tag{2.3} \]

Then, for computing the quantity \(\Delta M[v]\), we have the equality

\[ \Delta M[v]=P[A]\Delta_A v+\sum_{\substack{r=1\\ r\ne j}}^{\infty} P[B_r]\Delta_{B_r}v. \tag{2.4} \]

Substituting the probabilities \(P[A]\), \(P[B_r]\) into equality (2.4) and dividing by \(h\), we obtain

\[ \frac{\Delta M[v]}{h} = \frac{\bigl[v(x_{k+1},\eta^{(j)})-v(x,\eta^{(j)})\bigr](1-\alpha_{jj}h)}{h} + \]

\[ +\sum_{r\ne j}^{\infty}\bigl[v(x_{k+1},\eta^{(r)})-v(x,\eta^{(j)})\bigr]\alpha_{jr}. \tag{2.5} \]

For the system of differential equations (1.1), the notion of the first difference is replaced by the averaged derivative [4, 5, 7], which is defined as follows:

\[ \frac{dM[v]}{dt} = \lim_{\Delta t\to 0} \frac{ M\bigl[v(x(t+\Delta t),\eta(t+\Delta t))\mid x(t)=x,\eta(t)=\eta\bigr]-v(x,\eta) }{\Delta t}. \]

For the system (1.1) the derivative is computed by the formula

\[ \frac{dM[v]}{dt} = \sum_{i=1}^{\infty} \frac{\partial v}{\partial x^{(i)}}\, \varphi^{(i)}(x,\eta^{(j)}) + \]

\[ + \sum_{r\ne j}^{\infty} \bigl[v(x,\eta^{(r)})-v(x,\eta^{(j)})\bigr]\alpha_{jr}. \tag{2.6} \]

§ 3. Consider the stationary linear system (1.4). The following holds.

Lemma 3.1. If the solution \(x=0\) of system (1.4) is asymptotically stable in the mean, then it is also exponentially stable in the mean.

Proof. By the hypothesis of the lemma, the solution \(x=0\) of system (1.4) is stable in the mean. Then for any number \(\varepsilon>0\) one can specify a number \(\delta>0\) such that for all \(k\ge k_0\ge 0\) the inequality

\[ M\bigl[\|x_k(x_{k_0},\eta_{k_0})\|^2\mid x_{k_0},\eta_{k_0}\bigr]<\varepsilon, \]

holds, provided only that \(\|x_{k_0}\|\le \delta,\ \eta_{k_0}\in Y\).

From the stationarity of the system and the homogeneity of the process \(\eta\) it follows that this stability will be uniform with respect to \(k_0\ge 0\). Moreover, from the linearity of the equations it follows that asymptotic stability in the mean is uniform with respect to the initial perturbations \(x_{k_0}\). Then one can specify an integer positive number \(K\) such that for any initial data from the region \(-\infty<x^{(i)}<+\infty,\ \eta\in Y,\ k\ge k_0\ge 0\), the inequality

\[ M\bigl[\|x_{k_0+K}(x_{k_0},\eta_{k_0})\|^2\mid x_{k_0},\eta_{k_0}\bigr] \le \frac{1}{2}\|x_{k_0}\|^2. \]

holds. In view of the Markov property of the process \(\{x_k,\eta_k\}\), the following inequality is also valid:

\[ \begin{aligned} &M\bigl[\|x_{k_0+2K}(x_{k_0},\eta_{k_0})\|^2\mid x_{k_0},\eta_{k_0}\bigr] \\ &\quad = M\bigl[M\{\|x_{k_0+2K}(x_{k_0+K},\eta_{k_0+K})\|^2 \mid x_{k_0+K},\eta_{k_0+K}\}\mid x_{k_0},\eta_{k_0}\bigr] \\ &\quad \le \frac{1}{2}\, M\bigl[\|x_{k_0+K}(x_{k_0},\eta_{k_0})\|^2 \mid x_{k_0},\eta_{k_0}\bigr] \le \frac{1}{2^2}\|x_{k_0}\|^2. \end{aligned} \]

Arguing in an analogous manner, for any integer \(N>0\) one can obtain the inequality

\[ M\{\|x_{k_0+NK}(x_{k_0},\eta_{k_0})\|^2\mid x_{k_0},\eta_{k_0}\} \leq \frac{1}{2^N}\|x_{k_0}\|^2 . \]

Let \(k=k_0+NK+\Theta\), where \(\Theta<K\). Using relation [2]

\[ \|x_{k,\omega}(x_{k_0+NK,\omega},\eta_{k_0+NK,\omega})\|^2 \leq \|x_{k_0+NK,\omega}\|^2 e^{2nLh\Theta}, \]

which holds for any \(h\), we have

\[ M[\|x_k(x_{k_0},\eta_{k_0})\|^2\mid x_{k_0},\eta_{k_0}] \leq \frac{1}{2^N}\|x_{k_0}\|^2 e^{2nLh\Theta} \]

(\(L\) is the Lipschitz constant). Denoting

\[ \beta=\frac{1}{Kh}\ln 2,\qquad B=2e^{2nLhK}, \]

we obtain

\[ M[\|x_k(x_{k_0},\eta_{k_0})\|^2\mid x_{k_0},\eta_{k_0}] \leq B\|x_{k_0}\|^2 e^{-\beta(k-k_0)h}. \tag{3.1} \]

In the case where the solution \(x=0\) of system (1.4) is asymptotically stable uniformly in \(h\), it is also exponentially stable uniformly in \(h\). This follows from the fact that in this case the choice of the quantity \(Kh\) does not depend, according to Definition 1.1, on \(h\).

We shall prove the validity of the following assertions, analogous to the assertions in [4, 6].

Theorem 3.1. Let there exist, for system (1.4), a function \(v^{(h)}(x,\eta)\) satisfying, for all \(h\in(0,h_0)\), the following estimates:

\[ c_1\|x\|^2\leq v^{(h)}(x,\eta)\leq c_2\|x\|^2, \]

\[ \frac{\Delta M[v^{(h)}]}{h}\leq -c_3\|x\|^2, \tag{3.2} \]

where \(c_1,c_2,c_3\) are some positive constants. Then the solution \(x=0\) of system (1.4) will be exponentially stable in the mean uniformly in \(h\).

Proof. Let \(\{x_k(x_{k_0},\eta_{k_0}),\eta_k(\eta_{k_0})\}\) be the solution of equations (1.4) with initial conditions \(\{x_{k_0},\eta_{k_0}\}\). Consider the random function

\[ V_k=v^{(h)}(x_k(x_{k_0},\eta_{k_0}),\eta_k(\eta_{k_0})), \]

generated by the initial conditions \(\{x_{k_0},\eta_{k_0}\}\), and denote its mathematical expectation by \(v_k^{(h)}\). Then, taking into account the Markov property of the quantity \(\eta\), the following estimates are valid:

\[ c_1M[\|x_k\|^2\mid x_{k_0},\eta_{k_0}] \leq v_k^{(h)} \leq c_2M[\|x_k\|^2\mid x_{k_0},\eta_{k_0}], \]

\[ \frac{v_{k+1}^{(h)}-v_k^{(h)}}{h} = M\left[ \frac{M[v^{(h)}(x_{k+1},\eta_{k+1})\mid x_k,\eta_k]-v^{(h)}(x_k,\eta_k)}{h} \ \middle|\ x_{k_0},\eta_{k_0} \right] = \]

\[ = M\left[ \frac{\Delta M[v^{(h)}]}{h}\ \middle|\ x_{k_0},\eta_{k_0} \right] \leq -c_3M[\|x_k\|^2\mid x_{k_0},\eta_{k_0}]. \tag{3.3} \]

From inequality (3.3) we have

\[ \frac{v_{k+1}^{(h)}-v_k^{(h)}}{v_k^{(h)}} \leqslant -\frac{c_3}{c_2}h. \]

Hence we obtain

\[ v_k^{(h)} \leqslant \left(1-\frac{c_3}{c_2}h\right)^{(k-k_0)} v_{k_0}^{(h)} \leqslant e^{-\frac{c_3}{c_2}(k-k_0)h} v_{k_0}^{(h)} . \]

Then, applying the estimates (3.3) again, we shall have

\[ M\left[\|x_k(x_{k_0},\eta_{k_0})\|^2 \mid x_{k_0},\eta_{k_0}\right] \leqslant \frac{c_2}{c_1}\|x_{k_0}\|^2 e^{-\frac{c_3}{c_2}(k-k_0)h}, \]

which is equivalent to condition (1.5), and the quantities \(\dfrac{c_2}{c_1}\), \(\dfrac{c_3}{c_2}\), in accordance with the condition of the theorem, do not depend on \(h\).

The theorem proved admits a converse; namely, the following assertion is true.

Theorem 3.2. Let the solution \(x=0\) of system (1.4) be exponentially stable in the mean, uniformly with respect to \(h\). Then in the domain \(-\infty < x^{(i)} < +\infty\), \(\eta \in Y\), \(k \geqslant k_0 \geqslant 0\), there exists a function \(v^{(h)}(x,\eta)\) satisfying the estimates (3.2) uniformly with respect to \(h \in (0,h_0]\).

Proof. By the hypothesis of the theorem, the solution \(x=0\) of equations (1.4) is exponentially stable in the mean, i.e. the condition

\[ M\left[\|x_k(x_{k_0},\eta_{k_0})\|^2 \mid x_{k_0},\eta_{k_0}\right] \leqslant B\|x_{k_0}\|^2 e^{-\beta(k-k_0)h}. \tag{3.4} \]

is satisfied. Consider the function defined by the equality

\[ v^{(h)}(x,\eta)=\sum_{\tau=k}^{\infty} M\left[\|x_\tau(x,\eta)\|^2 \mid x_k=x,\eta_k=\eta\right]h, \tag{3.5} \]

and show that such a function satisfies all the conditions of the theorem. Indeed, according to (3.4), we have the estimate

\[ v^{(h)}(x,\eta)=\sum_{\tau=k}^{\infty} M\left[\|x_\tau(x,\eta)\|^2 \mid x,\eta\right]h \leqslant \sum_{\tau=k}^{\infty}B\|x\|^2 e^{-\beta(\tau-k)h}h \leqslant \]

\[ \leqslant B\|x\|^2\left(h+\int_{kh}^{\infty} e^{-\beta(\tau-kh)}\,d\tau\right) = B\left(h+\frac{1}{\beta}\right)\|x\|^2 = c_2\|x\|^2 . \]

Let us estimate the function \(v^{(h)}\) from below. Let \(h<\dfrac{1}{nL}\) (\(L\) is the Lipschitz constant of system (1.4)). Then

\[ \|x_{k,\omega}(x_{k_0},\eta_{k_0})\|^2 \geqslant \|x_{k_0}\|^2 \frac{1}{n^2}(1-nLh)^{2(k-k_0)} . \]

For the mathematical expectation we have

\[ M\left[\|x_k(x_{k_0},\eta_{k_0})\|^2 \mid x_{k_0},\eta_{k_0}\right] \geqslant \|x_{k_0}\|^2 \frac{1}{n^2}(1-nLh)^{2(k-k_0)} . \]

Further, one may write

\[ v^{(h)}(x,\eta)=\sum_{\tau=k}^{\infty} M\{\|x_\tau(x,\eta)\|^2\mid x,\eta\}h \ge \]

\[ \ge \frac{\|x\|^2}{n^2}\sum_{\tau=k}^{\infty}(1-nLh)^{2(\tau-k)}h = c_1^*\|x\|^2 \ge c_1\|x\|^2, \]

Here

\[ c_1^*=\frac{1}{n^3}\,\frac{1}{L(2-nLh)} \ge \frac{1}{2n^3L}=c_1. \]

Suppose now that \(h \ge \dfrac{1}{nL}\). Then

\[ v^{(h)}(x,\eta)=\sum_{\tau=k}^{\infty} M\{\|x_\tau(x,\eta)\|^2\mid x,\eta\}h \ge \|x\|^2\frac{1}{nL}>c_1\|x\|^2. \]

Thus, for the function \(v^{(h)}(x,\eta)\), the validity of the first line in conditions (3.2) has been proved.

It remains to show the negative definiteness of the first difference
\[ \frac{\Delta M[v^{(h)}]}{h} \]
of the mathematical expectation of the function \(v^{(h)}(x,\eta)\). To this end, in formula (2.1) we substitute the value (3.5) of the function \(v^{(h)}(x,\eta)\). Then

\[ \frac{\Delta M[v^{(h)}]}{h} = M\left[ \sum_{\tau=k+1}^{\infty} M\{\|x_\tau(x_{k+1},\eta_{k+1})\|^2\mid x_{k+1},\eta_{k+1}\} \right. \]

\[ \left. \mid x,\eta \right] - \sum_{\tau=k}^{\infty} M\{\|x_\tau(x,\eta)\|^2\mid x,\eta\}. \]

But, by virtue of the Markov property of the quantity \(\eta\),

\[ M\left[ \sum_{\tau=k+1}^{\infty} M\{\|x_\tau(x_{k+1},\eta_{k+1})\|^2\mid x_{k+1},\eta_{k+1}\} \mid x,\eta \right] = \]

\[ = \sum_{\tau=k+1}^{\infty} M\{\|x_\tau(x,\eta)\|^2\mid x,\eta\}. \]

Consequently, for the first difference the equality

\[ \frac{\Delta M[v^{(h)}]}{h} = \sum_{\tau=k+1}^{\infty} M\{\|x_\tau(x,\eta)\|^2\mid x,\eta\} - \]

\[ - \sum_{\tau=k}^{\infty} M\{\|x_\tau(x,\eta)\|^2\mid x,\eta\} = -\|x\|^2. \]

Thus, the constructed function \(v^{(h)}(x,\eta)\) possesses all the required properties, and the theorem is proved.

Lemma 3.2. Let the solution \(x=0\) of system (1.4) be exponentially stable in the mean uniformly with respect to \(h\). Then for any positive-definite form \(w(x,\eta)\), whose coefficients are continuous and bounded functions of \(\eta\), one can specify a positive-definite quadratic form \(v^{(h)}(x,\eta)\) satisfying the conditions of Theorem 3.1, and moreover

\[ \frac{\Delta M[v^{(h)}]}{h}=-w(x,\eta). \tag{3.6} \]

The proof of this lemma is carried out as follows. Assuming that

\[ w(x,\eta)=\sum_{i,\gamma=1}^{n} g_{i\gamma}(\eta)x^{(i)}x^{(\gamma)} \]

and forming the expression

\[ v^{(h)}(x,\eta)=\sum_{\tau=k}^{\infty} M\left[w\bigl(x_\tau(x;\eta),\eta_\tau(\eta)\bigr)\mid x,\eta\right]h, \tag{3.7} \]

analogous to (3.5), one can, taking into account the properties of \(w(x,\eta)\), estimate the quantity \(v^{(h)}(x,\eta)\) from above and below in the same way as in the case of Theorem 3.2. Here, by virtue of the condition of the lemma, the estimates will be uniform with respect to \(h\). Fulfillment of condition (3.6) is verified by direct calculation of the quantity
\[ \frac{\Delta M[v^{(h)}]}{h} \]
taking into account the Markov property of system (1.4).

The function \(v^{(h)}\) is a quadratic form; this can be verified by expressing the solutions \(x_\tau\) of system (1.4) in terms of the elements of the fundamental matrix of solutions \(x_\tau=F_{\tau,k}(\eta_k,\ldots,\eta_\tau)x_k\) and substituting them into (3.7).

§ 4. Let us consider the conditions under which the passage from the system of stochastic difference equations (1.4) to the corresponding system of differential equations (1.3) does not destroy the asymptotic stability in the mean of the zero solution (and conversely).

Theorem 4.1. If system (1.4) is asymptotically stable in the mean uniformly with respect to \(h\), then from this asymptotic stability of system (1.4) there will follow asymptotic stability in the mean of system (1.3).

Proof. By the hypothesis of the theorem, the solution of the linear system of difference equations (1.4) is asymptotically stable in the mean uniformly with respect to \(h\). Then, on the basis of Lemmas 3.1, 3.2, there exists a quadratic form

\[ v^{(h)}(x,\eta)=\sum_{i,\gamma=1}^{n} d_{i\gamma}^{(h)}(\eta)x^{(i)}x^{(\gamma)}, \]

satisfying the conditions

\[ c_1\|x\|^2 \leq v^{(h)}(x,\eta)\leq c_2\|x\|^2, \]

\[ \frac{\Delta M[v^{(h)}]}{h}\leq -c_3\|x\|^2, \]

where \(c_1,c_2,c_3\) are positive constants independent of \(h\).

Computing the first difference
\[ \frac{\Delta M[v^{(h)}]}{h} \]
of the mathematical expectation of the function \(v^{(h)}(x,\eta)\) along the solutions of system (1.4), we have, according to expression (2.5),

\[ \frac{\Delta M[v^{(h)}]}{h} = \bigl[v^{(h)}(x+\Delta x,\eta^{(j)})-v^{(h)}(x,\eta^{(j)})\bigr]\frac{(1-\alpha_{jj}h)}{h} + \]

\[ +\sum_{r\ne j}^{\infty}\bigl[v^{(h)}(x+\Delta x,\eta^{(r)})-v^{(h)}(x,\eta^{(j)})\bigr]\alpha_{jr}. \tag{4.1} \]

Here \(\Delta x=x_{k+1}-x_k\),

\[ \bigl[v^{(h)}(x+\Delta x,\eta^{(j)})-v^{(h)}(x,\eta^{(j)})\bigr]=\Delta v^{(h)} \]

is the ordinary increment of the function \(v^{(h)}(x,\eta)\) on an interval of length \(h\). Taking into account that \(v^{(h)}(x,\eta)\) is a quadratic form, we have

\[ \Delta v^{(h)} = \sum_{i=1}^{n}\frac{\partial v^{(h)}}{\partial x^{(i)}}\,\Delta x^{(i)} + \sum_{i,\gamma=1}^{n} \frac{\partial^2 v^{(h)}}{\partial x^{(i)}\partial x^{(\gamma)}} \frac{\Delta x^{(i)}\Delta x^{(\gamma)}}{2}. \]

Replacing the quantities \(\Delta x^{(i)}\), according to equation (1.4), and substituting the value of \(\Delta v^{(h)}\) into (4.1), we obtain

\[ \frac{\Delta M[v^{(h)}]}{h} = \left[ \sum_{i=1}^{n}\frac{\partial v^{(h)}}{\partial x^{(i)}}\, h\sum_{m=1}^{n}a_{im}(\eta^{(j)})x^{(m)} + \right. \]

\[ \left. + \sum_{i,\gamma=1}^{n} d_{i\gamma}^{(h)}(\eta^{(j)}) \sum_{m=1}^{n}a_{im}(\eta^{(j)})x^{(m)} \sum_{s=1}^{n}a_{\gamma s}(\eta^{(j)}) \times \right. \tag{4.2} \]

\[ \left. {}\times x^{(s)}\frac{h^2}{2} \right]\frac{(1-\alpha_{jj}h)}{h} + \sum_{r\ne j}^{\infty} \alpha_{jr}\bigl[v^{(h)}(x+\Delta x,\eta^{(r)})-v^{(h)}(x,\eta^{(j)})\bigr]. \]

Transform the expression \(v^{(h)}(x+\Delta x,\eta^{(r)})-v^{(h)}(x,\eta^{(j)})\). Then we obtain

\[ v^{(h)}(x+\Delta x,\eta^{(r)})-v^{(h)}(x,\eta^{(j)})= \]

\[ = v^{(h)}(x,\eta^{(r)})-v^{(h)}(x,\eta^{(j)}) + v^{(h)}(x+\Delta x,\eta^{(r)})- \]

\[ - v^{(h)}(x,\eta^{(r)}) = v^{(h)}(x,\eta^{(r)})-v^{(h)}(x,\eta^{(j)}) + \tag{4.3} \]

\[ + \sum_{i=1}^{n}\frac{\partial v^{(h)}}{\partial x^{(i)}}\, h\sum_{m=1}^{n}a_{im}(\eta^{(r)})x^{(m)} + \]

\[ + \sum_{i,\gamma=1}^{n} d_{i\gamma}^{(h)}(\eta^{(r)}) \sum_{m=1}^{n}a_{im}(\eta^{(r)})x^{(m)} \sum_{s=1}^{n}a_{\gamma s}(\eta^{(r)})x^{(s)} \frac{h^2}{2}. \]

Choose some small number \(h^*\in(0,h_0]\), and, taking the function \(v^{(h^*)}(x,\eta)\), compute its averaged derivative \(\dfrac{dM[v^{(h^*)}]}{dt}\) along solutions of system (1.3). We have

\[ \frac{dM[v^{(h^*)}]}{dt} = \sum_{i=1}^{n}\frac{\partial v^{(h^*)}}{\partial x^{(i)}} \sum_{m=1}^{n}a_{im}(\eta^{(j)})x^{(m)} + \]

\[ + \sum_{\substack{r=1\\ r\ne j}}^\infty \left[v^{(h^*)}(x,\eta^{(r)})-v^{(h^*)}(x,\eta^{(j)})\right]a_{jr}. \tag{4.4} \]

Assuming that \(kh^*\le t < (k+1)h^*\) and substituting (4.3), (4.4) into (4.2), we obtain, under the condition that \(x_k=x,\ \eta_k=\eta^{(j)}\),

\[ \frac{\Delta M[v^{(h^*)}]}{h} = \frac{dM[v^{(h^*)}]}{dt} + \sum_{r=1}^{\infty} p_{rj}(h^*) \left[ \sum_{i,\gamma=1}^{n} d_{i\gamma}^{(h^*)}(\eta^{(r)}) \times \right. \]

\[ \left. \times \sum_{m=1}^{n} a_{im}(\eta^{(r)})x^{(m)} \sum_{s=1}^{n} a_{\gamma s}(\eta^{(r)})x^{(s)} \right]\frac{h^*}{2} \tag{4.5} \]

\[ {}-h^* a_{jj} \sum_{i=1}^{n} \left(\sum_{\gamma=1}^{n} d_{i\gamma}^{(h^*)}(\eta^{(j)})x^{(\gamma)}\right) \left(\sum_{m=1}^{n} a_{im}(\eta^{(j)})x^{(m)}\right) + \]

\[ {}+h^* \sum_{\substack{r=1\\ r\ne j}}^{\infty} a_{jr} \left\{ \sum_{i=1}^{n} \left(\sum_{\gamma=1}^{n} d_{i\gamma}^{(h^*)}(\eta^{(r)})x^{(\gamma)}\right) \left(\sum_{m=1}^{n} a_{im}(\eta^{(r)})x^{(m)}\right) \right\}. \]

We shall show that for sufficiently small \(h^*\) the quantity

\[ \frac{dM[v^{(h^*)}]}{dt} \]

is negative definite.

Noting that in the square brackets on the right-hand side of expression (4.5) there is a positive definite form, we conclude that it is sufficient to estimate only the quantities

\[ C^{(h^*)}(x,\eta^{(s)}) = \sum_{i=1}^{n} \left( \sum_{\gamma=1}^{n} d_{i\gamma}^{(h^*)}(\eta^{(s)})x^{(\gamma)} \right) \left( \sum_{m=1}^{n} a_{im}(\eta^{(s)})x^{(m)} \right). \]

Taking into account the properties of the functions \(v^{(h)}(x,\eta)\) and the boundedness of the coefficients \(a_{ij}(\eta)\), we obtain that the form \(C^{(h^*)}(x,\eta^{(s)})\) satisfies the estimates

\[ c_4\|x\|^2 \le C^{(h^*)}(x,\eta^{(s)}) \le c_5\|x\|^2, \]

where \(c_4,\ c_5\) do not depend on \(h,\ s\) and may be either positive or negative. Then we have

\[ \frac{dM[v^{(h^*)}]}{dt} \le \frac{\Delta M[v^{(h^*)}]}{h^*} + |c_5|\|x\|^2 a_{jj}h^* + \]

\[ {}+ |c_4|\|x\|^2 \sum_{\substack{r=1\\ r\ne j}}^{\infty} a_{rj}h^* \le -c_3\|x\|^2 + \left(|c_5|+|c_4|\right)h^*a_{jj}\|x\|^2. \]

Hence it is clear that

\[ \frac{dM[v^{(h^*)}]}{dt} \]

is negative definite if \(h^*\) is chosen so that

\[ h^* < \frac{c_3}{|c_5|+|c_4|}. \]

Thus, the asymptotic stability in the mean of the difference equations (1.4), uniform with respect to \(h\), implies the asymptotic stability in the mean of the system (1.3) of stochastic differential equations. In this case the quadratic forms \(v^{(h)}\), guaranteeing

...which, for \(h \le h^{*}\), ensure the stability of the difference system, are Lyapunov functions also for the corresponding differential system.

The converse assertion is also true.

Theorem 4.2. If the solution \(x=0\) of system (1.3) is asymptotically stable in the mean, then one can indicate such an \(h^{*}\) that the solution \(x=0\) of system (1.4) will, for \(h \le h^{*}\), also be asymptotically stable in the mean uniformly with respect to \(h\) from \((0,h^{*}]\).

For the proof of this theorem one must use analogues of Lemmas 3.1 and 3.2, proved for differential equations in [4]. The course of the further reasoning is then analogous to that considered in the preceding theorem.

References

1. Lyapunov A. M. The general problem of the stability of motion. Gostekhizdat, Moscow–Leningrad, 1950.
2. Skalkina M. A. Dokl. Akad. Nauk SSSR, 104, No. 4, 1955.
3. Gnedenko B. V. Course in probability theory. Gostekhizdat, Moscow–Leningrad, 1950.
4. Kats I. Ya., Krasovskii N. N. PMM, 24, issue 5, 1960.
5. Lidskii E. A. Siberian Mathematical Journal, 4, issue 5, 1963.
6. Krasovskii N. N. Some problems in the theory of stability of motion. Fizmatgiz, Moscow, 1959.
7. Krasovskii N. N. PMM, vol. 25, issue 5, 1961.

Received by the editors
February 23, 1965.

Ural State University
named after A. M. Gorky