ON THE ANALYTICAL CONSTRUCTION OF A REGULATOR IN A SYSTEM WITH NOISE DEPENDING ON THE CONTROL

A. B. Kurzhanskii

The problem is considered of constructing an optimal regulator [1] that brings a linear system described by differential equations with random functions from a given initial position into a neighborhood of a specified terminal position over a given time interval. The problem is solved by the method of dynamic programming [2] for stochastic systems (see, for example, [3—4]).

1. Preliminary remarks. We study a controlled process described by the system of differential equations

\[ dz(t)=[A(t)z+b(t)u+y(t)]dt+r(t,u)dq. \tag{1.1} \]

Here \(z(t)\) is an \(n\)-dimensional vector of phase coordinates; \(A(t)\) and \(b(t)\) are, respectively, an \(n\)-dimensional matrix and an \(n\)-dimensional vector with continuous coefficients; \(u(z,y,t)\) is a scalar control; \(y(t)\) is an \(n\)-dimensional Markov random function describing a continuous Markov process [5], for which \(M\sum_i y_i^2(t)\) grows no faster than \(L\left(1+M\sum_i z_i^2(t)\right)\), \(L>0\), or else a bounded purely discontinuous [6] Markov process. The realizations of this process will, respectively, be almost all continuous or step functions. The range of variation of \(t\) will be taken to be the interval \([0,T]\).

By the quantity \(dq\) is meant a one-dimensional Brownian-motion process [6], i.e., a real Gaussian process with independent increments, for which

\[ M\{q(t_2)-q(t_1)\}=0; \]

\[ M\{[q(t_1)-q(t_2)]^2\}=|t_2-t_1|. \tag{1.2} \]

It is assumed that \(r_i(t,u)=r_i(t)u\) for all \(i=1,\ldots,n\) and that the coefficients of the vector \(r(t)\) are continuous functions. We shall also assume that

\[ \sum_{i=1}^{n} r_i^2(t)\ne 0 \]

for all \(t\in[0,T]\). We note that when \(y(t)\equiv0\) one has

\[ M\{dz_i dz_j\}=r_i(t,u)r_j(t,u)dt=k_{ij}(t,u)dt=r_i(t)r_j(t)u^2dt. \tag{1.3} \]

We shall, naturally, assume that the quadratic form of the matrix \(\{k_{ij}\}\) is positive definite.

In system (1.1) the vector \(y(t)\) has the meaning of a random load, and \(r(t,u)dq\) of a noise caused by the control. The solution of equation (1.1)

can, according to [6], be interpreted as a stochastic diffusion process.

With respect to the function \(u(z,y,t)\) we assume that it is continuous and satisfies the conditions

\[ \left|u(z',y',t)-u(z'',y'',t)\right|\leq k\left\|\{z',y'\}-\{z'',y''\}\right\|, \tag{1.4} \]

\[ |u(z,y,t)|\leq k(1+\|z\|^2+\|y\|^2)^{\frac12},\quad k>0. \]

Then, for any initial vector, there exists [6] a solution of system (1.1), almost all realizations of which are continuous on \([0,T]\) and which satisfies the relation

\[ M\left\{\max_t \|z(t)\|^2\right\}<\infty,\quad \text{where } t\in[0,T]. \tag{1.5} \]

Consequently, in what follows we in fact have a \(2n\)-dimensional Markov process \(\{z,y\}\), where \(z(t)\) is a solution of system (1.1), and \(y(t)\) is specified according to the description given above.

2. Statement of the problem. Consider equation (1.1).

Problem 1-A. Given a time interval \([0,T]\) and the coordinates of two points \(z'\) and \(z''\) of the phase space. It is required to find a control function \(u^0(z,y,t)\), satisfying conditions (1.4), which would ensure

\[ M\left[\|z(T)-z''\|^2 / z(0)=z'\right]=\min, \]

if it is known that \(y(0)=y_0\).

Here the symbol of conditional mathematical expectation is written. The norm \(\|z\|\) of the vector \(z\) is Euclidean.

Thus, the purpose of the problem is to synthesize such a control that would transfer the point \(z\) of the phase space from the prescribed initial state \(z'\) into a neighborhood of the prescribed point \(z''\) over a time interval \(T<\infty\), and would at the same time ensure the best, in the mean-square sense, average approach to the point \(z''\). The problem under consideration can be assigned to the general class of deterministic and probabilistic problems on the analytical construction of regulators [1—3] and is, consequently, one of the problems of the theory of optimal systems. A distinctive feature of the problem stated is the fact that the scattering coefficient \(r(t,u)\) of the process \(z\) depends on the control \(u\). Therefore, increasing the magnitude of the control causes an increase in the scatter of the random variables. This circumstance naturally limits the control.

For what follows it is convenient to transform the original problem 1-A. System (1.1), by means of the substitution \(z(t)=x(t)+z''\), is reduced to the form

\[ \frac{dx}{dt}=A(t)x+b(t)u+m(t)+y(t)+r(t,u)\frac{dq}{dt}, \tag{2.1} \]

where \(m(t)\) is a vector with components \(m_i(t)=\sum_{k=1}^n a_{ik}(t)z_k''\).

In the new variables we have the following problem.

Problem 1-B. Construct a control \(u^0(x,y,t)\) which brings the phase trajectory, under system (2.1), from the position \(x(0)=z'-z''=x_0\)

\(^1\) In the case of probabilistic systems, exact hitting of the point \(z''\) is impossible because of the irremovable error caused by the scatter of random variables.

1. Differential Equations

into a neighborhood of zero over the time \(T\) and at the same time ensure a minimum of the quantity

\[ M\{\|x(T)\|^2/x(0)=x_0\}=\min, \tag{2.2} \]

if it is known that \(y(0)=y_0\).

3. Optimality criterion. The following sufficient condition for optimality is valid; it is a modification, for our case, of the criterion [7].

Let functions \(V(x,y,t)\) and \(u^0(x,y,t)\) be found such that

\[ \text{1)}\qquad V(x,y,T)=\sum_{i=1}^{n}x_i^2(T)=\|x(T)\|^2, \tag{3.1} \]

2) for any \(x\) and \(t\in[0,T]\) one has

\[ \left(\frac{dMV}{dt}\right)_{u^0} = \min_u\left[\left(\frac{dMV}{dt}\right)_u\right]=0. \tag{3.2} \]

Then \(u^0(x,y,t)\) is the optimal control, and

\[ V(x_0,y_0,t)=M\{\|x(T)\|^2/x(t)=x_0,\ y(t)=y_0\mid u^0\}. \tag{3.3} \]

The quantity \(\left(\dfrac{dMV}{dt}\right)_u\) has the meaning of the averaged derivative along the motions of the system at the point \((x,y,t)\):

\[ \left(\frac{dMV}{dt}\right)_u = \lim_{\Delta t\to 0}\frac{1}{\Delta t} \bigl[M\{V(x(t+\Delta t,u), \]

\[ y(t+\Delta t),t+\Delta t)/x(t)=x,\ y(t)=y\}-V(x(t,u),y(t),t)\bigr]. \tag{3.4} \]

We shall use the indicated criterion to construct the desired control.

4. Construction of the optimal control.

In problems of the type under consideration, it is expedient [7] to seek the function \(V(x,y,t)\) in the form

\[ V=\sum_{i,j=1}^{n}\alpha_{ij}(t)x_i x_j+\sum_{i=1}^{n}\beta_i(t,y)x_i+\gamma(t,y). \tag{4.1} \]

We shall show that such a function satisfying (3.2) does indeed exist. According to (3.1), we take

\[ \alpha_{ij}(T)=\delta_{ij};\qquad \delta_{ij}= \begin{cases} 1 & \text{if } i=j,\\ 0 & \text{if } i\ne j; \end{cases} \qquad \beta_i(T,y)=0 \]

for all \(i=1,\ldots,n\); \(\gamma(T,y)=0\).

To determine the coefficients \(\alpha_{ij}(t)\), \(\beta_i(t,y)\), \(\gamma(t,y)\), we use condition (3.2).

Computing \(\dfrac{dMV}{dt}\) (a detailed computation of the analogous derivative is given in [3], and for problems of the type under consideration—in [7], [8]), we obtain,

\[ \frac{dMV}{dt} = \sum_{i,j=1}^{n} \frac{d\alpha_{ij}(t)}{dt} x_i x_j+ \]

\[ + \sum_{i,j=1}^{n} \sum_{k=1}^{n} \bigl(\alpha_{ik}(t)\alpha_{kj}(t)+\alpha_{jk}(t)\alpha_{ki}(t)\bigr)x_i x_j+ \]

\[ + \sum_{i,j=1}^{n} \bigl[\alpha_{ij}(t)x_j\bigl(b_i(t)u+m_i(t)+y_i(t)\bigr)+ \]

\[ +\alpha_{ij}(t)x_i\bigl(b_j(t)u+m_j(t)+y_j(t)\bigr)\bigr]+ \]

\[ +\frac{1}{2}\sum_{i,j=1}^{n}\alpha_{ij}(t)k_{ij}(t)u^2+ \sum_{i=1}^{n}\beta_i(t,y)\times \tag{4.2} \]

\[ \times\left[\sum_{k=1}^{n}\alpha_{ik}(t)x_k+b_i(t)u+y_i(t)+m_i(t)\right]+ \]

\[ +\sum_{i=1}^{n}x_i\frac{dM\beta_i(t,y)}{dt} +\frac{dM\gamma(t,y)}{dt}=0. \]

In accordance with condition (3.2), differentiating (4.2) with respect to \(u\), and taking (1.3) into account, we obtain

\[ u=-2\left[\sum_{i,j=1}^{n}\alpha_{ij}(t)r_i(t)r_j(t)\right]^{-1}\times \]

\[ \times\left[\sum_{i,j=1}^{n}\alpha_{ij}(t)b_i(t)x_j+ \frac{1}{2}\sum_{i=1}^{n}\beta_i(t,y)b_i\right]. \tag{4.3} \]

Denoting

\[ \frac{1}{2}\sum_{i,j=1}^{n}\alpha_{ij}(t)r_i(t)r_j(t)=c(t), \]

substituting (4.3) into (4.2), and then equating to zero the expressions at like products \(x_i x_j\) and the remaining terms, we obtain the following equations:

\[ \frac{d\alpha_{ij}}{dt} +\sum_{k=1}^{n}\bigl(\alpha_{ik}\alpha_{kj}(t)+\alpha_{jk}\alpha_{ki}(t)\bigr)- \]

\[ -\frac{1}{c(t)}\sum_{k,l=1}^{n}\alpha_{ki}\alpha_{lj}b_k(t)b_l(t) \tag{4.4} \]

\[ (i,j=1,\ldots,n), \]

\[ \sum_{i=1}^{n}\left\{ \frac{dM\beta_i(t,y)}{dt}x_i +2\sum_{j=1}^{n}\alpha_{ij}(t)x_j\bigl(y_i(t)+m_i(t)\bigr)+ \right. \tag{4.5} \]

\[ + \sum_{j=1}^{n} \beta_i(t,y)a_{ij}(t)x_j - \frac{1}{c(t)} \left[ \sum_{j=1}^{n}\alpha_{ji}(t)b_j(t)x_i \right] \times \]

\[ \times \left[ \sum_{j=1}^{n}\beta_j(t,y)b_j(t) \right] \Bigg\}=0, \tag{4.5} \]

\[ \frac{dM\gamma(t,y)}{dt} + \sum_{i=1}^{n}\beta_i(t,y)\bigl(y_i(t)+m_i(t)\bigr) - \]

\[ -\frac{1}{4c(t)} \sum_{i,j=1}^{n}\beta_i(t,y)\beta_j(t,y)b_i(t)b_j(t)=0. \tag{4.6} \]

The system (4.4) should be solved with the initial conditions \(\alpha_{ij}(T)=\delta_{ij}\).

Let us note at once that if in equations (2.1) one sets \(y(t)\equiv 0\) and \(m(t)\equiv 0\), then problem 1-B can be solved by the method described above, with the sole difference that the function \(V\) is sought in the form
\[ V(x,t)=\sum_{i,j}\alpha_{ij}(t)x_i x_j. \]
To find the coefficients \(\alpha_{ij}(t)\) one again obtains the system (4.4), which is to be solved with the same initial conditions \(\alpha_{ij}(T)=\delta_{ij}\) as before. Thus, if \(V(x,t)\) exists, then it has the meaning
\[ \min M\{\|x(T)\|^2/x(t)=x\} \]
by virtue of the system (2.1), under the condition of absence of the load \(y(t)\) and with zero vector \(m(t)\).

We shall show that the system (4.4) is solvable. In a neighborhood of the point \(T\) a local solution exists, since in this case the right-hand sides of the system (4.4) are defined and satisfy the Lipschitz conditions \(\left(\sum_i r_i^2(t)\ne 0\right)\). Suppose that the solution of the system under consideration has already been extended to the interval \([T_1,T]\), where \(T_1>0\). This means, in particular, that
\[ V(x(T_1),T_1)=\sum_{i,j}\alpha_{ij}(T_1)x_i x_j \]
is strictly positive on the sphere \(\|x(T_1)\|=1\) and, moreover,
\[ c(T_1)=\frac12 V(r(T_1),T_1)>\eta(T_1)>0. \]

Consider a point \(t\in[0,T_1]\), so close to \(T_1\) that the solution of the system exists at it by virtue of the continuity of the quantities \(\alpha_{ij}(t)\), \(r_i(t)\). According to what was said above,
\[ V(x,t)=\sum_{i,j=1}^{n}\alpha_{ij}(t)x_i x_j = M\{\|x(T)\|^2/x(t)=x\}, \]
where \(\|x(t)\|=1\).

Let us compute the quantity \(M\{\|x(T)\|^2/x(t)=x\}\).

Writing the solution of the system (2.1) in the form
\[ x(T)=F(T)F^{-1}(t)x(t) + \int_t^T F(T)F^{-1}(\tau)b(\tau)u(\tau)\,d\tau + \]
\[ + \int_t^T F(T)F^{-1}(\tau)r(\tau)u(\tau)\,dq \]

and taking into account the special form of the right-hand side of this expression, we obtain

\[ M\{\|x(T)\|^2/x(t)=x\} = \sum_{i=1}^{n} \left[ \sum_{j=1}^{n} f_{ij}(T,t)x_j(t) + \right. \]

\[ \left. + \int_t^T h_i(T,\tau)u(\tau)\,d\tau \right]^2 + \int_t^T \sigma^2(T,\tau)u^2(\tau)\,d\tau. \tag{4.7} \]

Here \(F(t)\) is the fundamental matrix of the homogeneous system corresponding to (2.1); \(F^{-1}(t)\) is the matrix inverse to \(F(t)\); \(f_{ij}(T,t)\) are the elements of the product \(F(T)F^{-1}(t)\);

\[ h_{ij}(T,\tau)=\sum_{j=1}^{n} f_{ij}(T,\tau)b_j(\tau);\quad \sigma^2(T,\tau)=\sum_{i=1}^{n}\left[\sum_{j=1}^{n} f_{ij}(T,\tau)r_j(\tau)\right]^2 . \]

Consider expression (4.7). As is known from the theory of optimal control (see, for example, [8]), for the controllable system \(\dot x=A(t)x+b(t)u\) one can always choose numbers \(\varepsilon>0\), \(\gamma>0\) such that, when

\[ \int_t^T u^2(\tau)\,d\tau \leq \varepsilon, \]

the inequality

\[ \sum_{i=1}^{n}\left[\sum_{j=1}^{n} f_{ij}(T,t)x_j(t)+ \int_t^T h_i(T,\tau)u(\tau)\,d\tau\right]^2>\gamma \]

holds. The numbers \(\varepsilon\) and \(\gamma\) can be chosen so that the preceding inequalities are valid for all \(t\in[0,T_1]\). Thus, if at the instant \(t\) it turns out that

\[ \int_t^T u^2(\tau)\,d\tau \leq \varepsilon, \]

then from the first term on the right-hand side of (4.7) we obtain

\[ M\{\|x(T)\|^2/x(t)=x\}>\gamma . \]

If, on the other hand,

\[ \int_t^T u^2(\tau)\,d\tau>\varepsilon, \]

then from the second term on the right-hand side of (4.7) we obtain that

\[ M\{\|x(T)\|^2/x(t)=x\}>\sigma^*\varepsilon, \]

where \(\sigma^*=\min_t \sigma^2(T,t)\) for \(t\in[0,T]\). From the obtained relations it follows directly that one can choose a number \(\delta>0\) such that

\[ M\{\|x(T)\|^2/x(t)=x\}>\delta . \tag{4.8} \]

If it proves possible to continue the solution in the direction of decreasing \(t\), then for subsequent values of \(t\) from \([0,T_1]\) condition (4.8) is also satisfied with the same number \(\delta\). The possibility of continuing the solution is, in turn, ensured, on the one hand, by condition (4.8), and, on the other hand, by the fact that, by virtue of (3.5), (1.3), the relation

\[ V(x,t)\leq M\{\|x(T)\|^2/x(t)=x\}_{u=0}<k(t)<\infty,\quad \|x\|=1 \]

holds.

Thus, on the interval \([0,T]\) there exists a unique solution of system (4.4) satisfying the conditions \(a_{ij}(T)=\delta_{ij}\).

Let us pass to equation (4.5). Consider the system

\[ \frac{dx_i}{dt} = \sum_{j=1}^{n} a_{ij}(t)x_j - \frac{1}{c(t)}\,b_i(t)\sum_{j,k=1}^{n}\alpha_{jk}(t)b_j(t)x_k \tag{4.9} \]

\[ (i=1,\ldots,n). \]

Then equation (4.5) can be written in the form

\[ \left( \frac{dM\left\{\sum_i \beta_i(t,y)x_i\right\}}{dt} \right)_{(4.9)} = -2\sum_{i,j=1}^{n}\alpha_{ij}(t)(y_i(t)+m_i(t))x_j . \tag{4.10} \]

The derivative on the left-hand side of (4.10) has been computed by virtue of system (4.9). According to [7], the function satisfying equation (4.10) has the form

\[ \sum_{i=1}^{n}\beta_i(t,y)x_i = 2\int_t^T M\left[ \sum_{i,j=1}^{n}\alpha_{ij}(\tau)(y_i(\tau)+ \right. \]

\[ \left. +m_i(\tau))x_j(\tau)/x(t)=x,\ y(t)=y \right]\,d\tau . \]

Here \(x(\tau)\) is the solution of system (4.8). Let us denote the fundamental matrix of this system by the symbol \(\Phi(t,\tau)\) \((\Phi(t,t)=E)\), and its elements by the symbols \(\varphi_{ij}(t,\tau)\). Then the last equation leads to the relations

\[ \beta_k(t,y)=2\int_t^T \sum_{i,j=1}^n \alpha_{ij}(\tau)\varphi_{jk}(t,\tau)M[y_i(\tau)/y(t)=y]\,d\tau+ \]

\[ +2\int_t^T \sum_{i,j=1}^n \alpha_{ij}(\tau)\varphi_{jk}(t,\tau)m_i(\tau)\,d\tau \quad (k=1,\ldots,n). \tag{4.11} \]

Denoting

\[ 2\int_t^T \sum_{i,j=1}^n \alpha_{ij}(\tau)\varphi_{jk}(t,\tau)M[y_i(\tau)/y(t)=y]\,d\tau=\mu_k(t,y) \]

and

\[ 2\int_t^T \sum_{i,j=1}^n \alpha_{ij}(\tau)\varphi_{jk}(t,\tau)m_i(\tau)\,d\tau=\nu_k(t), \]

we obtain that

\[ \beta_k(t,y)=\mu_k(t,y)+\nu_k(t);\quad \beta_k(T,y)=0. \tag{4.12} \]

To compute \(\mu_k\), it is necessary to know the quantities \(M[y_i(\tau)/y(t)=y]\), whereas \(\nu_k\) does not depend on \(y(t)\) and is computed directly from the quantities \(\alpha_{ij}(t), \varphi_{jk}, m_i(t)\).

Similarly, from equation (4.6) we obtain

\[ \gamma(t,y)=\int_t^T M\left\{\sum_{i=1}^n \beta_i(\tau,y)(y_i(\tau)+m_i(\tau))-\right. \]

\[ \left. -\frac{1}{4c(t)}\sum_{i,j=1}^n \beta_i(\tau,y)\beta_j(\tau,y)b_i(\tau)b_j(\tau)/y(t)=y \right\}d\tau. \tag{4.13} \]

It is evident from this that \(\gamma(T,y)=0\).

Introduce in (4.3), (4.12) the notation

\[ -\frac{1}{c(t)}\sum_{i=1}^n \alpha_{ij}(t)b_j(t)=\lambda_i(t), \]

\[ -\frac{1}{2c(t)}\sum_{i=1}^n \beta_i(t,y)b_i(t)= \]

\[ =-\frac{1}{2c(t)}\sum_{i=1}^n(\mu_i+\nu_i)b_i(t)=\mu(t,y)+\nu(t). \]

Then

\[ u^0(x,y,t)=\sum_{i=1}^n \lambda_i(t)x_i(t)+\mu(t,y)+\nu(t). \tag{4.14} \]

Thus, for Problem 1-B the following conclusion is valid.

1. Problem 1-B has a unique solution. The optimal control has the form indicated in (4.14), and consists of the linear part \(u_1^0 =\)

\[ = \sum_{i=1}^{n} \lambda_i(t)x_i(t) \]

and of a certain addition depending on the realized values of the process \(y(t)\). The linear term \(u_1^0\) of the control \(u^0(x,y,t)\) is also the solution of the analogous problem for system (2.1) with \(y(t)\equiv 0,\ m(t)\equiv 0\). The term \(\mu(t,y)\), which takes account of the realized values of the load \(y(t)\), is constructed on the basis of a forecast of the future mean values of this load; \(\nu(t)\) is an ordinary nonrandom quantity, computed according to (4.11), (4.12), and depending on the position of the point \(z'\). For \(z'=0\), \(\nu(t)\equiv 0\).

The part of the optimal control (4.14) that is linear with respect to \(x\) may be given the following interpretation. Consider the system

\[ \frac{dx}{dt}=A(t)x+b(t)u, \tag{4.15} \]

where \(A(t)\), \(b(t)\) are the same as in (2.1). System (4.15) is deterministic.

Problem 2-A. Determine, for the initial position \(x(t)\) specified at the instant \(t\), the control \(u(x,t)\) minimizing the functional

\[ \int_t^T c(\tau)u^2(\tau)\,d\tau+\|x(T)\|^2 \tag{4.16} \]

on the solutions of system (4.15).

If the function \(V(x,t)\) for this system is sought in the form \(\sum_{i,j=1}^{n} a_{ij}(t)\times\)

\[ \times x_i x_j \]

and, moreover, one assumes that \(c(\tau)\) in (4.15) has the form \(c(\tau)=\dfrac{1}{2}\times\)

\[ \times \sum_{i,j=1}^{n} a_{ij}(\tau) r_i(\tau) r_j(\tau), \]

where the quantities \(r_k(\tau)\), \((k=1,\ldots,n)\), are taken from system (2.1), then, in order to find the coefficients \(a_{ij}(t)\), we again arrive at system (4.4), which must be solved with the same initial conditions \(a_{ij}(T)=\delta_{ij}\). As was shown above, system (4.4) is always solvable on \([0,T]\) in the case considered; therefore the coefficients \(a_{ij}(t)\) exist, and the new problem described has meaning. The optimal control for problem (4.15), (4.16) then coincides with the linear term of the control (4.14). The function \(V(x,t)=\sum_{i,j} a_{ij}(t)x_i x_j\) for system (4.15) has the meaning of the minimum of the functional (4.16) on the solutions of system (4.15) issuing at the instant \(t\) from the point \(x\).

Consequently, the following conclusion is valid.

1. The linear part of the optimal control (4.14) of Problem 1-B minimizes, on the solutions of the deterministic system (4.15), the functional (4.16) of Problem 2-A, in which \(c(t_0)\) is the minimum value at \(t=t_0\) of this same functional on the solutions issuing at the instant \(t_0\) from the point with coordinates

\[ \frac{1}{\sqrt{2}}\,r_i(t_0),\quad (i=1,\ldots,n), \]

where \(r_i(t)\) are the scattering coefficients in system (2.1) of Problem 1-B.

Remark. If one assumes that system (4.15) is completely controllable uniformly in \(t\) [10], then for problem 1-B, in which \(y(t)\equiv 0\) and \(m(t)\equiv 0\), one may consider a limiting passage analogous to those considered, for example, in [9], [10]. Such a limiting passage, however, is possible only for sufficiently small coefficients \(r(t)\). Let us explain this assertion. Consider the auxiliary problem.

Problem 2-B. Find a control \(u^*(x,t)\) minimizing, on the solutions of system (4.15), the functional

\[ V(x,t)=\int_t^\infty \left[\sum_i x_i^2(\tau)+u^2(\tau)\right]\,d\tau . \]

Under the assumptions made concerning (4.15), this problem is solvable, and

\[ V^*(x,t)=\int_t^\infty \left[\sum_{i=1}^n x_i^2(\tau)+\bigl(u^*(\tau)\bigr)^2\right]\,d\tau \]

is a Lyapunov function given by the positive definite quadratic form

\[ V^*(x,t)=\sum_{i,j}\beta_{ij}(t)x_i x_j, \]

which admits the estimate [10]:

\[ \sum_{i,j=1}^n \beta_{ij}(t)x_i x_j \leq \gamma \sum_{i=1}^n x_i^2 . \]

Here \(\gamma\) is a constant depending on the coefficients of system (4.15). The optimal control in this case depends linearly on the current coordinates:

\[ u^*(x,t)=\sum_{i=1}^n k_i(t)x_i . \]

For the derivative \(dV^*(x,t)/dt\) along system (4.15), the equality

\[ \left(\frac{dV^*}{dt}\right)_{(4.15)} = -\sum_{i=1}^n x_i^2(t) - \left[\sum_{i=1}^n k_i(t)x_i(t)\right]^2 \]

is valid. Adding to both sides the sum

\[ \frac{1}{2}\sum_{i,j=1}^n \beta_{ij}(t)r_i(t)r_j(t) \left[\sum_{i=1}^n k_i(t)x_i(t)\right]^2, \]

we obtain

\[ \left(\frac{dV^*(x,t)}{dt}\right)_{(2.1)} = -\sum_{i=1}^n x_i^2(t) - \left[\sum_{i=1}^n k_i(t)x_i(t)\right]^2 + \]

\[ +\frac{1}{2}\sum_{i,j=1}^n \beta_{ij}(t)r_i(t)r_j(t) \left[\sum_{i=1}^n k_i(t)x_i(t)\right]^2 . \]

If the quantities \(r_i(t)\) are now chosen so that

\[ \sum_{i=1}^n r_i^2(t) < \frac{2}{\gamma},\qquad t\in[0,\infty), \tag{4.17} \]

then we arrive at the conclusion [11] that the form \(\sum_{i,j}\beta_{ij}(t)x_i x_j\) is a Lyapunov function ensuring asymptotic stability in the mean of system (2.1) under the control

\[ u^*(x,t)=\sum_i k_i(t)x_i, \]

corresponding to problem 2-B. Thus, for sufficiently small values \(r_i(t)\), there exists a control under which

\[ \lim M\{\|x(T)\|^2/x(t)=x\}\to 0. \]

As shown in [10], the control \(u^*\) and the function \(V^*(x,t)\) of problem 2-B are obtained by passage to the limit from the analogous problem considered on a finite time interval, i.e.,

\[ u^*(x,t)=\lim_{T\to\infty} u^*(x,t,T), \qquad V^*(x,t)=\lim_{T\to\infty} V^*(x,t,T). \]

Returning to problem 1-B, by virtue of the optimality of the control \(u^0(t)\) for this problem we obtain

\[ 0 \le M\{\|x(T)\|^2 / x(t)=x\}_{u^0(x,t,T)} \le M\{\|x(T)\|^2 / x(t)=x\}_{u^*(x,t,T)} . \]

Passing in both parts of the inequality to the limit as \(T \to 0\) and taking into account the stabilizability of system (2.1) by the control \(u^*(x,t)\), we have

\[ \lim_{T\to\infty} M\{\|x(T)\|^2 / x(t)=x\}_{u^0(x,t,T)} \ge 0; \qquad \overline{\lim}_{T\to\infty} M\{\|x(T)\|^2 / x(t)=x\}_{u^0(x,t,T)} \le 0, \]

whence

\[ \lim_{T\to\infty} M\{\|x(T)\|^2 / x(t)=x\}_{u^0(x,t,T)} = 0. \]

References

1. Letov A. M. Avtomatika i telemekhanika, 22, no. 4, 1961.
2. Bellman R., Glicksberg I., Gross O. Some questions in the mathematical theory of control processes. IL, Moscow, 1962.
3. Krasovskii N. N. PMM, 25, no. 1, 1960.
4. Krasovskii N. N., Lidskii E. A. Avtomatika i telemekhanika, 22, nos. 9–11, 1961.
5. Gnedenko B. V. Course in Probability Theory. Fizmatgiz, 1962.
6. Doob J. Stochastic Processes. IL, Moscow, 1956.
7. Krasovskii N. N. SMZh, vol. IV, no. 3, 1963.
8. Krasovskii N. N. PMM, 23, no. 4, 1959.
9. Lidskii E. A. PMM, 27, no. 1, 1963.
10. Kalman R. E. Contributions to the theory of optimal control. Proceedings of the Symposium on Ord. Diff. Equations. Bol. Soc. Mat. Mex., Mexico City, 1961.
11. Katz I. Ya., Krasovskii N. N. PMM, 24, no. 5, 1960.

Received by the editors
September 28, 1964

Ural State University
named after A. M. Gorky