On the Calculation of Optimal Control in a System with Incomplete Information

A. B. Kurzhanskii

A controlled system described by stochastic differential equations is considered. It is assumed that information on the current phase coordinates of the system may arrive at the control device at random moments of time with a Poisson distribution, and that an increase in the frequency of the Poisson process leads to the growth of additional random disturbances in the system. Under these assumptions, the optimal control and the frequency of transmission of feedback signals are found. The problem is solved by the methods of Lyapunov functions and dynamic programming. The work is related to the investigations [1, 2].

1. Preliminary information. Consider a system described by differential equations of the following type [3, p. 248]:

\[ \frac{dx}{dt}=A(t)x+b(t)u+R(t,\lambda,u)\frac{dq}{dt}, \tag{1.1} \]

where \(x\) is an \(n\)-dimensional vector of phase coordinates; \(A(t)\), \(b(t)\) are, respectively, an \((n\times n)\)-matrix and an \(n\)-vector with continuous coefficients; \(u\) is a scalar control; the components \(q_k\) of the \(n\)-vector \(q\) are independent Brownian-motion processes that randomly disperse the motion \(x(t)\); \(R(t,\lambda,u)\) is a continuous \((n\times n)\)-matrix possessing the property that its coefficients increase without bound as \(\lambda\to\infty\).

It is assumed that the values of the phase coordinates \(x(t)\) are not completely known at time \(t\) in the device that produces the control \(u(t)\). In other words, continuous measurement of these quantities is impossible. We shall suppose that information concerning the current coordinates of the system arrives at the regulator, which forms the control, at discrete random instants of time distributed according to Poisson’s law [4, p. 295], with frequency \(\lambda\).

The problem consists in synthesizing such a control \(u^0\) and in finding such a frequency \(\lambda^0\) that would ensure

\[ \min M\left[\int_{0}^{T} F(x(t),u(t),t)\,dt+G(x(T))/x(0)=x_0\right], \tag{1.2} \]

where \(F, G\) are given nonnegative functions, the symbol \(M[\alpha/\beta]\) denotes conditional mathematical expectation, and \(T\) is a given terminal time.

The best conditions for solving the problem from the point of view of the information flow would be obtained for \(\lambda\to\infty\), since in this case the signals on the state of the object would arrive practically continuously. However, such a solution

does not hold, because an increase in the frequency \(\lambda\) leads to an unbounded growth of the disturbances caused by the random noise \(R\dfrac{dq}{dt}\).

It follows from this that there is an optimal problem for a system with incomplete information*. The problem under consideration can be assigned to the class of stochastic problems on the analytical construction of regulators [7, 2].

Remark. 1.1. The frequency \(\lambda^0\) that is optimal in the sense described can be sought in the class of constant quantities or in the class of continuous functions of \(t\). In the present work the first case is considered.

1.2. It is assumed that the organs forming the control can remember the values \(u(t)\) during the time interval \([0,T]\).

2. Formulation of the problem.

Let there be a certain system

\[ \frac{dx}{dt}=A(t)x+b(t)u+r^{(1)}(t)\frac{dq_1}{dt}+\lambda r^{(2)}(t)\frac{dq_2}{dt}, \tag{2.1} \]

where \(A(t)\), \(b(t)\), \(\dfrac{dq_i}{dt}\) \((i=1,2)\) are described in Sec. 1. The vectors \(r^{(1)}(t)\), \(r^{(2)}(t)\) are continuous on \([0,T]\). It is assumed that at the initial instant \(t_0=0\) the position \(x_0\) is known. Thereafter information about the magnitude \(x\) (the values of the coordinates of the vector \(x(t)\)) is supplied to the regulator at random Poisson instants of time \(t_1,t_2,\ldots,t_n\). The number \(n\) of signal arrivals during the time interval \([0,T]\) is a random variable depending on \(T\) and on the frequency \(\lambda\). Moreover \(M[n(T,\lambda)]\to\infty\) for given \(T\) and as \(\lambda\to\infty\). Thus, at an arbitrary instant of time \(t\ge0\), the regulator knows the values of the quantities \(x(\tau)\) at the instant \(\tau=t_k\) of arrival of the last signal about the vector \(x\), and the control quantity \(u(t+\vartheta)\) for all \(\vartheta\) from the interval \([\tau-t,0)\).

Problem 2.1. For a given interval \([0,T]\) and initial position \(x(0)=x_0\), find a control \(u^0[t,\tau,x(\tau),\lambda^0,u^0(t+\vartheta),\tau-t\le\vartheta<0]\) and a constant frequency \(\lambda^0(x_0,T)\) such that, on the solutions of system (2.1), there is ensured

\[ M\left[\int_0^T u^2(t)\,dt+\|x(T)\|^2 \big/ x(0)=x_0\right] \tag{2.2} \]

(the norm \(\|x\|\) of the vector \(x\) is Euclidean).

Let us note that, by virtue of the properties of the Brownian-motion process, for system (2.1) the equalities

\[ M\{dx_i dx_j-dx_i^* dx_j^*\}=r_i^{(1)}(t)r_j^{(1)}(t)\,dt+\lambda^2 r_i^{(2)}(t)r_j^{(2)}(t). \tag{2.3} \]

are valid. Here the quantities \(dx_i^*\) \((i=1,\ldots,n)\) are the components of the vector

\[ dx^*=(A(t)x(t)+b(t)u(t))\,dt, \]

which characterizes the mean change of the vector \(x\), if at the instant \(t\) the quantities \(x(t)\), \(u(t)\) are present. Equality (2.3) is a consequence of the independence of the quantities \(q_1,q_2\).

Introduce the notation

\[ \eta(t,\tau)=\{x(\tau),\,u(t+\vartheta),\ \tau-t\le\vartheta<0\}. \tag{2.4} \]

The quantity \(\eta(t,\tau)\), where \(\tau\) is the instant of arrival of the last signal about the vector \(x\), determines the probabilistic state \(x(t)\) of the object at the instant \(t\), constructed on the basis of the quantities accessible to measurement. In accordance with the principles of dynamic programming [8], the quantity

* A very general scheme of such problems is considered in [5]. A classification of control systems depending on the properties of the object, the regulator, and the nature of the information entering the latter is given in [6].

\[ \min_{u,\lambda}\left[\int_t^T u^2(\tau)\,d\tau+\|x(T)\|^2/\eta(t,\tau)\right],\quad 0\leq \tau \leq t<T \]
may be regarded as a functional defined on \(\eta(t,\tau)\). In this case the control \(u^0\) should also be sought in the form of a functional
\[ u^0(t,\eta(t,\tau)). \tag{2.5} \]

In other words, the quantity \(\eta(t,\tau)\) here plays the role of sufficient coordinates [5].

The quantity \(\eta(t,\tau)\) creates an aftereffect in the system; therefore the solution of equation (2.1) for \(u=u^0\) (2.5) can no longer be interpreted as an ordinary diffusion process. Nevertheless, under the assumptions made above concerning \(A(t)\), \(b(t)\), \(r(t)\), and assuming that the control \(u\) satisfies sufficient smoothness conditions ([3], p. 251), one can verify by the method of successive approximations that the solution of system (2.1) exists on the entire interval \([0,T]\) and \(\max_{t\in[0,T]}\|x(t)\|^2/x(0)=x_0<\infty\). A functional \(u(t,\eta)\) of the type (2.5) that ensures the existence of a solution of system (2.1) will be called an admissible control. We shall seek the control \(u^0\) in the class of admissible quantities.

3. Optimality criterion. Problem (2.1) can be formulated in a more general form as applied to the system
\[ \frac{dx}{dt}=f(t,x,u)+R(t,\lambda,u)\frac{dq}{dt}, \tag{3.1} \]
where \(f(t,x,u)\) is a continuous vector function ensuring the existence of a solution of (2.1). Let us consider a criterion of optimality for the control \(u^0\) and the frequency \(\lambda^0\), which solve Problem 2.1 for system (3.1) with criterion (1.2).

We shall agree to denote by the symbol
\[ \left(\frac{dM\{V\}}{dt}\,\middle|\,t,\eta(t,\tau),\lambda,u\right) \]
the averaged derivative of the functional \(V(t,\eta(t,\tau))\) at the point \(t,\eta(t,\tau)\) along the solutions of system (3.1) under the control \(u\) and frequency \(\lambda\), taking into account the possibility of receiving a signal about the value of the vector \(x\) over an infinitely small time interval \([t,t+\Delta t]\). (Derivatives of this kind are described visually in [2, 9]. The rigorous meaning of these quantities is studied in the theory of Markov processes [10].) We shall immediately assume that the optimal control in the criterion considered below is a sufficiently regular functional ensuring the existence of solutions of system (3.1), almost all realizations of which are continuous. We shall also say that the functional \(w(t,\eta(t,\tau))\) is continuous if
\[ \lim_{t\to\theta-0} w(t,\eta(t,\tau)) = \lim_{t\to\theta-0} M\,[w(\theta,\eta(\theta,s(\theta)))\mid \eta(t,\tau)]. \]
Here the mathematical expectation is calculated along the motions of system (3.1) generated by \(\eta(t,\tau)\).

Criterion 3.1. Suppose that the following have been found: 1) a positive-definite continuous functional \(V(t,\eta(t,\tau),\lambda)\), 2) a functional \(u^0(t,\eta(t,\tau),\lambda)\), and 3) a number \(\lambda^0\), satisfying the conditions

3.1-a.
\[ V(T,\eta(T,\tau),\lambda)=M\,[G(x(T))/\eta(T,\tau),\lambda] \]
for all \(\tau\in[0,T]\), any finite \(\lambda\), and with mathematical expectation calculated by virtue of system (3.1) under the control \(u\in\eta(T,\tau)\).

3.1-b.
On the solutions of (3.1), for all \(\eta(t,\tau)\), \(0\leq \tau \leq t \leq T\), and any \(\lambda < \infty\), the following relation holds:
\[ [dM\{v\}/dt \mid t,\eta(t,\tau),\lambda,u^0] +M[F(x(t),u^0(t),t)\mid \eta(t,\tau)] \]
\[ =\min_u \{[dM\{v\}/dt \mid t,\eta(t,\tau),\lambda,u] +M[F(x(t),u(t),t)\mid \eta(t,\tau)]\}=0. \]

3.1-c.
\[ V(0,\eta(0,0),\lambda^0)=\min_\lambda V(0,\eta(0,0),\lambda) \qquad (\eta(0,0)=x(0)). \]

Then \(u^0(t,\eta(t,\tau),\lambda^0)\) is the optimal control, \(\lambda^0\) is the optimal frequency, and
\[ V(0,\eta(0,0),\lambda^0) =\min_{u,\lambda} M\left[\int_0^T F(x,u,t)\,dt+G(x(T))\mid x(0)=x_0\right] \]
by virtue of system (3.1).

Consider the functional \(\widetilde V\), which, under the assumptions made above, with respect to the solutions of (3.1), will be continuous in \(s\):
\[ \widetilde V(s,\eta(t,\tau),\lambda,u) =M[V(s,\eta(s,\sigma),\lambda)\mid \eta(t,\tau),u,\lambda]. \]

Computing the right derivative for \(V(s,\eta(t,\tau),\lambda,u^0)\), we find
\[ \frac{d\widetilde V(s,\eta(t,\tau),\lambda,u^0)}{ds} = M\left\{ \left[\frac{dM\{V\}}{dt}\mid s,\eta(s,\sigma),\lambda,u^0\right] \mid \eta(t,\tau),\lambda,u^0 \right\}. \tag{3.2} \]

The last equality holds by virtue of the analogy between ordinary Markov processes and the process under consideration with aftereffect, if it is regarded in the space of quantities \(\eta(t,\tau)\).

Similarly, as a consequence of condition 3.1-b, we have
\[ d\widetilde V(s,\eta(t,\tau),\lambda,u^0)/ds = -M\{F(x(s),u^0(s),s)\mid \eta(t,\tau),\lambda,u^0\}. \tag{3.3} \]

Integrating this equation from \(t\) to \(T\) and taking 3.1-a into account, we arrive at the equalities:
\[ \widetilde V(t,\eta(t,\tau),\lambda,u) =\widetilde V(T,\eta(t,\tau),\lambda,u^0)+ \]
\[ +M\left[\int_t^T F(x(s),u^0(s),s)\,ds\mid \eta(t,\tau),\lambda,u^0\right] =M\left[G(x(T))\mid \eta(t,\tau),\lambda,u^0\right]+ \]
\[ +M\left[\int_t^T F(x(s),u^0(s),s)\,ds\mid x(0),u^0,\lambda\right]. \tag{3.4} \]

From (3.3), in particular, we obtain
\[ V(0,\eta(0,0),\lambda) =\widetilde V(0,\eta(0,0),\lambda,u) =M[G(x(T))\mid x(0),u^0,\lambda]+ \]
\[ +M\left[\int_t^T F(x(s),u^0(s),s)\,ds\mid x(0),u^0,\lambda\right] \tag{3.5} \]
or, taking into account condition 3.1-c and substituting \(\lambda^0\),
\[ V(0,\eta(0,0),\lambda^0) =\widetilde V(0,\eta(0,0),\lambda^0,u^0) =\min_\lambda \widetilde V(0,\eta(0,0),\lambda,u^0). \]

If on the interval \([t,T]\) a control \(u\) different from \(u^0\) is chosen, then, according to (3.1) and condition 3.1-b,
\[ d\widetilde V(s,\eta(s,\tau),\lambda,u)/ds \geq -M[F(x(s),u(s),s)\mid \eta(t,\tau),u,\lambda]. \]

Integrating the last inequality, we obtain

\[ \tilde V(t,\eta(t,\tau),\lambda,u)\leq \tilde V(T,\eta(t,\tau),\lambda,u)+ \]

\[ +\int_t^T M\{F(x(s),u(s),s)\mid \eta(t,\tau),u,\lambda\}\,ds = M\left\{\left[G(x(T))+\right.\right. \]

\[ \left.\left.+\int_t^T F(x(s),u(s),s)\,ds\right]\middle|\,\eta(t,\tau),u,\lambda\right\}. \tag{3.6} \]

In particular,

\[ M\left\{\left[G(x(T))+\int_0^T F(x(s),u(s),s)\,ds\right]\middle|\,x_0,u^0,\lambda\right\} \leq M\left\{\left[G(x(T))+\right.\right. \]

\[ \left.\left.+\int_0^T F(x(s),u(s),s)\,ds\right]\middle|\,x_0,u,\lambda\right\}. \]

If it turns out that \(\lambda\ne \lambda^0\), then we have the additional condition

\[ V(0,\eta(0,0),\lambda^0) = M\left\{\left[G(x(T))+\int_0^T F(x,u,s)\,ds\right]\middle|\,x_0,u^0,\lambda^0\right\} \leq \]

\[ \leq M\left\{\left[G(x(T))+\int_0^T F(x,u,s)\,ds\right]\middle|\,x_0,u^0,\lambda\right\}. \tag{3.7} \]

Relations (3.6), (3.7) prove the optimality of \(u^0\) and \(\lambda^0\).

Remark. 3.1. The proof presented is not complete, since in deriving relations (3.2), (3.4), (3.6) in the general case of system (3.1) it is necessary to justify the interchangeability of the operations \(M\{\}\) and \(d/dt\), \(\int\{\}\,ds\). A detailed consideration of these questions lies outside the scope of the present work, whose aim is to derive concrete relations determining the optimal control. For the concrete system considered below, all the required properties hold.

3.2. The meaning of the quantity \((dM\{V\}/dt\mid t,\eta(t,\tau),\lambda,u)\) can be interpreted as follows. Taking into account the property of the Poisson process that the probability of the arrival of a single signal in time \(\Delta t\) ([4], p. 296) is \(P_1(\Delta t)=\lambda\Delta t+o(\Delta t)\approx \lambda\Delta t\), and computing the quantity \(dM\{V\}/dt\), we obtain

\[ (dM\{V\}/dt\mid t,\eta(t,\tau),\lambda,u) = \lim_{\Delta t\to 0}\frac{1}{\Delta t} \left\{M\left[V(t+\Delta t,\eta(t+\Delta t,t^*),\lambda)\middle|\,\eta(t,\tau)\right]\right. \]

\[ \left.-V(t,\eta(t,\tau),\lambda)\right\}. \]

Here the quantity \(t^*\) is such that either \(t^*=t'\), \(t'\in(t,t+\Delta t]\), or \(t^*=\tau\), depending on whether during the time \(\Delta t\) a signal about the value of the vector \(x\) has arrived or not. Then

\[ (dM\{V\}/dt\mid t,\eta(t,\tau),\lambda,u) = \lim_{\Delta t\to 0}\frac{1}{\Delta t} \left\{\left[M\left[V(t+\Delta t,\eta(t+\Delta t,\tau),\lambda)\middle|\,\eta(t,\tau)\right]\right.\right. \]

\[ \left.\left.-V(t,\eta(t,\tau),\lambda)\right](1-\lambda\Delta t) + \left[M\left[V(t+\Delta t,\eta(t+\Delta t,t'),\lambda)\middle|\,\eta(t,\tau)\right]\right.\right. \]

\[ \left.\left.-V(t,\eta(t,\tau),\lambda)\right]\lambda\Delta t\right\} = (\partial M\{V\}/\partial t\mid t,\eta(t,\tau),\lambda,u)+ \]

\[ +\lambda\left[M\left[V(t,\eta(t,t),\lambda)\middle|\,\eta(t,\tau)\right] - V(t,\eta(t,\tau),\lambda)\right]. \tag{3.8} \]

The symbol \((\partial M\{V\}/\partial t\mid t,\eta(t,\tau),\lambda,u)\), thus, denotes the averaged derivative of the function \(V(t,\eta(t,\tau))\) under the condition that during the interval \([t,t+\Delta t]\) no signal about the value of the vector \(x\) arrives.

3.3. The optimal problem, analogous to problem 2.1 but with minimization of the functional (1.2) on solutions of system (3.1), can also be solved in this case if one seeks the frequency \(\lambda\) in the form of a variable quantity (see Remark 1.1). In this case the frequency \(\lambda\) will be a functional depending, in the general case, on the past values of the control and of the frequency known on \([t^{--}t,t]\),

\[ \lambda=\lambda(t,\eta(t,\tau)), \tag{3.9} \]

where \(\eta(t,\tau)=\{x(\tau),\lambda(\vartheta),u(\vartheta),\vartheta\in[\tau-t,t),t\}\). For the case under discussion one can formulate a criterion which is proved, by the same scheme, to be criterion 3.1. Let us formulate this criterion.

Criterion 3.2. Suppose that there have been found 1) a positive-definite continuous functional \(V(t,\eta(t,\tau))\), 2) functionals \(u^0(t,\eta(t,\tau))\), \(\lambda^0(t,\eta(t,\tau))\), satisfying the following conditions:

3.2-a) \(V(T,\eta(T,\tau))=M[G(x(T))\mid\eta(T,\tau)]\) for all \(t\in[0,T]\) and conditional expectation computed on solutions of (3.1) under \(u\in\mathcal U(T,t)\);

3.2-b) for any \(\eta(t,\tau)\), \(t\), \(0\le\tau\le t<T\), on solutions of (3.1) the relation holds

\[ [dM\{V\}/dt\mid t,\eta(t,\tau),u^0,\lambda^0]+M[F(x(t),u^0(t),t)\mid\eta(t,\tau)]= \]

\[ =\min_{u,\lambda}\{[dM\{V\}/dt\mid t,\eta(t,\tau),u,\lambda]+M[F(x(t),u(t),t)\mid\eta(t,\tau)]\}=0. \]

Then \(u^0(t,\eta(t,\tau))\), \(\lambda^0(t,\eta(t,\tau))\) are the optimal control and frequency, and

\[ V(0,\eta(0,0))=\min_{u,\lambda}M\left[\,G(x(T))+\int_0^T F(x(\tau),u(\tau),\tau)\,d\tau\ \middle|\ x_0,u,\lambda\right] \]

by virtue of system (3.1).

Here it is assumed that the functionals \(\lambda(t,\eta(t,\tau))\), \(u(t,\eta(t,\tau))\) are such that a solution of system (3.1) exists.

4. Solution of problem 2.1. To solve problem 2.1 it is necessary to find a functional \(V\) that would satisfy conditions 3.1-a, b, c by virtue of system (2.1) under the optimality criterion (2.2).

Consider the function \(V^*(t,x)=\sum_{i,j=1}^{n}\alpha_{ij}(t)x_i x_j\), positive definite and such that

\[ \alpha_{ij}=\delta_{ij},\quad \text{i.e.}\quad V^*(T,x(T))=\sum_{i=1}^{n}x_i^2(T). \tag{4.1} \]

(Here \(\delta_{ij}=1\) for \(i=j\), \(\delta_{ij}=0\) for \(i\ne j\).) We shall seek \(V\) in the following form: for \(\tau\le t<T\),

\[ V(t,\eta(t,\tau),\lambda)=M[V^*(t,x(t))\mid\eta(t,\tau),\lambda]+\varphi(t,\tau,\lambda), \tag{4.2} \]

where \(\varphi(t,\tau,\lambda)\) is a certain as yet undetermined positive functional satisfying the condition

\[ \varphi(T,\tau,\lambda)=0 \quad \text{for any } \tau \text{ from } [0,T]. \tag{4.3} \]

We shall show that \(\alpha_{ij},\varphi\) can be chosen so that the functional \(V\) satisfies conditions 3.1-a, b, c.

By virtue of (4.1), (4.3) we have

\[ V(T,\eta(T,\tau),\lambda)=M[\|x(T)\|^2\mid\eta(T,\tau),\lambda]. \]

Let us compose the equation corresponding to condition 3.1-b, using the property of repeated conditional mathematical expectations as applied to the process in the space of quantities \(\eta\). Taking (3.8) into account, we obtain

\[ (dM\{V\}/dt\mid t,\eta(t,\tau),\lambda,u)=(\partial M\{V^*\}/\partial t\mid t,\eta(t,\tau),\lambda,u)+ \]

\[ +\lambda\{M[M[V^*(t,x(t))\mid\eta(t,t)]\mid\eta(t,\tau),\lambda]-M[V^*(t,x(t))\mid\eta(t,\tau),\lambda]\}+ \]

\[ +(dM\{\varphi\}/dt\mid t,\eta(t,\tau),\lambda,u)=(\partial M\{V^*\}/\partial t\mid t,\eta(t,\tau),\lambda,u)+ \]

\[ +(dM\{\varphi\}/dt\mid t,\eta(t,\tau),\lambda,u)=0. \]

Calculating the last expression by virtue of system (2.1)*, we have

\[ (dM\{V\}/dt \mid t,\eta(t,\tau),\lambda,u)= \sum_{i,j=1}^{n}\left[ \frac{d a_{ij}(t)}{dt}+ \right. \]

\[ \left. +\sum_{k=1}^{n}\{a_{ik}(t)a_{kj}(t)+a_{jk}(t)a_{ki}(t)\} \right]M[x_i(t)x_j(t)\mid \eta(t,\tau),\lambda]+ \]

\[ +2\sum_{i,j=1}^{n}a_{ij}(t)b_j(t)u(t)M[x_i(t)\mid \eta(t,\tau),\lambda]+ \tag{4.5} \]

\[ +\frac{1}{2}\sum_{i,j=1}^{n}a_{ij}(t)r_i^{(1)}(t)r_j^{(1)}(t) +\frac{1}{2}\lambda^2\sum_{i,j=1}^{n}a_{ij}(t)r_i^{(2)}(t)r_j^{(2)}(t)+ \]

\[ +(dM\{\varphi\}/dt\mid t,\eta(t,\tau),\lambda,u)+u^2(t)=0. \]

Denoting \(x_i^*(t,\tau)=M[x_i(t)\mid \eta(t,\tau),\lambda]\), \([x_i(t,\tau)x_j(t,\tau)]^*=M[x_i(t)\times x_j(t)\mid \eta(t,\tau),\lambda]\), differentiating (4.5), we find the control \(u^0\) for \(\tau<t<T\)

\[ u^0=-\sum_{i,j=1}^{n}a_{ij}(t)b_j(t)x_i^*(t,\tau). \tag{4.6} \]

Substituting (4.6) into equation (4.5) and equating the expression for identical products \(x_i^*(t,\tau)x_j^*(t,\tau)\) to zero, we obtain the system of equations for determining \(a_{ij}(t)\):

\[ \frac{d a_{ij}(t)}{dt} =-\sum_{i=1}^{n}\bigl(a_{ik}(t)a_{kj}(t)+a_{jk}(t)a_{ki}(t)\bigr)+R_{ij}(t), \tag{4.7} \]

\[ (i,j=1,\ldots,n) \]

where

\[ R_{ij}(t)= \left[\sum_{k=1}^{n}a_{ki}(t)b_k(t)\right] \left[\sum_{k=1}^{n}a_{jk}(t)b_k(t)\right]. \]

The remaining terms give the equation

\[ \sum_{i,j=1}^{n}\left[ \frac{d a_{ij}(t)}{dt} +\sum_{k=1}^{n}\bigl(a_{ik}(t)a_{kj}(t)+a_{jk}(t)a_{ki}(t)\bigr) \right]\sigma_{ij}(t,\tau,\lambda)+ \]

\[ +(dM\{\varphi\}/dt\mid t,\eta(t,\tau),\lambda,u)+ \]

\[ +\frac{1}{2}\sum_{i,j=1}^{n}a_{ij}(t)\bigl(r_i^{(1)}(t)r_j^{(1)}(t)+\lambda^2 r_i^{(2)}(t)r_j^{(2)}(t)\bigr). \tag{4.8} \]

Here \(\sigma_{ij}(t,\tau,\lambda)=[x_i(t,\tau)x_j(t,\tau)]^*-x_i^*(t,\tau)x_j^*(t,\tau)\).

* Here the property of repeated conditional mathematical expectations has also been used:

\[ (\partial M\{V\}/\partial t\mid t,\eta(t,\tau),\lambda,u) = M[(\partial M\{V^*\}/\partial t\mid t,\eta(t,t),\lambda,u)\mid \eta(t,\tau),\lambda]. \]

A quantity analogous to \((\partial M\{V^*\}/\partial t\mid t,\eta(t,t),\lambda,u)\) is computed in detail in [9].

We compute \(\sigma_{ij}(t,\tau)\). Using Cauchy’s formula, we obtain

\[ \begin{aligned} x_i(t)={}& \sum_{k=1}^{n} f_{ik}(t,\tau)x_k(\tau) +\int_{\tau}^{t}\sum_{k=1}^{n} f_{ik}(t,\vartheta)b_k(\vartheta)u(\vartheta)\,d\vartheta + \\ &+\int_{\tau}^{t}\sum_{k=1}^{n} f_{ik}(t,\vartheta)r_k^{(1)}(\vartheta)\,dq_1 +\lambda \int_{\tau}^{t} f_{ik}(t,\vartheta)r_k^{(2)}(\vartheta)\,dq_2 . \end{aligned} \tag{4.9} \]

The symbols \(f_{ik}(t,\tau)\) denote the elements of the fundamental matrix \(F(t,\tau)\) \((F(\tau,\tau)=E)\) of the homogeneous system corresponding to equations (2.1).

Taking into account that \(q_1, q_2\) are independent Brownian-motion processes, we find

\[ \begin{aligned} \sigma_{ij}(t,\tau,\lambda) &=\sigma_{ij}^{(1)}(t,\tau)+\lambda^2\sigma_{ij}^{(2)}(t,\tau)= \\ &=\int_{\tau}^{t} f_i^{(1)}(t,\vartheta)f_j^{(1)}(t,\vartheta)\,d\vartheta +\lambda^2\int_{\tau}^{t} f_i^{(2)}(t,\vartheta)f_j^{(2)}(t,\vartheta)\,d\vartheta , \end{aligned} \tag{4.10} \]

where

\[ f_s^{(k)}(t,\vartheta)=\sum_{q=1}^{n} f_{sq}(t,\vartheta)r_q^{(k)}(\vartheta),\qquad \int_{\tau}^{t} f_i^{(k)}(t,\vartheta)f_j^{(k)}(t,\vartheta)\,d\vartheta =\sigma_{ij}^{(k)}\quad (k=1,2). \]

\(\|\sigma_{ij}(t,\tau)\|\) has the meaning of the matrix of second central moments of the random variable \(x(t)\), whose distribution is determined by the known condition \(\eta(t,\tau)\).

System (4.7) should, according to (4.1), be solved with the initial conditions \(a_{ij}(T)=\hat a_{ij}\). The existence and uniqueness of the solution of this system follow from the arguments given in [9], where this system gives the solution of the optimal problem with complete information.

Consider equation (4.8). According to (4.8), (4.7), we have

\[ (dM\{\varphi\}/dt\mid t,\eta(t,\tau),\lambda,u) =-\sum_{i,j=1}^{n} R_{ij}(t)\sigma_{ij}(t,\tau,\lambda)- \]

\[ -\frac{1}{2}\sum_{i,j=1}^{n} a_{ij}(t)\bigl(r_i^{(1)}(t)r_j^{(1)}(t)+\lambda^2 r_i^{(2)}(t)r_j^{(2)}(t)\bigr). \tag{4.11} \]

Integrating the preceding equation, we obtain

\[ \varphi(t,\tau,\lambda) =\int_{t}^{T}\sum_{i,j=1}^{n} R_{ij}(\vartheta)\, M[\sigma_{ij}(\vartheta,\xi(\vartheta),\lambda)\mid t,\tau,\lambda]\,d\vartheta+ \]

\[ +\frac{1}{2}\int_{t}^{T}\sum_{i,j=1}^{n} a_{ij}(\vartheta) \bigl(r_i^{(1)}(\vartheta)r_j^{(1)}(\vartheta) +\lambda^2 r_i^{(2)}(\vartheta)r_j^{(2)}(\vartheta)\bigr)\,d\vartheta . \tag{4.12} \]

Expression (4.12) was obtained by interchanging the operations of integration and averaging in the first term. The mathematical expectation at the instant \(\vartheta\) is computed taking into account only the Poisson jumps (since the quantities \(\sigma_{ij}(t,\tau,\lambda)\) do not depend on \(u,x\)), under the condition that the most recent signal, relative to the instant \(t\), concerning the vector \(x\) arrived at the instant \(\tau\). The circumstance that, as the quantity \(\vartheta\) changes from \(t\) to \(T\), the value \(\xi\) of the time of arrival of the most recent signal relative to \(\vartheta\) also changes is indicated here by the symbol \(\xi(\vartheta)\). In particular, \(\xi(t)=\tau\). Here \(\xi(\vartheta)\), for fixed \(\vartheta\), is a random variable.

If the derivative in (4.11) is written out in detail (see Remark 3.2), then we obtain that \(\varphi(t,\tau)\) satisfies the following partial differential equation:

\[ \frac{\partial \varphi(t,\tau,\lambda)}{\partial t} +\lambda(\varphi(t,t,\lambda)-\varphi(t,\tau,\lambda))= \]

\[ =-\sum_{i,j=1}^{n} R_{ij}(t)\sigma_{ij}(t,\tau,\lambda)- \]

\[ -\frac{1}{2}\sum_{i,j=1}^{n}\alpha_{ij}(t) \bigl(r_i^{(1)}(t)r_j^{(1)}(t)+\lambda^2 r_i^{(2)}(t)r_j^{(2)}(t)\bigr) \tag{4.13} \]

with boundary condition \(\varphi(T,\tau)=0\).

Thus, the functional (4.2) satisfies conditions 3.1 a, b. This makes it possible to find the optimal control \(u^0(t,\eta(t,\tau))\) for any fixed frequency \(\lambda\). The quantity \(u^0\) then has the form (4.6).

From (4.9) we find

\[ x_i^*(t,\tau)=\sum_{k=1}^{n} f_{ik}(t,\tau)x_k(\tau) +\int_{\tau}^{t}\sum_{k=1}^{n} f_{ik}(t,\vartheta)b_k(\vartheta)u(\vartheta)\,d\vartheta. \]

Then the optimal control \(u^0\), taking into account the notation

\[ -\sum_{i,j=1}^{n}\alpha_{ij}(t)b_j(t)f_{ik}(t,\tau)=\psi_k(t,\tau), \]

\[ -\sum_{i,j=1}^{n}\alpha_{ij}(t)b_j(t) \int_{\tau}^{t} f_{ik}(t,\vartheta)b_k(\vartheta)u(\vartheta)\,d\vartheta = \int_{\tau}^{t}\rho_k(t,\vartheta)u(\vartheta)\,d\vartheta \]

can be represented in the form

\[ u^0(t,\eta(t,\tau))= \sum_{k=1}^{n} \left[ \psi_k(t,\tau)x_k(\tau)+ \int_{\tau}^{t}\rho_k(t,\vartheta)u(\vartheta)\,d\vartheta \right]. \tag{4.14} \]

To determine the optimal frequency, we use condition 3.1 b. Taking (3.5) into account, we obtain that the optimal frequency \(\lambda^0\) is determined from the minimization with respect to the variable \(\lambda\) of the function \(V(0,\eta(0,0),\lambda)\). Noting that the quantities \(\alpha_{ij}\) do not depend on \(\lambda\), we obtain that \(\lambda^0\) is determined from the condition

\[ \frac{\partial \varphi(0,0,\lambda)}{\partial \lambda}=0. \]

Hence we find that \(\lambda^0\) is a solution of the equation

\[ \frac{\partial \varphi(0,0,\lambda)}{\partial \lambda} = \frac{\partial}{\partial \lambda} \left[ \int_{0}^{T}\sum_{i,j=1}^{n} R_{ij}(\vartheta) M\{\sigma_{ij}^{(1)}(\vartheta,\xi(\vartheta))+ \right. \]

\[ \left. +\lambda^2\sigma_{ij}^{(2)}(\vartheta,\xi(\vartheta))\mid t,\tau,\lambda\}\,d\vartheta + \right. \]

\[ \left. +\frac{1}{2}\int_{0}^{T}\sum_{i,j=1}^{n}\alpha_{ij}(\vartheta) \bigl(r_i^{(1)}(\vartheta)r_j^{(1)}(\vartheta) +\lambda^2 r_i^{(2)}(\vartheta)r_j^{(2)}(\vartheta)\bigr)\,d\vartheta \right] =0. \tag{4.15} \]

Calculating the last expression and examining it, one can verify that \(0<\lambda_0<\infty\).

Thus, we arrive at the following conclusions. Problem 2.1 has a unique solution. The optimal control \(u^0\) (4.6), (4.14) at time \(t\) is a functional depending on the known values of the coordinates \(x_i(\tau)\)

at time \(\tau\) and the control values over the elapsed interval \([\tau,t)\) from the moment \(\tau\) of receipt of the last signal about the value of \(x\) up to the current time \(t\).

The optimal value of the performance index (2.2) of system (2.1) is equal to

\[ V(0,\eta(0,0),\lambda^0) = M\left[ \left. \int_0^T (u^0(\vartheta))^2\,d\vartheta+\|x(T)\|^2 \,\right|\, x_0,u^0,\lambda^0 \right] = \]

\[ = \sum_{i,j=1}^{n} a_{ij}(0)x_i(0)x_j(0)+\varphi(0,0,\lambda^0). \tag{4.16} \]

From formula (4.6) it is clear that the optimal control has the same form as in the system with complete information, in which \(\lambda\) is simply a numerical coefficient multiplying \(r^{(2)}(t)\), but with the difference that the values of the current coordinates are here replaced by their expected values \(x_i^*(t,\tau)\).

The first term on the right-hand side of (4.16) has the meaning

\[ \min_u \left[ \int_0^T u^2(\tau)\,d\tau+\|x(T)\|^2 \right] \]

under the initial condition \(x(0)\) for the case of complete information in system (2.2) and under the condition that \(r^{(1)}(t)=0\) and \(r^{(2)}(t)=0\), while the second is caused by the presence of random disturbances and incomplete feedback and increases as \(T\to\infty\).

Let us calculate the second term in (4.16) starting from expression (4.12). For this purpose, note that the distribution function of the length of the interval between two successive signals, under the condition that the first of them arrived at time \(\sigma\), is
\(F_s(s-\sigma)=1-e^{-\lambda(s-\sigma)}\), and the corresponding distribution density is
\(f_\sigma(s-\sigma)=\lambda e^{-\lambda(s-\sigma)}\).

Then we obtain

\[ \varphi(t,\tau,\lambda) = \int_t^T \sum_{i,j=1}^{n} R_{ij}(s) \left[ \int_t^s \sigma_{ij}(s,\sigma)\lambda e^{-\lambda(s-\sigma)}\,d\sigma + e^{-\lambda(s-t)}\sigma_{ij}(s,\tau) \right]ds + \]

\[ + \frac{1}{2} \int_t^T \sum_{i,j=1}^{n} \alpha_{ij}(\vartheta) \left( r_i^{(1)}(\vartheta)r_j^{(1)}(\vartheta) + \lambda^2 r_i^{(2)}(\vartheta)r_j^{(2)}(\vartheta) \right)d\vartheta . \tag{4.17} \]

Substituting (4.17) into equation (4.13), we verify that \(\varphi(t,\tau,\lambda)\) indeed satisfies this equation.

Remarks. 4.1. The optimal frequency \(\lambda^0\) does not depend on the coordinates \(x_0\) of the initial point of phase space, but changes when the length of the interval \([0,T]\) changes.

4.2. The reasoning carried out in the present paragraph also makes it possible to solve the following problem.

Find

\[ \min_{\lambda}\max_{\|x_0\|=1} M\left[ \left. \left[ \int_0^T u^2(\tau)\,d\tau+\|x(T)\|^2 \right] \,\right|\, x_0,u^0,\lambda \right], \tag{4.18} \]

where \(u^0\) is chosen according to problem 2.1. The desired solution is obtained directly from (4.16) by maximizing it over all \(x_0\) lying on the unit sphere.

4.3. Problem 2.1 can also be considered in the case when system (2.1) is subjected to the action of a random load described by a Markov-type process [10]. This means that system (2.1) takes, for example, the following form:

\[ \frac{dx}{dt} = A(t)x+b(t)u+m(t)\omega(t) + r^{(1)}(t)\frac{dq_1}{dt} + \lambda r^{(2)}(t)\frac{dq_2}{dt}, \tag{4.19} \]

where \(m(t)\) is a continuous vector, and \(\omega(t)\) is a continuous or purely discontinuous Markov function.

It may be assumed that information on the current value of \(\eta(t)\) is unknown, and arrives with a delay or else at random instants of time \(t_k,\ k=1,\ldots,n(T,\lambda^*)\), distributed according to Poisson with frequency \(\lambda^*\), different from \(\lambda\) or equal to \(\lambda\).

In particular, if it is assumed that \(\lambda^*=\lambda\), and it is assumed that signals about the vector \(x(t)\) and about the load \(\eta(t)\) can arrive only simultaneously, then the optimal control corresponding to problem 2.1 with the additions described has the form

\[ u^0(t,\eta(t,\tau)) = -\sum_{i,j=1}^{n}\alpha_{ij}(t)b_i(t)x_j^*(t,\tau) -\frac{1}{2}\sum_{i=1}^{n}\beta_i(t,\tau)b_i(t), \tag{4.20} \]

where \(\alpha_{ij}(t)\) are the same as for \(\omega(t)\equiv 0\). The control (4.20) differs from the one found earlier by the presence of an additional term caused by the load \(\omega(t)\).

The quantities \(\beta_i(t,\tau)\) are determined by the integral

\[ \beta_i(t,\tau) = M\left[ \int_t^T \xi_i(s,\sigma(s))\,M(\omega(s)/\omega(\sigma))\,ds \,\middle|\, \tau,\xi(t,\tau),\omega(\tau) \right], \]

where \(\xi_i(s,\sigma)\) are random variables depending on the instants of arrival of signals and determined by the quantities \(a_{ij}(t), b_j(t), \alpha_{ij}(t)\).

The optimal frequency is determined according to the same scheme as before, but with the difference that the term caused by the presence of the load \(\omega(t)\) appears in the free term \(\varphi(t,\tau,\lambda)\).

5. Limiting transition

For the system (2.1) with incomplete information one can consider the following problem.

Problem 5.1. Under the conditions of problem 2.1, find a control \(u_\infty\) and a constant frequency \(\lambda_\infty\) such that they ensure the minimum of the quantity

\[ \lim_{T\to\infty}\frac{1}{T} M\left[ \int_0^T u^2(t)\,dt+\|x(T)\|^2 \,\middle|\,x(0)=x_0 \right] =\min. \]

Problem 5.1 can be solved by passing to the limit from the solution of problem 2.1, if it is assumed that the frequency \(\lambda\) is fixed, with subsequent minimization of the optimal value of the functional of problem 5.1 over all \(\lambda\).

Let us show this. Let the system

\[ \frac{dx}{dt}=A(t)x+b(t)u \tag{5.1} \]

be stabilizable [11] for arbitrary initial positions. A sufficient condition for this property is uniform controllability [12] of the system under consideration for all \(t\ge 0\). (Necessary and sufficient conditions for stabilizability of a stationary system of type (5.1) are set forth in [11].) We shall also assume that the coefficients \(r^{(1)}(t)\), \(r^{(2)}(t)\) of the system (2.1) are bounded. Consider the auxiliary problem.

Problem 5.2. On the solutions of the system (5.1), with complete information, minimize the functionals:

\[ \text{a) }\quad I_T(t,x(t),u) = \int_t^T u^2(\tau)\,d\tau+\|x(T)\|^2 \]

and

\[ \text{b) }\quad F(t,x(t),u) = \int_t^\infty u^2(\tau)\,d\tau. \]

We shall denote the optimal values of these functionals as follows:

\[ I_T(t,x(t),u^0)=V^{(1)}(t,x(t),T)=\sum_{i,j=1}^{n}\alpha_{ij}(t,T)x_i(t)x_j(t), \]

\[ F(t,x(t),u^0)=V^{(2)}(t,x(t))=\sum_{i,j=1}^{n}\beta_{ij}(t)x_i(t)x_j(t). \]

Let us note that \(V^{(1)}(t,x(t),T)\) coincides for every \(T\) with the form \(V^*(t,x(t))\) (4.1) of problem (2.1), posed on the interval \([0,T]\), since the coefficients \(\alpha_{ij}\) for \(V^{(1)}\) and \(V^*\) coincide and are determined from the system (4.7). The quantities \(V,\varphi,\alpha_{ij}\) for problem 2.1 will henceforth be denoted by \(\alpha_{ij}=\alpha_{ij}(t,T)\), \(V=V(t,\eta(t,\tau),\lambda,T)\), \(\varphi=\varphi(t,\tau,\lambda,T)\), thereby emphasizing their dependence on the length of the interval \([0,T]\). Let us also note that the solution of problem 5.2 b) is obtained by passing to the limit from the solution of the problem of minimizing the functional
\[ F_T(t,x(t),u)=\int_t^T u^2(\tau)\,d\tau, \]
\[ F_T(t,x(t),u^0)=V_T^{(2)}(t,x(t))=\sum_{i,j=1}^{n}\beta_{ij}(t,T)x_i(t)x_j(t). \]
Then we obtain the inequalities
\[ V_T^{(2)}(t,x(t))\leq V^{(1)}(t,x(t),T)\leq I_T(t,x(t),u^0(\beta,x,t,T)), \tag{5.2} \]
where \(u^0(\beta,x,t,T)\) denotes the control minimizing \(F_T(t,x(t),u)\). From the stabilizability (5.1) it follows that, as \(T\to\infty\),
\[ I_T(t,x(t),u^0(\beta,x,t,T))\to v^2(t,x(t)). \]
Passing to the limit in inequalities (5.2), we obtain that
\[ V^{(1)}(t,x(t),T)\to V^{(2)}(t,x(t)) \]
as \(T\to\infty\), for any initial vector \(x(t)\). Hence it follows that
\[ \lim_{T\to\infty}\alpha_{ij}(t,T)=\beta_{ij}(t)\quad (i,j=1,\ldots,n). \tag{5.3} \]

It should be noted, however, that the convergence in the last expression is not uniform.

We shall show that
\[ \frac{1}{T}V(0,\eta(0,0),T) \]
tends, as \(T\to\infty\), to a finite limit. For this it is sufficient to show that
\[ \lim_{T\to\infty}\varphi(0,0,\lambda,T) \]
exists and is finite, since, according to (5.3),
\[ \lim_{T\to\infty}\frac{1}{T}V^{(1)}(0,x(0),T)=0. \]

Consider the quantity \(M[\sigma_{ij}(s,\sigma)\mid t,\tau=0]\). Taking into account that the elements \(f_{ij}(t,\tau)\) of the fundamental matrix of the system \(\dot x=Ax\) can be estimated by the inequality
\[ |f_{ij}(t,\tau)|<e^{nc(t-\tau)}\quad (i,j=1,\ldots,n) \]
(\(c=\max a_{ij}(t)\) over all \(t\geq0\) and all \(i,j\)), we obtain that, for \(\lambda^*=nc+a\), \(a>0\), the quantities
\[ M[\sigma_{ij}(s,\sigma)\mid t,\tau=0] =\int_0^s \sigma_{ij}(s,\sigma)\lambda^* e^{-\lambda^*(s-\sigma)}\,d\sigma +e^{-\lambda^*s}\sigma_{ij}(s,0)=\varepsilon_{ij}(s) \]
are bounded for all \(s\geq0\) and form a positive definite matrix \(\|\varepsilon_{ij}(s)\|\) (this follows from the positivity of the matrices of second moments of \(n\)-dimensional random variables).

From the methods for solving optimal problems of the type under consideration (see, for example, [13]) it is known that for any \(\varepsilon > 0\) one can indicate such \(T_0, K > 0\) that, for \(T > T_0\), the relation \(|\alpha_{ij}(t,T)-\beta_{ij}(t)| < \varepsilon\) holds for all \(t\) in \([0,T-K]\). Denote the integrand in (4.17) for \(t=0\) by the symbol \(\omega(s)\). Also denote

\[ \sum_{i,j=1}^{n}\sum_{k,l=1}^{n}\beta_{ki}(s)\beta_{jl}(s)b_k(s)b_l(s)\varepsilon_{ij}(s) + \sum_{i,j=1}^{n}\beta_{ij}(s)\bigl(r_i^{(1)}(s)r_j^{(1)}(s) + \]

\[ +(\lambda^*)^2 r_i^{(2)}(s)r_j^{(2)}(s)\bigr)=\gamma(s). \]

Then, taking into account everything said above, we obtain for all \(T>T_0\)

\[ \frac{1}{T}\left( \int_{0}^{T-K}\gamma(s)\,ds + \int_{T-K}^{T}\omega(s)\,ds - \varepsilon\int_{0}^{T-K}\bigl[ \|r^{(1)}(s)\|^2+(\lambda^*)^2\|r^{(2)}(s)\|^2+ \right. \]

\[ \left. +\|b(s)\|^2\bigr]\,ds \right) \le \frac{1}{T}\varphi(0,0,\lambda^*,T) \le \frac{1}{T}\left( \int_{0}^{T-K}\gamma(s)\,ds + \int_{T-K}^{T}\omega(s)\,ds + \right. \]

\[ \left. +\varepsilon\int_{0}^{T-K}\bigl[ \|r^{(1)}(s)\|^2+(\lambda^*)^2\|r^{(2)}(s)\|^2+\|b(s)\|^2\bigr]\,ds \right). \]

The quantity

\[ \frac{1}{T}\int_{0}^{T-K}\gamma(s)\,ds \]

is bounded above (this follows from the properties of the functions \(r^{(1)}\), \(r^{(2)}\), \(\varepsilon_{ij}\) and from the uniform controllability of system (5.1)) and is monotone for sufficiently large \(T\) (this can be verified by differentiating it with respect to \(T\)). Consequently, as \(T\to\infty\) it tends to a finite limit \(L<\infty\). Taking into account the arbitrariness of \(\varepsilon>0\) and the boundedness of \(\omega(s)\), we obtain that

\[ \lim_{T\to\infty}\varphi(0,0,\lambda^*,T)=L<\infty, \tag{5.4} \]

if \(\lambda^*>cp\).

Relations (5.3), (5.4) show that the solution of problem 5.1 is obtained by a limiting transition from the solutions of problem 2.1 at a fixed frequency \(\lambda^*\), if the latter is sufficiently large. In this case there is (generally speaking, nonuniform) convergence of both the controls and the functionals, i.e.,

\[ u_\infty(t,\eta(t,\tau),\lambda^*)= \lim_{T\to\infty}u(t,\eta(t,\tau),T,\lambda^*), \]

\[ V_\infty(x_0,\lambda^*)= \lim_{T\to\infty}\frac{1}{T}V(t,\eta(t,\tau),\lambda^*T), \]

where \(u_\infty, V_\infty\) are the optimal values of the control and of the functional of problem 5.1 at the fixed frequency \(\lambda^*\). The optimal value \(\lambda_\infty\) of problem 5.1 is determined from the condition that \(V_\infty(x_0,\lambda_\infty)=\min V(x_0,\lambda^*)\) over all \(\lambda^*>pc\), while the optimal control has the form

\[ u_\infty=u_\infty(t,\eta(t,\tau),\lambda_\infty). \]

References

1. Krasovskii N. N. Avtomatika i telemekhanika, vol. XXIV, No. 8, 1963.
2. Krasovskii N. N., Lidskii E. A. Avtomatika i telemekhanika, vol. XXII, No. 9–11, 1961.
3. Doob J. L. Stochastic Processes. IL, 1956.
4. Gnedenko B. V. Course in Probability Theory. Fizmatgiz, 1961.
5. Stratonovich R. L. Avtomatika i telemekhanika, vol. XXIII, No. 7, 1962.
6. Feldbaum A. A. Foundations of the Theory of Optimal Automatic Systems. Fizmatgiz, 1963.
7. Letov A. M. Avtomatika i telemekhanika, vol. XXI, No. 4, 1960; vol. XXII, No. 4, 1961.
8. Bellman R., Glicksberg I., Gross O. Some Questions in the Mathematical Theory of Control Processes. IL, 1962.
9. Krasovskii N. N. PMM, vol. XXVI, No. 2, 1962.
10. Dynkin E. B. Markov Processes. Fizmatgiz, 1963.
11. Galperin E. A., Krasovskii N. N. PMM, vol. XXVII, No. 4, 1963.
12. Kalman R. E., Ho Y. C., Narendra K. S. Controllability of linear dynamic systems. Contributions to Diff. Equations. Intersc. Publ., 1, No. 2, 1963.
13. Kalman R. E. Contributions to the theory of optimal control. Bol. Soc. Mat. Mex. Mexico City, 1961.

Received by the editors
December 21, 1964

Ural State University
named after A. M. Gorky