UDC 517.917

ON THE QUESTION OF AN ASYMPTOTIC SOLUTION OF AN OPTIMAL CONTROL PROBLEM

N. Kh. Bagirova, A. B. Vasil’eva, M. I. Imanaliev

In works [1, 2] certain variational problems were considered in which the functional contains a small parameter \(\mu\)

\[ J[y]=\int_a^b F(t,y,\mu y')\,dt, \]

and for \(\mu=0\) the integrand does not depend on \(y'\), and the variational problem degenerates. An asymptotic representation with respect to the parameter \(\mu\) was obtained for the extremals of such a functional and for the extremal value itself. This asymptotic form holds under the fulfillment of a certain condition, called the stability condition.

In the present note it will be shown that the same method is applicable to obtaining the asymptotics of the classical solution of the following optimal control problem (\(u\) is the control function; \(\mu>0\) is a small parameter):

\[ \mu \frac{dy}{dt}=f(t,y,u), \tag{1} \]

\[ J[y,u]=\int_a^b F(t,y,u)\,dt=\min, \tag{2} \]

\[ y(a)=y^0,\quad y(b)=y^1. \tag{3} \]

As was noted in [3], this problem is equivalent to the problem of a conditional extremum of a functional in the presence of differential constraints. By means of the method of undetermined Lagrange multipliers, the problem of a conditional extremum reduces to finding the unconditional extremum of the functional

\[ G(y,u,\lambda)=\int_a^b \{F(t,y,u)+\lambda(t)[\mu y'-f(t,y,u)]\}\,dt= \]

\[ =\int_a^b \Phi(t,y,\mu y',u,\lambda)\,dt. \tag{4} \]

The system of Euler equations together with the constraint condition (1) has the form

\[ \mu \frac{d\lambda}{dt}=F_y(t,y,u)-\lambda f_y(t,y,u)=P(t,y,u,\lambda), \]

\[ \mu \frac{dy}{dt}=f(t,y,u)=Q(t,y,u,\lambda), \tag{5} \]

\[ 0=F_u(t,y,u)-\lambda f_u(t,y,u)\equiv R(t,y,u,\lambda) \]

and is solved under the additional conditions (3).

Let us note that these same equations are obtained from Pontryagin’s maximum principle. Indeed, using the notation of [3], we write (1), (2) in the form

\[ \frac{dx_i}{dt}=f_i(x_1,x_2,u), \tag{1′} \]

\[ J=\int_a^b f_0(x_1,x_2,u)\,dt \tag{2′} \]

\[ \left( i=1,2;\quad x_1=y,\quad x_2=t,\quad f_1=\frac{f(t,y,u)}{\mu},\quad f_2=1,\quad f_0=F(t,y,u) \right). \]

We have

\[ \Pi(\psi,x,u)=\psi_0 f_0+\psi_1 f_1+\psi_2 f_2, \tag{6} \]

\[ \frac{dx_i}{dt}=\frac{\partial \Pi}{\partial \psi_i}=f_i(x_1,x_2,u), \tag{7} \]

\[ \frac{d\psi_i}{dt}=-\frac{\partial \Pi}{\partial x_i} =-\left\{\psi_0\frac{\partial f_0}{\partial x_i} +\psi_2\frac{\partial f_2}{\partial x_i}\right\}. \tag{8} \]

Obviously, (7) for \(i=1,2\) coincides with (1′), while (8) for \(i=0,1\) (the equation corresponding to \(i=2\) is separated off) has the form

\[ \frac{d\psi_0}{dt}=0, \tag{8_0} \]

\[ \frac{d\psi_1}{dt} =-\left\{\frac{\partial F}{\partial y}\psi_0 +\frac{\partial f}{\partial y}\frac{1}{\mu}\psi_2\right\}. \tag{8_1} \]

The condition of maximum of \(\Pi\) with respect to \(u\) gives

\[ \frac{\partial \Pi}{\partial u} =\psi_0\frac{\partial f_0}{\partial u} +\psi_1\frac{\partial f_1}{\partial u} =\psi_0\frac{\partial F}{\partial u} +\psi_1\frac{1}{\mu}\frac{\partial f}{\partial u}=0. \tag{9} \]

Together with (8\(_1\)), if we take into account that, according to (8\(_0\)), \(\psi_0=\mathrm{const}\), this leads to the relations

\[ \frac{d}{dt}\left(\frac{\psi_2}{\psi_0}\right) =-\frac{\partial F}{\partial y} -\frac{\partial f}{\partial y}\left(\frac{1}{\mu}\frac{\psi_1}{\psi_0}\right), \]

\[ 0=\frac{\partial F}{\partial u} +\frac{\partial f}{\partial u}\left(\frac{1}{\mu}\frac{\psi_1}{\psi_0}\right). \tag{10} \]

Denoting \(\dfrac{1}{\mu}\dfrac{\psi_1}{\psi_0}\) by \(-\lambda\), we obtain, together with (7) (\(i=1,2\)), the system of equations (5).

System (5) differs from that considered in [4] only by the presence of a third (non-differential) equation. Suppose that this latter equation is uniquely solvable with respect to \(u\) in some domain \(D\) of variation of \(t,y,\lambda\), so that \(u=U(t,y,\lambda)\), and, in addition, suppose that

\[ R_u\big|_{u=U}\ne 0. \]

In terms of \(F\) and \(f\), this condition has the form

\[ F_{uu}+\frac{f_u}{F_u}f_{uu}\ne 0. \]

Let us note that if \(R_u\equiv 0\), i.e., the equation \(R(t,y,u,\lambda)=0\) does not co-

contains \(u\), then system (5) in fact represents a single first-order differential equation, whose solution, generally speaking, cannot satisfy both conditions (3).

Let us rewrite (5) in the form

\[ \begin{gathered} \mu \frac{d\lambda}{dt}=\varphi(t,y,\lambda),\\[4pt] \mu \frac{dy}{dt}=\psi(t,y,\lambda) \end{gathered} \tag{11} \]

\[ \bigl(\varphi(t,y,\lambda)=P(t,y,U(t,y,\lambda),\lambda),\quad \psi(t,y,\lambda)= \]

\[ =Q(t,y,U(t,y,\lambda),\lambda)\bigr). \]

To this system one can apply the results of [4], concerning the construction of the asymptotics of the solution of this system under the boundary conditions (3). According to [4], in order that, as \(\mu\to0\), there exist a finite limit of the solution of problem (11)—(3), the so-called stability conditions must be satisfied; they are as follows.

Suppose that the system of equations \(\varphi(t,y,\lambda)=0,\ \psi(t,y,\lambda)=0\), obtained from (11) for \(\mu=0\), is solvable with respect to \(y\) and \(\lambda\). Consider one of such possible isolated solutions and denote it by \(y=\overline{y}_0(t)\), \(\lambda=\overline{\lambda}_0(t)\). The stability condition consists in requiring that the roots \(\Lambda\) of the characteristic equation

\[ \left| \begin{array}{cc} \overline{\varphi}_{\lambda}-\Lambda & \overline{\varphi}_{y}\\ \overline{\psi}_{\lambda} & \overline{\psi}_{y}-\Lambda \end{array} \right|=0, \tag{12} \]

where \(\overline{\varphi}_{\lambda}=\varphi_{\lambda}(t,\overline{y}_0,\overline{\lambda}_0)\) (and similarly \(\overline{\varphi}_{y},\overline{\psi}_{\lambda},\overline{\psi}_{y}\)), be real and of different signs everywhere on \([a,b]\). In a certain neighborhood of the solution \(y=\overline{y}_0(t)\), \(\lambda=\overline{\lambda}_0(t)\), satisfying condition (12), in [4] the existence of a unique solution of the boundary-value problem (3) is proved and its asymptotics with respect to the parameter \(\mu\) are constructed.

We note that, by virtue of the nonlinearity of system (5) both as functions \(U(t,y,\lambda)\) and as stable solutions \(y_0(t),\overline{\lambda}_0(t)\), there may be several of them. Therefore, in general, there is no uniqueness in the large for the solution of problem (3).

The stability condition can obviously be written in the form of the requirement

\[ \overline{\varphi}_{\lambda}\overline{\psi}_{y} -\overline{\varphi}_{y}\overline{\psi}_{\lambda}<0,\qquad a\leq t\leq b. \tag{13} \]

Returning to system (5), we obtain this same inequality in the form

\[ \overline{R}_{u}(\overline{P}_{\lambda}\overline{Q}_{y}-\overline{Q}_{\lambda}\overline{P}_{y}) +\overline{R}_{y}(\overline{P}_{u}\overline{Q}_{\lambda}-\overline{Q}_{u}\overline{P}_{\lambda}) +\overline{R}_{\lambda}(\overline{Q}_{u}\overline{P}_{y}-\overline{P}_{u}\overline{Q}_{y})<0, \tag{14} \]

where the bar means that all functions are taken along the solution
\(y=\overline{y}_0(t)\), \(u=\overline{u}_0(t)=u(t,\overline{y}_0(t),\overline{\lambda}_0(t))\), \(\lambda=\overline{\lambda}_0(t)\) of the system obtained from (5) for \(\mu=0\) (the so-called degenerate system):

\[ P(t,\overline{y}_0,\overline{u}_0,\overline{\lambda}_0)=0, \]

\[ Q(t,\overline{y}_0,\overline{u}_0,\overline{\lambda}_0)=0, \tag{15} \]

\[ R(t,\overline{y}_0,\overline{u}_0,\overline{\lambda}_0)=0. \]

According to [4], one can construct a formal solution of system (11), whose principal term is \(y_0(t),\overline{\lambda}_0(t)\). This solution is an asymptotic expansion of the solution of the boundary-value problem (1)—(3).

We shall formulate here an algorithm for constructing such an expansion, applicable directly to (5). We shall seek the solution of system (5) in the form of a formal expansion in powers of \(\mu\) (by \(x\) we shall understand \(y, u, \lambda\) in aggregate)

\[ x=\bar{x}_0+\mu \bar{x}_1+\ldots+\mu^p \bar{x}_p+\ldots \tag{16} \]

Substituting into (5) and equating the coefficients of like powers of \(\mu\), we obtain a sequence of algebraic systems from which, by virtue of the condition \(R_u\ne 0\), we determine \(\bar{x}_0,\bar{x}_1,\ldots,\bar{x}_p,\ldots\). In particular, \(\bar{x}_0\) satisfies the degenerate system (15), and for \(\bar{x}_0\) we take precisely that solution \(\bar{y}_0(t), \bar{u}_0(t), \bar{\lambda}_0(t)\) satisfying the stability condition (14), which was discussed above.

Next we rewrite (5) in the form (making the change of variables \(\tau_0=\dfrac{t-a}{\mu}\))

\[ \frac{d\lambda}{d\tau_0}=P(a+\tau_0\mu,y,u,\lambda), \]

\[ \frac{du}{d\tau_0}=Q(a+\tau_0\mu,y,u,\lambda), \tag{17} \]

\[ 0=R(a+\tau_0\mu,y,u,\lambda). \]

We shall construct the solution of this system in the form of a formal expansion in powers of \(\mu\)

\[ x=x_0^0+\mu x_1^0+\ldots+\mu^p x_p^0+\ldots \tag{18} \]

Substituting (18) into (17) and equating the coefficients of like powers of \(\mu\), we obtain a sequence of systems of differential equations for determining \(x_0^0,x_1^0,\ldots,x_p^0,\ldots\). In order to determine from these systems uniquely \(x_0^0,x_1^0,\ldots,x_p^0,\ldots\), additional conditions are prescribed:

\[ y_0^0\big|_{\tau_0=0}=y^0,\qquad y_i^0\big|_{\tau_0=0}=0\quad (i>0), \tag{19} \]

\[ (\lambda_p^0-\bar{\lambda}_p^0)_{\tau_0\to\infty}\to 0\quad (p=0,1,\ldots). \]

Here the notation is adopted

\[ \bar{x}_p^0=\sum_{k=0}^{p}\tau_0^k \bar{x}_{k,p-k}^0, \tag{20} \]

where \(\bar{x}_{k,p-k}^0\) is the \(k\)-th coefficient in the expansion of \(\bar{x}_{p-k}\) in powers of \(t-a\). In (19) only \(\bar{\lambda}_p^0\) is used, but for the subsequent construction all components \(\bar{x}_p^0\) will be needed.

Analogous constructions are also carried out in a neighborhood of \(t=b\) (at the right endpoint). Instead of (17), one writes a system with \(a\) replaced by \(b\) and \(\tau_0\) by \(\tau_1=\dfrac{t-b}{\mu}\). Instead of (18), one takes the expansion

\[ x=x_0^1+\mu x_1^1+\ldots+\mu^p x_p^1+\ldots, \tag{21} \]

which differs from (18) by the upper index: index \(0\) denotes belonging to the left endpoint, and index \(1\) belonging to the right endpoint. Instead of (19), the additional conditions are prescribed:

\[ y_0^1\big|_{\tau_1=0}=y^1,\qquad y_i^1\big|_{\tau_1=0}=0\quad (i>0), \]

\[ (\lambda_p^1-\bar{\lambda}_p^1)_{\tau_1\to-\infty}\to 0\quad (p=0,1,\ldots), \tag{22} \]

where

\[ \bar{x}_p^{\,1}=\sum_{k=0}^{p}\tau_1^k \bar{x}_{k,p-k}^{\,1}, \tag{23} \]

and \(\bar{x}_{k,p-k}^{\,1}\) is the \(k\)-th coefficient in the expansion of \(\bar{x}_{p-k}^{\,1}\) in powers of \(t-b\).

Introduce the notation

\[ \Pi_k^l(x)=x_k^l-\bar{x}_k^l \quad (l=0,\,1) \tag{24} \]

and form a sum of the form

\[ X_n=\sum_{k=0}^{n}\mu^k\bigl[\bar{x}_k+\Pi_k^0(x)+\Pi_k^1(x)\bigr]. \tag{25} \]

According to [4], if, in some neighborhood of the solution \(\bar{x}_0\) of the degenerate system (15),

\[ B=\{|x-\bar{x}_0|\leq \varepsilon,\ a\leq t\leq b\} \]

(\(\varepsilon\) being a sufficiently small constant independent of \(\mu\)), the right-hand sides of (5) are continuously differentiable and, moreover, the stability condition (14) is satisfied and \(|y^0-\bar{y}_0(a)|<\varepsilon,\ |y^1-\bar{y}_0(b)|<\varepsilon\), then, for sufficiently small \(\mu\leq \mu^0\), there exists in \(B\) a unique solution of system (5) satisfying the conditions (3).

If it is additionally assumed that in \(B\) there exist continuous partial derivatives of the right-hand sides of (5) up to order \(n+2\) inclusive, then expression (25) will serve for this solution as an asymptotic formula with a remainder term of order not less than \(O(\mu^{n+1})\), i.e.

\[ |X_n-x|<c\mu^{n+1}, \tag{26} \]

where \(c\) is independent of \(t\) and \(\mu\) for \(a\leq t\leq b,\ \mu\leq \mu^0\).

The methods [1, 2, 4] make it possible to give an asymptotic representation not only for the solution of (5), (3), but also for the extremal value itself of the functional (4). Indeed, an expression analogous to (25) can be written not only for \(x\), but also for any sufficiently often differentiable function of \(x\) (the right-hand side of (25) may be regarded as an operator defined on the set of such functions), for example, for \(z=\xi(x,t)\): \(z_p\) is equal to the \(p\)-th term in the expansion of \(\xi(x_0^0+\mu x_1^0+\ldots,t)\) in powers of \(\mu\), \(z_p^0\) is the \(p\)-th term in the expansion of \(\xi(x_0^0+\mu x_1^0+\ldots,a+\tau_0\mu)\) in powers of \(\mu\), and so on.

Denote

\[ G_p=\int_a^b \Phi_p\,dt+\int_0^\infty \Pi_{p-1}^0(\Phi)\,d\tau_0+\int_0^{-\infty}\Pi_{p-1}^1(\Phi)\,d\tau_1 \tag{27} \]

(for example,

\[ G_0=\int_a^b \Phi(t,\bar{y}_0,\bar{u}_0,\bar{\lambda}_0)\,dt \]
). There hold inequalities analogous to (26):

\[ \left|G_{\mathrm{extr}}-\sum_{k=0}^{n}\mu^k G_k\right|<c\mu^{n+1}. \tag{28} \]

All that has been said can be summarized in the form of the following theorem.

Theorem. Let the control \(u\) be such that the integral curve \(y\) of equation (1) corresponding to it satisfies the conditions (3) and realizes the minimum of the functional (2). Suppose that, in some neighborhood of the solution \(\bar{x}_0\) of the degenerate system (15), \(B=\{|x-\bar{x}_0|<\varepsilon,\ a\leq t\leq b\}\), the right-hand sides of (5) are differentiable \(n+2\) times, the stability condition (14) is fulfilled, and \(|y^0-y_0(a)|<\varepsilon,\ |y^1-y_0(b)|<\varepsilon\). Then \(u\) and \(y\) are asymptotically represent-

are given by formula (25) with the estimate of the remainder term (26) (where by \(x\) one understands \(u\) and \(y\) together), while for the minimum value itself of the functional (2) the estimate (28) is valid.

Formula (25) shows that, both for \(y\) and for \(u\), near the endpoints \(a\) and \(b\) a boundary-layer phenomenon occurs.

In conclusion, let us consider an example. Let (1), (2), (3) have the form

\[ \mu y'=-y+u+1, \]

\[ J=\int_0^1 (y^2+u^2)\,dt=\min, \]

\[ y(0)=y^0,\quad y(1)=y^1. \]

Let us write the functional \(G\) (in order that the following expressions have a more symmetric form, here we have taken not \(\lambda\), but \(2\lambda\)):

\[ G=\int_0^1 \{(y^2+u^2)+2\lambda(t)(\mu y'+y-u-1)\}\,dt \]

and the system (5):

\[ \mu\lambda'=y+\lambda, \]

\[ \mu y'=-y+u+1, \]

\[ 0=u-\lambda, \]

the exact solution of this system under the imposed boundary conditions is the expression

\[ \lambda=u= -\frac{ \left(y^0-\dfrac12\right)-\left(y^1-\dfrac12\right)e^{-\frac{\sqrt2}{\mu}} }{ (\sqrt2+1)\left(1-e^{-\frac{2\sqrt2}{\mu}}\right) } e^{-\frac{\sqrt2}{\mu}t} + \]

\[ +\frac{ \left(y^1-\dfrac12\right)+\left(y^0-\dfrac12\right)e^{-\frac{\sqrt2}{\mu}} }{ (\sqrt2-1)\left(1-e^{-\frac{2\sqrt2}{\mu}}\right) } e^{-\frac{\sqrt2(1-t)}{\mu}} -\frac12, \]

\[ y= \frac{ \left(y^0-\dfrac12\right)-\left(y^1-\dfrac12\right)e^{-\frac{\sqrt2}{\mu}} }{ 1-e^{-\frac{2\sqrt2}{\mu}} } e^{-\frac{\sqrt2}{\mu}t} + \]

\[ +\frac{ \left(y^1-\dfrac12\right)-\left(y^0-\dfrac12\right)e^{-\frac{\sqrt2}{\mu}} }{ 1-e^{-\frac{2\sqrt2}{\mu}} } e^{-\frac{\sqrt2(1-t)}{\mu}} +\frac12. \tag{29} \]

Let us find the zeroth approximation to this solution according to the rule indicated above. The degenerate system (15) has the form

\[ 0=\bar y_0+\bar\lambda_0, \]

\[ 0=-\overline{y}_0+\overline{u}_0+1, \]

\[ 0=\overline{u}_0-\overline{\lambda}_0. \]

Hence

\[ \overline{u}_0=\overline{\lambda}_0=-\frac{1}{2},\qquad \overline{y}_0=\frac{1}{2}. \]

The stability condition (14) is satisfied here, since the left-hand side of (14) is equal to \(-2\).

Let us now determine \(x_0^0\) and \(x_0^1\) from the expansions (18) and (21). The zero approximation to the system (17) determining \(x_0^0\) will be the system (in our simplest example it coincides with the original system)

\[ \frac{d\lambda_0^0}{d\tau_0}=y_0^0+\lambda_0^0, \]

\[ \frac{dy_0^0}{d\tau_0}=-y_0^0+u_0^0+1, \]

\[ 0=u_0^0-\lambda_0^0. \]

As the boundary conditions (19) we take

\[ y_0^0(0)=y^0,\qquad \lambda_0^0(\infty)=\overline{\lambda}_0^0=-\frac{1}{2}. \]

The solution of this problem is

\[ \lambda_0^0=u_0^0=-\frac{\left(y^0-\frac{1}{2}\right)}{\sqrt{2}+1}\,e^{-\sqrt{2}\tau_0}-\frac{1}{2}, \]

\[ y_0^0=\left(y^0-\frac{1}{2}\right)e^{-\sqrt{2}\tau_0}+\frac{1}{2}, \]

\(x_0^1\) is determined by the analogous system

\[ \frac{d\lambda_0^1}{d\tau_1}=y_0^1+\lambda_0^1, \]

\[ \frac{dy_0^1}{d\tau_1}=-y_0^1+u_0^1+1, \]

\[ 0=u_0^1-\lambda_0^1 \]

and the boundary conditions

\[ y_0^1(0)=y^1,\qquad \lambda_0^1(-\infty)=\overline{\lambda}_0^0=-\frac{1}{2}. \]

and has the form

\[ \lambda_0^1=u_0^1=\frac{\left(y^1-\frac{1}{2}\right)}{\sqrt{2}-1}\,e^{\sqrt{2}\tau_1}-\frac{1}{2}, \]

\[ y_0^1=\left(y^1-\frac{1}{2}\right)e^{\sqrt{2}\tau_1}+\frac{1}{2}. \]

Formula (25) for \(n=0\) gives a uniform zeroth approximation

\[ X_0=\bar{x}_0+\left(x_0^0-\bar{x}_0^0\right)+\left(x_0^1-\bar{x}_0^1\right). \]

Substituting here the expressions already obtained, we shall have

\[ \Lambda_0=U_0= \frac{y^0-\dfrac{1}{2}}{\sqrt{2}+1}\, e^{-\frac{\sqrt{2}\,t}{\mu}} + \frac{y^1-\dfrac{1}{2}}{\sqrt{2}-1}\, e^{-\frac{\sqrt{2}\,(1-t)}{\mu}} -\frac{1}{2}, \]

\[ Y_0= \left(y^0-\frac{1}{2}\right)e^{-\frac{\sqrt{2}\,t}{\mu}} + \left(y^1-\frac{1}{2}\right)e^{-\frac{\sqrt{2}\,(1-t)}{\mu}} +\frac{1}{2}. \]

A direct comparison with the exact solution (29) shows that

\[ |\lambda-\Lambda_0|,\ |y-Y_0|,\ |u-U_0|<c\mu . \]

References

1. Bagirova N. Kh. Vestnik MGU, ser. 1, No. 1, 33—41, 1966.
2. Bagirova N. Kh. Uch. zap. AzGU, ser. phys.-mat. and chem. sciences, No. 1, 19—30, 1965.
3. Gelfand I. M., Fomin S. V. Calculus of Variations. Moscow, Fizmatgiz, 1961 (Supplement II).
4. Tupchiev V. A. DAN SSSR, 143, No. 16, 1962.

Received by the editors
18 July 1966.

Azerbaijan State University,
Moscow State University,
Sector of Computational Mathematics
of the Academy of Sciences of the Kirgiz SSR