M. G. KREIN

ON A NEW APPLICATION OF THE FIXED-POINT PRINCIPLE IN THE THEORY OF OPERATORS IN A SPACE WITH AN INDEFINITE METRIC

(Presented by Academician L. S. Pontryagin on 21 X 1963)

1. Let $\mathfrak H$ be a Hilbert space in which, along with the usual scalar product $(x,y)$, an indefinite scalar product is given:
\[ [x,y]=(Jx,y),\qquad J=P_+-P_-, \]
where $P_\pm$ are two mutually complementary orthogonal projectors. Put
\[ \mathfrak P_+=\{x:\ x\in\mathfrak H,\ [x,x]\ge 0\}. \]
A subspace $L\ (\subset \mathfrak H)$ is called $J$-nonnegative if $L\subset \mathfrak P_+$. Denote by $\mathfrak M_+$ the set of all maximal $J$-nonnegative subspaces. Denote by $\mathfrak K_+$ the set of all linear operators $K$ defined on $\mathfrak H_+=P_+\mathfrak H$ and mapping $\mathfrak H_+$ into $\mathfrak H_-=P_-\mathfrak H$ in a nonexpanding manner: $\|K\|\le 1$.

Analogously one defines $J$-nonpositive subspaces, the sets $\mathfrak P_-$, $\mathfrak M_-$, and $\mathfrak K_-$.

In (¹, ²) (see also (³)) it is shown that the sets $\mathfrak M_+$ and $\mathfrak K_+$ are in one-to-one correspondence $L\leftrightarrow K$, by virtue of which
\[ L=\{x_+ + Kx_+:\ x_+\in\mathfrak H_+\} \]
(the operator $K\in\mathfrak K_+$ is called the angular operator of the subspace $L\in\mathfrak M_+$). Of course, the sets $\mathfrak M_-$ and $\mathfrak K_-$ are in an analogous correspondence.

Denote by $\mathfrak R$ the ring of all linear bounded operators $A$ acting in $\mathfrak H$, and by $\mathfrak S_\infty$ its ideal consisting of all completely continuous $A\in\mathfrak R$.

Theorem 1. Suppose that, for some $A\in\mathfrak R$, the following conditions are fulfilled: 1) $0\ne Ax\in\mathfrak P_+$ for $0\ne x\in\mathfrak P_+$; 2) $P_+AP_-\in\mathfrak S_\infty$; and 3) there exists at least one $L_0\in\mathfrak M_+$ such that $AL_0\in\mathfrak M_+$.

Then the operator $A$ maps every $L\in\mathfrak M_+$ onto $AL\in\mathfrak M_+$, and there exists at least one $L_a\in\mathfrak M_+$ such that $AL_a=L_a$.

Proof. Let $L\in\mathfrak M_+$ and let $K$ be its angular operator. Then for
\[ x=x_+ + Kx_+\in L\qquad (x_+\in\mathfrak H_+) \]
we shall have
\[ P_+Ax=(A_{11}+A_{12}K)x_+,\qquad P_-Ax=(A_{21}+A_{22}K)x_+, \tag{1} \]
where $A_{jk}$ is the operator acting from $\mathfrak H_k$ into $\mathfrak H_j$ according to the formula
\[ A_{jk}x=P_jAP_kx=P_jAx,\quad x\in\mathfrak H_k\quad (j,k=1,2), \]
with $P_1=P_+$, $P_2=P_-$, $\mathfrak H_1=\mathfrak H_+$, $\mathfrak H_2=\mathfrak H_-$.

Since from $x\in L$ it follows that $Ax\in\mathfrak P_+$, i.e.
\[ |P_+Ax|\ge |P_-Ax|\qquad (|x|=(x,x)^{1/2}), \]
it follows from $x\in L,\ P_+Ax=0$ that $Ax=0$, $x=0$. Therefore, according to (1),
\[ (A_{11}+A_{12}K)x_+\ne 0 \]
for $x_+\ne 0$, $x_+\in\mathfrak H_+$, whatever the operator $K\in\mathfrak K_+$ may be. In particular, the operator
\[ W_0=A_{11}+A_{12}K_0, \]
acting in $\mathfrak H_+$, where $K_0$ is the angular operator for $L_0$, vanishes only at zero. On the other hand, by the condition
\[ L'_0=AL_0\in\mathfrak M_+, \]
and therefore
\[ P_+L'_0=\mathfrak H_+, \]
i.e.
\[ \mathfrak H_+=\{P_+Ax:\ x\in L_0\} =\{(A_{11}+A_{12}K_0)x_+:\ x_+\in\mathfrak H_+\}. \]
Consequently, the operator
\[ W_0=A_{11}+A_{12}K_0 \]
maps $\mathfrak H_+$ one-to-one and continuously onto itself and hence, by Banach’s theorem, has a continuous inverse.

Comparing the operator
\[ W=A_{11}+A_{12}K \]
for an arbitrary $K\in\mathfrak K_+$ with the operator $W_0$, we observe that their difference
\[ W-W_0=A_{12}(K-K_0)\in\mathfrak S_\infty \]
(by the condition $A_{12}=P_+AP_-\in\mathfrak S_\infty$), and consequently
\[ W=W_0(I+T), \]
where
\[ T=W_0^{-1}(W-W_0)\in\mathfrak S_\infty. \]
Since $W$ vanishes only at zero, we conclude:

a) For any \(K \in \mathfrak{K}_+\), the operator \(W(K)=A_{11}+A_{12}K\) maps \(\mathfrak{H}_+\) onto itself one-to-one and continuously.

Thus, for any \(K \in \mathfrak{K}_+\), the fractional-linear transformation

\[ K'=(A_{21}+A_{22}K)(A_{11}+A_{12}K)^{-1}\;(=\Phi(K)). \tag{2} \]

is meaningful.

According to (1), for \(L(\in \mathfrak{M}_+)\) with angular operator \(K\) we have:
\(Ax=P_+Ax+K'P_+Ax\;(x\in L)\). Since, as was proved, \(P_+Ax\) runs through all of \(\mathfrak{H}_+\) when \(x\) runs through \(L\), for \(L'=AL(\subset \mathfrak{P}_+)\) we obtain:
\(L'=\{x_+ + K'x_+:\,x_+\in\mathfrak{H}_+\}\).
Hence the subspace \(L'\in\mathfrak{M}_+\), and \(K'\) is its angular operator. The first assertion is proved.

At the same time it has been shown that \(\Phi(K)\) maps \(\mathfrak{K}_+\) into itself. As is known (4), \(\mathfrak{K}_+\) is a convex bicompact in the weak topology. If we show that the mapping \(\Phi(K)\) is continuous in this topology, then, by the Schauder–Tikhonov theorem, \(\Phi(K)\) will have a fixed point, and thereby the second assertion of the theorem will be proved.

From conclusion a) it follows that the operator \(A_{11}=W(0)\) is continuously invertible. Therefore \(W(K)=A_{11}(I-SK)\), where \(S=-A_{11}^{-1}A_{12}\) is an operator mapping \(\mathfrak{H}_-\) completely continuously into \(\mathfrak{H}_+\). Put \(s=\|S\|\), and let \(\psi(\in\mathfrak{H}_-)\) be a unit vector (\(\|\psi\|=1\)) for which \(S\psi(\in\mathfrak{H}_+)\) has norm equal to \(s\). Then \(S\psi=s\varphi\), \(\|\varphi\|=1\). Obviously, for any \(q\;(0\le q\le 1)\) the operator \(K_q=q(\cdot,\varphi)\psi\in\mathfrak{K}_+\), and for it \((I-SK_q)\varphi=(1-qs)\varphi\). Since the number \(1-qs\;(0\le q\le 1)\) must be nonzero, \(s=\|S\|<1\). The function \(\Phi(K)\;(K\in\mathfrak{K}_+)\) can be written in the form

\[ \Phi(K)=(A_{21}+A_{22}K)(I-SK)^{-1}A_{11}^{-1}. \]

Since, as was proved, \(\|S\|<1\), on \(\mathfrak{K}_+\) the function \(\Phi(K)\) is the uniform limit, in the uniform norm, of the sequence of functions

\[ \Phi_n(K)=(A_{21}+A_{22}K)\sum_{p=0}^{n}(SK)^p A_{11}^{-1}. \]

Taking into account that the operator \(S\), being completely continuous, can be approximated with arbitrary accuracy in the uniform norm by a finite-dimensional operator, it is not difficult to show that each function \(\Phi_n(K)\) is continuous on the bicompact \(\mathfrak{K}_+\) in the weak topology. Consequently, the function \(\Phi(K)\) has the same property, and the theorem is completely proved.

Theorem 2. Let, for the operator \(A(\in\mathfrak{R})\), the following conditions be satisfied:
1) the operator is continuously invertible: \(A^{-1}\in\mathfrak{R}\); 2) \(A\mathfrak{P}_+\subset\mathfrak{P}_+\); and 3) the operators \(P_+AP_-\) and \(P_-AP_+\) are completely continuous. Then there exist at least one \(L_+\in\mathfrak{M}_+\) and one \(L_-\in\mathfrak{M}_-\) such that \(AL_+=L_+\) and \(AL_-=L_-\).

Proof. If \(A\mathfrak{P}_+\subset\mathfrak{P}_+\) (\(A\mathfrak{P}_-\subset\mathfrak{P}_-\)) and there exists \(A^{-1}(\in\mathfrak{R})\), then, obviously, \(A^{-1}\mathfrak{P}_-\subset\mathfrak{P}_-\) (\(A^{-1}\mathfrak{P}_+\subset\mathfrak{P}_+\)). Therefore, when conditions 1) and 2) of Theorem 2 are fulfilled, we always have \(A^{-1}\mathfrak{P}_\pm\subset\mathfrak{P}_\pm\), and \(A\) maps any \(L\in\mathfrak{M}_+\) (\(L\in\mathfrak{M}_-\)) into \(AL\in\mathfrak{M}_+\) (\(AL\in\mathfrak{M}_-\)). Thus, Theorem 2 is a simple consequence of Theorem 1.

1. An operator \(U(\in\mathfrak{R})\) is called \(J\)-unitary if \(U\mathfrak{H}=\mathfrak{H}\) and \(U^*JU=J\). Denote by \(\mathfrak{G}\) the set of all \(J\)-unitary operators \(U\) for which the condition \(P_+UP_-\in\mathfrak{S}_\infty\) is fulfilled. It is not difficult to show that this condition for a \(J\)-unitary operator \(U\) is equivalent to the condition \(P_-UP_+\in\mathfrak{S}_\infty\), and also that:

\(1^\circ.\) The set \(\mathfrak{G}\) is a subgroup of the group of all \(J\)-unitary operators in \(\mathfrak{H}\), and \(\mathfrak{G}=\mathfrak{G}^*\). The latter equality means that if \(U\in\mathfrak{G}\), then also \(U^*\in\mathfrak{G}\).

We shall show that, for each \(U\in\mathfrak{G}\), one can construct a unitary operator \(V\) (\(V^*V=VV^*=I\)) such that \(U-V\in\mathfrak{S}_\infty\).

Indeed, for \(U\in\mathfrak{G}\) we have
\(T=U-JUJ=(P_+ + P_-)U(P_+ + P_-)-(P_+-P_-)U(P_+-P_-)=2(P_+UP_-+P_-UP_+)\in\mathfrak{S}_\infty\), since \((U^*)^{-1}=JUJ=U-T\) and \(U^*U=I+S\), where \(S=U^*T\in\mathfrak{S}_\infty\). Define the self-adjoint-

operator \(S_1 \in \mathfrak S_\infty\) by the equality \(S_1=(I+S)^{1/2}-I\). Then \(I+S=(I+S_1)^2\), and the continuously invertible operator \(V=U(I+S_1)^{-1}\) will be unitary, since
\(V^*V=(I+S_1)^{-1}(I+S)(I+S_1)^{-1}=I\). On the other hand, \(VS_1\in\mathfrak S_\infty\).

On the basis of the general theorem of perturbation theory (see, for example, \((^5)\), Theorem 2.3) it follows from this that:

\(2^\circ\). Every nonunitary point \(\rho\) \((|\rho|\ne1)\) of the spectrum \(\sigma(U)\) of the operator \(U\in\mathfrak G\) is an eigenvalue of this operator, to which there corresponds a finite-dimensional normally separated root subspace \(\mathfrak L_\rho(U)\).

Consequently, the nonunitary spectrum
\[ \sigma_0(U)=\{\rho:\rho\in\sigma(U),\ |\rho|\ne1\} \]
of the operator \(U\in\mathfrak G\) consists of isolated points. Let us note that the nonunitary spectrum \(\sigma_0(U)\) of any \(J\)-unitary operator \(U\) has the following properties: if \(\rho\in\sigma_0(U)\), then also \(\bar\rho^{-1}\in\sigma_0(U)\), and if the root subspace \(\mathfrak L_\rho(U)\) is finite-dimensional and normally separated, then the root subspace \(\mathfrak L_{1/\bar\rho}(U)\) has the same properties; moreover, in \(\mathfrak L_\rho\) and in \(\mathfrak L_{1/\bar\rho}\) the operator \(U\) has identical elementary divisors (with \(\rho\) replaced by \(1/\bar\rho\)); furthermore, in this case \(\mathfrak L_\rho\) and \(\mathfrak L_{1/\bar\rho}\) are skewly related,* and the subspace \(\mathfrak N\) of all vectors \(J\)-orthogonal to the direct sum \(\mathfrak L_\rho+\mathfrak L_{1/\bar\rho}\) is invariant with respect to \(U\), and
\[ (U-\rho I)\mathfrak N=(U-\bar\rho^{-1}I)\mathfrak N=\mathfrak N \]
(cf. \((^6,^7)\)).

Theorem 3. Suppose that the nonunitary spectrum \(\sigma_0\) of the operator \(U\in\mathfrak G\) is in some way divided into two disjoint sets \(\sigma_I\) and \(\sigma_0\setminus\sigma_I\), symmetric with respect to the unit circle. Then there exist two subspaces \(L_\pm\in\mathfrak M_\pm\) with the properties: 1) \(UL_\pm=L_\pm\) and 2) the nonunitary spectrum of the restriction of the operator \(U\) to \(L_\pm\) coincides with \(\sigma_I\).

From properties 1) and 2), taking into account that \(L_\pm\in\mathfrak M_\pm\), it follows (cf. Lemma II.9 in \((^7)\)): 3) if \(\rho\in\sigma_I\), then \(\mathfrak L_\rho(U)\subset L_\pm\) and \(\mathfrak L_{1/\bar\rho}\cap L_\pm=\{0\}\).

Proof. According to Theorem 2, there exists \(L_+^{(0)}\in\mathfrak M_+\) with the first property: \(UL_+^{(0)}=L_+^{(0)}\). If \(L_+^{(0)}\) does not possess the second property, then it may be modified into some \(L_+\in\mathfrak M_+\) with properties 1) and 2) by the method which was applied in the analogous case in \((^8)\). The existence of \(L_-\in\mathfrak M_-\) with properties 1) and 2) is obtained from “symmetric” considerations.

1. A linear operator \(H\), acting in \(\mathfrak H\) with dense domain of definition \(\mathfrak D(H)\), is called \(J\)-self-adjoint if \(JH\) is an ordinary self-adjoint operator in \(\mathfrak H\) (for other equivalent definitions see \((^6,^7)\)). Such an operator is, evidently, closed. Therefore, if \(\mathfrak H_+\subset\mathfrak D(H)\), then \(H\) will be a bounded operator on \(\mathfrak H_+\), or, what is the same thing, the operators
   \(H_{11}=P_+HP_+\) and \(H_{21}=P_-HP_+\) will be bounded. In this case the closure of the operator \(H_{12}=P_+HP_-\) will also be bounded, since \(H_{12}^*=-H_{21}\). As for the operator \(H_{22}=P_-HP_-\), it is easy to see that, like the operator \(H_{11}\), it will be a self-adjoint operator, but unbounded, unless \(\mathfrak D(H)=\mathfrak H\).

Theorem 4. Let \(H\) be a \(J\)-self-adjoint operator such that: a) \(\mathfrak H_+\subset\mathfrak D(H)\) and b) the operator \(H_{12}(H_{22}-\xi I)^{(-1)}\), for some** \(\xi\notin\sigma(H_{22})\), maps \(\mathfrak H\) completely continuously into \(\mathfrak H_+\). Then the nonreal spectrum \(\sigma_0(H)\) of the operator \(H\) consists of isolated eigenvalues with normally separated root subspaces. Moreover, if this spectrum is in some way divided into two disjoint sets \(\Lambda\) and \(\bar\Lambda=\sigma_0\setminus\Lambda\), symmetric with respect to the real axis, then there is always a subspace \(L_+\in\mathfrak M_+\) with the properties: 1) \(L_+\subset\mathfrak D(H)\), 2) \(HL_+\subset L_+\), and 3) the spectrum of the restriction of \(H\) to \(L_+\) coincides with \(\Lambda\).

Proof. By elementary calculations it is shown that, for \(\operatorname{Im}\xi\) sufficiently large in absolute value, the operator \(H\)

* That is, in neither of the subspaces \(\mathfrak L_\rho\) and \(\mathfrak L_{1/\bar\rho}\) is there a vector \((\ne0)\) \(J\)-orthogonal to the other subspace.

** If condition b) is satisfied for some \(\xi\in\sigma(H_{22})\), then it is satisfied for all \(\xi\in\sigma(H_{22})\); by \((H_{22}-\xi I)^{(-1)}\) is denoted the resolvent at the point \(\xi\) of the restriction of the operator \(H_{22}\) to \(\mathfrak H_-\).

(with properties a) and b)) there exists a \(J\)-unitary Cayley transform
\[ U_\zeta=(H-\bar\zeta I)(H-\zeta I)^{-1}, \]
where
\[ \begin{aligned} P_+U_\zeta P_+&=\left[I+(\zeta-\bar\zeta)(I-T)^{-1}(H_{11}-\zeta I)^{-1}\right]P_+,\\ P_+U_\zeta P_-&=-(\zeta-\bar\zeta)(I-T)^{-1}(H_{11}-\zeta I)^{-1}H_{12}(H_{22}-\zeta I)^{-1}P_-, \end{aligned} \tag{3} \]
where
\[ T=(H_{11}-\zeta I)^{-1}H_{12}(H_{22}-\zeta I)^{-1}H_{21}\in\mathfrak S_\infty,\quad \|T\|<1. \]
From the second relation (3) it follows that \(U_\zeta\in\mathfrak S\), whence we obtain the first assertion of the theorem. According to Theorem 3, there will exist an \(L_+\in\mathfrak M_+\) such that \(U_\zeta L_+=L_+\), and the spectrum of the restriction of \(U_\zeta\) to \(L_+\) coincides with the image \(\Lambda\) under the mapping
\[ \mu=(\lambda-\bar\zeta)(\lambda-\zeta)^{-1}. \]
Using the boundedness of \(H_{11}\) and the first relation (3), one can show that \((U_\zeta-I)L_+=L_+\). The three indicated properties of \(L_+\) with respect to \(U_\zeta\) are equivalent to the properties 1), 2), and 3) asserted in Theorem 4.

Let us note that condition b) will be satisfied if it is replaced by the condition
\[ P_-HP_+\in\mathfrak S_\infty; \]
in this case Theorem 4 becomes a theorem proved by another method by G. Langer \((^7,^8)\) (under the assumption of separability of \(\mathfrak H\)). In turn, G. Langer’s theorem was an important generalization of a well-known theorem of L. S. Pontryagin \((^9)\) (see also \((^{10})\)), into which Theorem 4 passes when
\[ \varkappa=\dim\mathfrak H_+<\infty \]
(in this case conditions a) and b) of Theorem 4 are automatically satisfied).

The method of proof of L. S. Pontryagin’s theorem and of its generalizations on the basis of the fixed-point principle was first proposed by the author in \((^{11})\) (see also \((^{10})\)). This method received further development in a witty note by M. L. Brodskii \((^{12})\), in which, in particular, for the case \(\varkappa<\infty\), a proposition more complete than Theorem 1 was proved. As in \((^{12})\), Theorem 1 can be reformulated and proved for operators acting in a Banach space endowed with an indefinite metric.

The author expresses gratitude to G. K. Langer for the remark used above that the arguments presented here do not require the separability of the space \(\mathfrak H\).

Odessa Civil Engineering Institute

Received
17 X 1963

REFERENCES

1. R. S. Phillips, Trans. Am. Math. Soc., 90, No. 2, 193 (1959).
2. Yu. P. Ginzburg, DAN, 139, No. 4, 775 (1961).
3. Yu. P. Ginzburg, I. S. Iokhvidov, UMN, 17, 4 (106), 3 (1962).
4. N. Dunford, J. T. Schwartz, Linear Operators, Moscow, 1962.
5. I. Ts. Gokhberg, M. G. Krein, UMN, 12, 2 (74), 43 (1957).
6. I. S. Iokhvidov, Proc. V All-Union Conf. on Functional Analysis and Its Applications, Baku, 1961.
7. H. Langer, Math. Ann., 146, 60 (1962).
8. H. Langer, Math. Ann., 152, 434 (1963).
9. L. S. Pontryagin, Izv. AN SSSR, Ser. Math., 8, 243 (1944).
10. I. S. Iokhvidov, M. G. Krein, Tr. Moscow Math. Soc., 5, 367 (1956).
11. M. G. Krein, UMN, 5, 2 (36), 180 (1950).
12. M. L. Brodskii, UMN, 14, 1 (85), 147 (1959).