UDC 513.88

MATHEMATICS

B. D. ROMM

RESTRICTION OF A REPRESENTATION OF THE COMPLEMENTARY SERIES OF THE COMPLEX UNIMODULAR GROUP OF SECOND ORDER TO A REAL SUBGROUP

(Presented by Academician I. M. Vinogradov, 16 IX 1965)

I. In paper \((^6)\), representations of the principal series of the group \(G_0\) of complex matrices of second order with determinant equal to 1 are considered. If these representations are restricted to the real subgroup \(G\), then reducible representations arise, which there decompose into irreducible ones. In the present article a similar problem is solved for the complementary series.

As is known \((^1)\), a representation of the complementary series is determined by a parameter \(\sigma\), \(0<\sigma<2\), and is realized in a Hilbert space \(\mathfrak H_\sigma\), which is the closure of the set of functions \(f(z)\) on the complex plane satisfying the condition

\[ (f,f)=\int |z_1-z_2|^{-2+\sigma} f(z_1)\overline{f(z_2)}\,d z_1 d z_2<\infty, \tag{1} \]

where \(dz=dx\,dy\), \(x=\operatorname{Re} z\), \(y=\operatorname{Im} z\). The operators \(V_a\) of the representation have the form

\[ V_a f(z)=|\beta z+\delta|^{-2-\sigma} f(z\widetilde a), \tag{2} \]

where

\[ a=\begin{pmatrix}\alpha&\beta\\ \gamma&\delta\end{pmatrix}, \qquad z\widetilde a=\frac{\alpha z+\gamma}{\beta z+\delta}. \]

For the group \(G\) the operators \(V_a\) define a certain reducible unitary representation \(d\). The problem of the present article is to decompose the representation \(d\) into irreducible representations.

II. Consider the subrepresentation \(d^+\) of the representation \(d\) in the subspace \(\mathfrak H_\sigma^+\) of elements \(f\in \mathfrak H_\sigma\) with support in the upper half-plane \(Z^+\). The decomposition of \(d\) into irreducible representations reduces to the decomposition of the representations of the group \(G\) analogous to \(d^+\). Therefore we first decompose the representation \(d^+\).

Let \(\mathfrak H=\mathfrak H_\sigma^+\cap L_2^\mu(Z^+)\cap L_1^{\mu'}(Z^+)\), where the measures \(\mu(z)\) and \(\mu'(z)\) have the form
\(d\mu(z)=(\operatorname{Im} z)^\sigma dz\), \(d\mu'(z)=(\operatorname{Im} z)^{\sigma/2-1}dz\). Put, for \(z=i\widetilde a\), \(a\in G\),

\[ \psi(a)=(\operatorname{Im} z)^{\sigma/2+1} f(z). \tag{3} \]

It is easy to see that for any \(a_0\in G\)

\[ \psi(aa_0)=(\operatorname{Im} z)^{\sigma/2+1} V_{a_0}f(z). \tag{4} \]

The totality of all matrices \(u_\varphi\)

\[ u_\varphi= \begin{pmatrix} \cos\varphi&-\sin\varphi\\ \sin\varphi&\cos\varphi \end{pmatrix} \tag{5} \]

forms a subgroup \(U\) in the group \(G\). If the subgroup \(K\subset G\) consists of matrices

\[ k=\begin{pmatrix} \lambda&0\\ \mu&\lambda^{-1} \end{pmatrix},\quad \lambda>0, \]

then the unique decomposition holds:

\[ a=uk, \tag{6} \]

where \(u\in U\), \(k\in K\). For the Haar measure \(d\mu(a)\) on the group \(G\) we have

\[ d\mu(a)=d\varphi\,dz\,/\,2\pi(\operatorname{Im} z)^2, \tag{7} \]

where \(z=i\widetilde a=i\widetilde k=i\lambda^2+\lambda\mu\).

If \(u_1 \in U\), then \(\tilde u u_1=i\); therefore from the decomposition (6) it follows that

\[ \psi(ua)=\psi(a)=\psi(k). \]

From formulas (3) and (7) it follows that

\[ \int |\psi(a)|^2\,d\mu(a)=\int_{Z^+}|f(z)|^2(\operatorname{Im} z)^\sigma\,dz, \]

\[ \int |\psi(a)|\,d\mu(a)=\int_{Z^+}|f(z)|(\operatorname{Im} z)^{\sigma/2-1}\,dz. \tag{9} \]

Thus, \(\psi(a)\in L=L_2(G)\cap L_1(G)\), where the spaces \(L_p(G)\) are defined by the measure \(d\mu(a)\), if in (3) \(f(z)\in \mathfrak H\).

Let \(z=i\tilde a,\ z_1=i\tilde a_1\), where \(a,a_1\in G\). Denote

\[ \alpha_\sigma(a,a_1)=|z-z_1|^{-2+\sigma}/(\operatorname{Im} z\,\operatorname{Im} z_1)^{\sigma/2-1}. \tag{10} \]

According to (1), (3), and (7), for \(f(z)\in\mathfrak H\)

\[ (f,f)=\int \alpha_\sigma(a,a_1)\psi(a)\overline{\psi(a_1)}\,d\mu(a)\,d\mu(a_1), \tag{11} \]

\[ \alpha_\sigma(aa_0,a_1a_0)=\alpha_\sigma(a,a_1),\qquad a_0\in G, \tag{12} \]

\[ (f,f)=\int \alpha_\sigma(e,a_1)\int \psi(a)\overline{\psi(a_1a)}\,d\mu(a)\,d\mu(a_1), \tag{13} \]

where

\[ e=\begin{pmatrix}1&0\\0&1\end{pmatrix}. \]

We note that if a sequence of compact sets \(Q_i,\ i=1,2,\ldots,\) in \(G\) satisfies the conditions \(Q_1\subset Q_2\subset\cdots\) and \(\bigcup_i Q_i=G\), then

\[ (f,f)=\lim_{n\to\infty}\int_{Q_n}\alpha_\sigma(e,a_1)\int \psi(a)\overline{\psi(a_1a)}\,d\mu(a)\,d\mu(a_1), \tag{13a} \]

because an analogous equality holds for any summable function \(\varphi(a)\) on \(G\):

\[ \int \varphi(a)\,d\mu(a)=\lim_{n\to\infty}\int_{Q_n}\varphi(a)\,d\mu(a). \tag{13b} \]

Conversely, if \(\varphi(a)\) is a measurable function and the limit in the right-hand side of (13b) exists, then \(\varphi(a)\) is summable and (13b) holds.

III. Let \(a\to T_a\) be some irreducible unitary representation of the group \(G\). Put

\[ T_\psi=\int \psi(a)T_a^{-1}\,d\mu(a), \tag{14} \]

\(\psi(a)\in L\), because \(f(z)\in\mathfrak H\) according to (9); \(\Phi(b)=\int \psi(a)\overline{\psi(ba)}\,d\mu(a)\). According to (8), \(\Phi(bu_1)=\Phi(b)\); therefore \(T_\Phi=T_uT_\Phi\) for any \(u\in U\). It is clear that the inequality \(T_\Phi\ne0\) is possible only in the case of a representation of class 1.

Of the principal series described in the article \({}^{(2)}\), only the series \(C_q^0\) belongs to this class. It can be realized in the space \(L_2(-\infty,\infty)\) by the formula \({}^{(4)}\)

\[ T_a f(x)=|\beta x+\delta|^{i\rho-1}f\!\left(\frac{\alpha x+\gamma}{\beta x+\delta}\right),\qquad a=\begin{pmatrix}\alpha&\beta\\ \gamma&\delta\end{pmatrix}. \tag{15} \]

The parameter \(\rho\in(-\infty,\infty)\) determines the representation.

In \(L_2(-\infty,\infty)\) there exists, unique up to a numerical factor, a normalized vector \(f_0(x)\) satisfying the condition

\[ T_u f_0=f_0. \tag{16} \]

for all \(u \in U\) (see (2), §5). In view of the one-dimensionality of the operator \(T_\Phi\) in the case of the series \(C_q^0\),

\[ \operatorname{Sp} T_\Phi = (T_\Phi f_0, f_0)_1, \tag{17} \]

where \((f,f_1)_1\) is the scalar product in \(L_2(-\infty,\infty)\), and has the form:
\[ (f,f_1)_1=\int_{-\infty}^{\infty} f(x)\overline{f_1(x)}\,dx. \]

According to the results of the article \((^4)\)

\[ \int \psi(a)\overline{\psi(a_1a)}\,d\mu(a) = \frac{1}{32\pi^2}\int_0^\infty \rho\,\operatorname{th}\frac{\pi\rho}{2} \,(T_\psi^{*}T_\psi f_0,T_{a_1}f_0)_1\,d\rho = \]

\[ = \frac{1}{32\pi^2}\int_0^\infty \rho\,\operatorname{th}\frac{\pi\rho}{2} \,(T_\psi f_0,T_\psi f_0)_1 (f_0,T_a f_0)_1\,d\rho, \tag{18} \]

because the space of vectors \(f_0\) satisfying condition (16) is one-dimensional, so that from equalities (8) and (14) it follows that \(T_\psi^{*}f_0=T_uT_\psi^{*}f_0=\nu f_0\), where \(\nu=\mathrm{const}\), and the asterisk denotes passage to the adjoint operator.

The integral with respect to \(\rho\) in (18) converges uniformly in \(a_1\); therefore from equality (13a) we obtain

\[ (f,f)=\lim_{n\to\infty}\frac{1}{32\pi^2} \int_0^\infty \int_{Q_n} \alpha_\sigma(e,a)(f_0,T_af_0)_1\,d\mu(a)\, \rho\,\operatorname{th}\frac{\pi\rho}{2}\, (T_\psi f_0,T_\psi f_0)_1\,d\rho. \tag{19} \]

IV. For almost all \(a\in G\) there is a decomposition

\[ a=u_\varphi \varepsilon u_{\varphi_1}, \tag{20} \]

where \(u_\varphi\) is determined by formula (5), and
\[ \varepsilon= \begin{pmatrix} e^t & 0\\ 0 & e^{-t} \end{pmatrix} \]
with \(t>0\) (\((^4)\), appendix). \(a\) uniquely determines \(t,\varphi\), and \(\varphi_1\), if \(a\ne e\), \(\varphi,\varphi_1\in[0,\pi)\). In view of (16), \(\nu(a)=\nu(\varepsilon)\), where \(\nu(a)=(f_0,T_af_0)\).

It is easy to show \((^5)\) that

\[ \nu(\varepsilon)=\frac12\left[P_{(-i\rho-1)/2}(\operatorname{ch}2t)+P_{(i\rho-1)/2}(\operatorname{ch}2t)\right], \tag{21} \]

where \(P_\nu(z)\) is the spherical Legendre function of the 1st kind.

From equalities (12) and (20) it follows that \(\alpha_\sigma(e,a)=|2\operatorname{sh}t|^{-2+\sigma}\). If we put \(y=\operatorname{ch}2t\), then equality (19) may be written in the form

\[ (f,f)=\lim_{n\to\infty}\frac{2^{\sigma/2-7}}{\pi} \int_0^\infty \int_{\widetilde Q_n} (y-1)^{-1+\sigma/2} \left[P_{(i\rho-1)/2}(y)+P_{(-i\rho-1)/2}(y)\right]\,dy\,\rho \times \]

\[ {}\times \operatorname{th}\frac{\pi\rho}{2} \int h(x,\rho)\overline{h(x,\rho)}\,dx\,d\rho, \tag{22} \]

where, according to (8) and (15), \(h(x,\rho)=T_\varphi f_0\), \(\widetilde Q_n=\{y:\ y=\operatorname{ch}2t,\ a\in Q_n\}\).

If we put

\[ K(z,x,\rho)=\frac{1}{\sqrt{\pi}}(\operatorname{Im}z)^{(\sigma-1-i\rho)/2} \left|i(x-\operatorname{Re}z)-\operatorname{Im}z\right|^{i\rho-1}, \]

then, according to (3) (\(K\) is the integral operator with kernel \(K(z,x,\rho)\)),

\[ h(x,\rho)=\int_{Z^+} K(z,x,\rho)f(z)\,dz=Kf(z),\qquad f(z)\in\mathfrak H. \tag{23} \]

From the decomposition (20) it follows that \(a^2+\beta^2+\gamma^2+\delta^2=2\operatorname{ch}2t\), so that

\[ \widetilde Q_n=\left\{y:\ y=\frac{a^2+\beta^2+\gamma^2+\delta^2}{2},\ a\in Q_n\right\}. \tag{24} \]

The inversion formula has the form

\[ f(z)=\frac{1}{64\pi^2}(\operatorname{Im}z)^{-\sigma} \int\!\!\int_{-\infty}^{\infty} \rho\,\operatorname{th}\frac{\pi\rho}{2}\, \overline{K(z,x,\rho)}\,h(x,\rho)\,dx\,d\rho . \tag{25} \]

In view of (4) and (14), from the relation \(h=T_{\psi}f_0\) it follows that \(h(x,\rho)\) passes into \(T_{a_0}h(x,\rho)\) when \(f(z)\) passes into \(V_{a_0}f(z)\).

Theorem 1. Let \(\mathfrak H_+\) denote the Hilbert space obtained by completing, in the scalar product

\[ [h,h_1]_+= \lim_{n\to\infty}\frac{2^{\sigma/2-7}}{\pi} \int_0^\infty\int_{\widetilde Q_n} (y-1)^{-1+\sigma/2} \left[P_{(i\rho-1)/2}(y)+P^{*}_{(-i\rho-1)/2}(y)\right]\,dy\,\rho \]
\[ {}\times \operatorname{th}\frac{\pi\rho}{2} \int h(x,\rho)\overline{h_1(x,\rho)}\,dx\,d\rho \]

the set of all measurable functions \(h(x,\rho)\) for which

\[ (h,h)=\iint |h(x,\rho)|^2\rho\,\operatorname{th}\frac{\pi\rho}{2}\,dx\,d\rho<\infty \]

and the limit on the right-hand side of equality (22) exists. Then \(\mathfrak H_+\) is isometric to \(\mathfrak H_\sigma^+\), and the isometry formula (22) is the Plancherel formula for the representation \(d^+\). (23) and (25) are mutually inverse formulas, which hold when \(f(z)\in\mathfrak H\).

V. Let now \(f(z)\in\mathfrak H_\sigma\). Put \(\varphi(z)=(f(z)+f(\bar z))/2\), \(\varphi_1(z)=(f(z)-f(\bar z))/2\).

Theorem 2. Let \(\mathfrak H(\mathfrak H_1)\) be the Hilbert space obtained by completing, in the scalar product \([h,h_1]\) \(([h,h_1]_1)\),

\[ [h,h_1]= \lim_{n\to\infty}\frac{2^{\sigma/2-6}}{\pi} \int_0^\infty\int_{\widetilde Q_n} \left[(y-1)^{-1+\sigma/2}+(y+1)^{-1+\sigma/2}\right] \left[P_{(i\rho-1)/2}(y)+\right. \]
\[ \left.{}+P_{(-i\rho-1)/2}(y)\right]\,dy\,\rho\, \operatorname{th}\frac{\pi\rho}{2} \int h(x,\rho)\overline{h_1(x,\rho)}\,dx\,d\rho, \]

\[ [h,h_1]_1= \lim_{n\to\infty}\frac{2^{\sigma/2-6}}{\pi} \int_0^\infty\int_{\widetilde Q_n} \left[(y-1)^{-1+\sigma/2}-(y+1)^{-1+\sigma/2}\right] \left[P_{(i\rho-1)/2}(y)+\right. \]
\[ \left.{}+P_{(-i\rho-1)/2}(y)\right]\,dy\,\rho\, \operatorname{th}\frac{\pi\rho}{2} \int h(x,\rho)\overline{h_1(x,\rho)}\,dx\,d\rho \]

the set \(M(M_1)\) of all measurable functions \(h(x,\rho)\) for which \((h,h)<\infty\), \([h,h]<\infty\), \(((h,h)<\infty,\ [h,h]_1<\infty)\). Then the Plancherel formula for the representation \(d\) holds:

\[ (f,f)=[h,h]+[h_1,h_1]_1, \]

where \(h(x,\rho)=K\varphi(z)\), \(h_1(x,\rho)=K\varphi_1(z)\), and \(\varphi(z)\) and \(\varphi_1(z)\) belong to certain dense sets, which are defined analogously to \(\mathfrak H\). There is an inversion formula of the form (25).

Moscow Institute of Physics and Technology

Received
9 IX 1965

CITED LITERATURE

1. M. A. Naimark, Linear Representations of the Lorentz Group, Moscow, 1958.
2. V. Bargmann, Ann. Math., 47, 568 (1948).
3. Harish-Chandra, Proc. Nat. Acad. Sci. U.S.A., 38, No. 4, 855 (1952).
4. D. B. Romm, Izv. AN SSSR, ser. matem., 29, No. 5 (1965).
5. N. N. Lebedev, Special Functions and Their Applications, Moscow, 1963.
6. B. D. Romm, DAN, 152, No. 1 (1963).