Holonomicity of relative characters and applications to multiplicity bounds for spherical pairs

Abstract

We prove that any relative character (a.k.a. spherical character) of any admissible representation of a real reductive group with respect to any pair of spherical subgroups is a holonomic distribution on the group. This implies that the restriction of the relative character to an open dense subset is given by an analytic function. The proof is based on an argument from algebraic geometry and thus implies also analogous results in the p-adic case. As an application, we give a short proof of some results of Kobayashi-Oshima and Kroetz-Schlichtkrull on boundedness and finiteness of multiplicities of irreducible representations in the space of functions on a spherical space. To deduce this application we prove the relative and quantitative analogs of the Bernstein–Kashiwara theorem, which states that the space of solutions of a holonomic system of differential equations in the space of distributions is finite-dimensional. We also deduce that, for every algebraic group ${{G}}$ defined over $\mathbb {R}$, the space of ${{G(\mathbb {R})}}$-equivariant distributions on the manifold of real points of any algebraic ${{G}}$-manifold ${{X}}$ is finite-dimensional if ${{G}}$ has finitely many orbits on ${{X}}$.

1 Introduction

1.1 The relative character

Let $G$ be a reductive group¹ defined over $\mathbb {R}$. In this paper, we prove that a relative character (a.k.a. spherical character) of a smooth admissible Fréchet  representation of moderate growth of $G(\mathbb {R})$ is holonomic. The relative character is a basic notion of relative representation theory that generalizes the notion of a character of a representation. Let us now recall the notions of spherical pair, relative character and holonomic distribution. For the notion of smooth admissible Fréchet  representation of moderate growth and its relation to Harish-Chandra modules, we refer the reader to [11] or [32, Chapter 11].

Definition 1.1

Let $H\subset G$ be a (closed) algebraic subgroup defined over $\mathbb {R}$. Let P denote a minimal parabolic subgroup of $G$ defined over $\mathbb {R}$ and $B$ denote a Borel subgroup of $G$ (possibly not defined over $\mathbb {R}$). The subgroup $H$ is called spherical if it has finitely many orbits on $G{/}B$. We will call $H$ strongly real spherical if it has finitely many orbits on $G{/}P$.

It is known that a pair $(G,H)$ is spherical if and only if $H$ has an open orbit on $G{/}B$.

Definition 1.2

Let $H_1,H_2\subset G$ be spherical subgroups and let $\mathfrak {h}_i$ be the Lie algebras of $H_i$. Let $\chi _i$ be characters of ${{\mathfrak {h}_i}}$. Let $\pi$ be a smooth admissible Fréchet  representation of moderate growth of $G{{(\mathbb {R})}}$, $\pi ^*$ the continuous dual of $\pi$, and let $\hat{\pi }\subset \pi ^*$ be the smooth contragredient representation to $\pi$ (i.e., the only smooth admissible Fréchet  representation of moderate growth with the same underlying Harish-Chandra module as $\pi ^*$). Let $\phi _1 \in (\pi ^*)^{{{\mathfrak {h}}}_1,\chi _1}$ and $\phi _2 \in (\hat{\pi }^*)^{{{\mathfrak {h}}}_2,\chi _2}$ be equivariant functionals. Fix a Haar measure on $G{{(\mathbb {R}}})$. It gives rise to an action of the space of Schwartz functions $\mathcal {S}(G{{(\mathbb {R}}}))$ on $\pi ^*$ and $\hat{\pi }^*$, and this action maps elements of $\pi ^*$ and $\hat{\pi }^*$ to elements of $\hat{\pi }$ and $\hat{\hat{\pi }}=\pi$ respectively. For the definition of the space of Schwartz functions $\mathcal {S}(G{{(\mathbb {R}}}))$ see, e.g., [4, 11, 32].

The relative character $\xi _{\phi _1,\phi _2}$ of $\pi$, with respect to $\phi _1$ and $\phi _2$, is the tempered distribution on $G{{(\mathbb {R}}})$ (i.e., a continuous functional on $\mathcal {S}(G{{(\mathbb {R}}}))$) defined by $\langle \xi _{\phi _1,\phi _2} , f \rangle = \langle \phi _1, \pi (f) \cdot \phi _2 \rangle$.

Definition 1.3

Let $X$ be an algebraic manifold defined over $\mathbb {R}$. Let $\xi \in \mathcal {S}^*(X(\mathbb {R}))$ be a tempered distribution. The singular support² ${{\mathrm{SS}}}(\xi )$ of $\xi$ is the zero locus in $T^*X$ of all the symbols of (algebraic) differential operators that annihilate $\xi$. The distribution $\xi$ is called holonomic if $\dim {{\mathrm{SS}}}(\xi ) = \dim X$.

In this paper we prove the following theorem.

Theorem A

(See § 4.2) In the case of Definition 1.2, the relative character $\xi _{\phi _1,\phi _2}$ is holonomic.

We prove Theorem A using the following well-known statement.

Proposition 1.4

(See § 4.2) Let $\mathfrak {g},\mathfrak {h}_i$ be the Lie algebras of $G$ and $H_i$, $i=1,2$. Identify $T^*G$ with $G\times \mathfrak {g}^*$ and let

$$\begin{aligned} S:=\{(g,\alpha ) \in G\times \mathfrak {g}^*\, \mid \, \alpha \text { is nilpotent, } \langle \alpha , \mathfrak {h}_1 \rangle =0, \ \langle \alpha , Ad^*(g)(\mathfrak {h}_2) \rangle =0\}. \end{aligned}$$

Then ${{\mathrm{SS}}}(\xi _{\phi _1,\phi _2}) \subset S.$

Note that the Bernstein inequality states that the dimension of the singular support of any non-zero distribution is at least the dimension of the underlying manifold. Thus Theorem A follows from the following more precise version, which is the core of this paper.

Theorem B

(See § 2) We have $\dim S= \dim G$.

Let $U:=\left\{ g \in G\, \mid \, S\cap T_g^*G=\{(g,0)\}\right\} .$ Note that $U$ is Zariski open since $S$ is conic and closed. It is easy to see that Theorem B implies the following corollary.

Corollary C

The set $U$ is a Zariski open dense subset of $G$.

This corollary is useful in view of the next proposition, which follows from Proposition 1.4 and Corollary 3.3 below.

Proposition 1.5

The restriction $\xi _{\phi _1,\phi _2}|_{{{U(\mathbb {R})}}}$ is an analytic function.

Remark 1.6

In general, $S$ has irreducible components that cannot lie in ${{\mathrm{SS}}}(\xi _{\phi _1,\phi _2})$ for any $\phi _1,\phi _2$. Indeed, ${{\mathrm{SS}}}(\xi _{\phi _1,\phi _2})$ is coisotropic by [14, 22, 27], and thus, by Theorem A, Lagrangian. On the other hand, one can show that when $G=\mathrm {GL}_4(\mathbb {R})$, and $H_1=H_2=\mathrm {GL}_2(\mathbb {R})\times \mathrm {GL}_2(\mathbb {R})$ embedded as block matrices inside $G$, the variety $S$ has non-isotropic (and thus non-Lagrangian) components.

1.2 Bounds on the dimension of the space of solutions

Next we apply our results to representation theory. For this, we use the following theorem.

Theorem 1.7

(Bernstein–Kashiwara) Let $X$ be an algebraic manifold defined over $\mathbb {R}$. Let

$$\begin{aligned}\{D_i \xi =0\}_{i=1... n}\end{aligned}$$

be a system of linear PDE on ${{\mathcal {S}^*(X(\mathbb {R}))}}$ with algebraic coefficients. Suppose that the joint zero set of the symbols of $D_i$ in $T^*X$ is $\dim X$-dimensional. Then the space of solutions of this system is finite-dimensional.

It seems that this theorem is not found in the literature in this formulation; however it has two proofs, one due to Kashiwara (see [20, 21] for similar statements) and another due to Bernstein (unpublished).

In order to make our applications in representation theory more precise, we need an effective version of this theorem. We prove such a version (see Theorem 3.13 below) following Bernstein’s approach, as it is more appropriate for effective bounds. We use this effective version to derive a relative version. Namely, we show that if the system depends on a parameter in an algebraic way, then the dimension of the space of solutions is bounded (see § 3.3 below).

This relative version allows us to deduce the following theorem.

Theorem D

(See § 3.3) Let $G$ be an algebraic group defined over $\mathbb {R}$ and let $X$ be an algebraic $G$-manifold with finitely many orbits. Let $\mathfrak {g}$ be the Lie algebra of $G$. Let $\mathcal {E}$ be an algebraic $G$-equivariant bundle on $X$. Then, for any natural number $n\in \mathbb {N}$, there exists $C_n \in \mathbb {N}$ such that for every n-dimensional representation $\tau$ of $\mathfrak {g}$, we have

$$\begin{aligned} \dim {{\mathrm{Hom}}}_{\mathfrak {g}}(\tau , \mathcal {S}^*(X(\mathbb {R}),\mathcal {E}))\le C_{n}, \end{aligned}$$

where ${{\mathrm{Hom}}}_{\mathfrak {g}}$ denotes the space of all continuous $\mathfrak {g}$-equivariant maps.

Remark 1.8

Note that the condition that $G$ has finitely many orbits on $X$ is equivalent to $G(\mathbb {C})$ having finitely many orbits on $X(\mathbb {C})$ but not equivalent to (and not implied by) $G(\mathbb {R})$ having finitely many orbits on $X(\mathbb {R})$.

1.3 Applications to representation theory

Using § 3, we give a short proof of some results from [24, 25]. Namely, we prove:

Theorem E

(See § 4) Let G be a reductive group defined over $\mathbb {R}$, $H\subset G$ a Zariski closed subgroup, and let $\mathfrak {h}$ be the Lie algebra of $H$.

• (i) If $H$ is a strongly real spherical subgroup then, for every irreducible smooth admissible Fréchet  representation of moderate growth $\pi \in {{\mathrm{Irr}}}({{G(\mathbb {R}))}}$, and natural number $n\in \mathbb {N}$ there exists $C_n \in \mathbb {N}$ such that for every n-dimensional representation $\tau$ of $\mathfrak {h}$ we have

  $$\begin{aligned}\dim {{\mathrm{Hom}}}_{\mathfrak {h}}(\pi ,\tau ) \le C_n.\end{aligned}$$

• (ii) If H is a spherical subgroup and we consider only one-dimensional $\tau$, then the space is universally bounded, i.e., there exists $C \in \mathbb {N}$ such that $\dim (\pi ^*)^{\mathfrak {h},\chi } \le C$ for any $\pi \in {{\mathrm{Irr}}}({{G(\mathbb {R}))}})$ and any character $\chi$ of $\mathfrak {h}$.

Corollary F

Let G be a reductive group defined over $\mathbb {R}$, $H\subset G$ a Zariski closed reductive subgroup, and let $\mathfrak {h}$ be the Lie algebra of H.

• (i) If the diagonal $\varDelta H$ is a strongly real spherical subgroup in $G\times H$, then for every $\pi \in {{\mathrm{Irr}}}({{G(\mathbb {R}))}})$ and $\tau \in {{\mathrm{Irr}}}({{H(\mathbb {R}))}})$ we have finite multiplicities, i.e.,

  $$\begin{aligned} \dim {{\mathrm{Hom}}}_{\mathfrak {h}}(\pi ,\tau )< \infty . \end{aligned}$$

• (ii) If the diagonal $\varDelta H$ is a spherical subgroup in $G\times H$, then the multiplicities are universally bounded, i.e., there exists $C \in \mathbb {N}$ such that for every $\pi \in {{\mathrm{Irr}}}({{G(\mathbb {R}))}})$, $\tau \in {{\mathrm{Irr}}}({{H(\mathbb {R}))}})$ we have

  $$\begin{aligned} \dim {{\mathrm{Hom}}}_{\mathfrak {h}}(\pi ,\tau ) \le C. \end{aligned}$$

This corollary follows from Theorem E since ${{\mathrm{Hom}}}_{\mathfrak {h}}(\pi ,\tau )$ lies in the space of $\varDelta \mathfrak {h}$-invariant functionals on the completed tensor product $\pi \widehat{\otimes } \hat{\tau }\in {{\mathrm{Irr}}}({{G(\mathbb {R}))}}\times {{H(\mathbb {R}))}})$ (see [5, Corollary A.0.7 and Lemma A.0.8]). All symmetric pairs satisfying the conditions of the corollary were classified in [23].

The inverse implications for Theorem E(ii) and Corollary F(ii) are proved in [24].

The results on multiplicities in [24, 25] are stronger than Theorem E since they do not require H to be algebraic, and consider maps from the Harish-Chandra space of $\pi$ to $\tau$. Also, in [24, 25] Theorem E(i) and Corollary F(i) are proved in the wider generality of real spherical subgroups.

In addition, [24, Theorem B] implies that if $H\subset G$ is an algebraic spherical subgroup, there exists $C \in \mathbb {N}$ such that $\dim {{\mathrm{Hom}}}_{\mathfrak {h}}(\pi ,\tau ) \le C\dim \tau ,$ for every $\pi \in {{\mathrm{Irr}}}(G{{(\mathbb {R})}})$ and every finite-dimensional continuous representation $\tau$ of $H{{(\mathbb {R})}}$. It is easy to modify our proof of Theorem E(ii) to show the boundedness of multiplicities for any $\pi \in {{\mathrm{Irr}}}(G{{(\mathbb {R})}})$ and any $\tau$ of a fixed dimension, but the proof that the bound depends linearly on this dimension would require more work.

Our methods are different from the methods of [24], which in turn differ from those of [25], and the bounds given in the three works are probably very different.

1.4 The non-Archimedean case

Theorem B and Corollary C hold over arbitrary fields of characteristic zero. They are useful also for p-adic local fields F, since the analogs of Propositions 1.4 and 1.5 hold in this case, see [8, Theorem A and Corollary F]. Namely, we have the following theorem.

Theorem 1.9

([8]) Let G be a reductive group defined over a non-Archimedean field F of characteristic 0 and let $\xi$ be a relative character of a smooth admissible representation with respect to two spherical subgroups $H_1,H_2\subset G$. Let S and U be the sets defined in Proposition 1.4 and Corollary C. Then

• (i) The wave front set of $\xi$ lies in $S{{(F)}}$.

• (ii) The restriction of $\xi$ to $U{{(F)}}$ is given by a locally constant function.

1.5 Related results

In the group case, i.e., the case when $G=H\times H$ and $H_1=H_2=\varDelta H \subset H \times H$, Theorem A essentially becomes the well-known fact that characters of admissible representations are holonomic distributions.

As we mentioned above, Theorem E was proved earlier in [24, 25] using different methods. An analog of Theorem E(i) over non-Archimedean fields is proved in [13] and [30, Theorem 5.1.5] for many spherical pairs, including arbitrary symmetric pairs.

The group case of Corollary C, Proposition 1.5, and Theorem 1.9(ii) is (the easy part of) the Harish-Chandra regularity theorem (see [16, 17]). Another known special case of these results is the regularity of Bessel functions, see [6, 7, 26].

1.6 Future applications

Our proof of Theorem E(ii) does not use the Casselman embedding theorem (Theorem 4.3). This gives us hope that it can be extended to the non-Archimedean case. The main difficulty is the fact that our proof heavily relies on the theory of modules over the ring of differential operators, which does not act on distributions in the non-Archimedean case. However, in view of Thereom 1.9 we believe that this difficulty can be overcome. Namely, one can deduce an analog of Theorem E(ii) for many spherical pairs from the following conjecture.

Conjecture G

Let G be a reductive group defined over a non-Archimedean field F of characteristic 0 and let $H_1,H_2\subset G$ be its (algebraic) spherical subgroups. Let $\chi _i$ be characters of $H_i{{(F)}}$. Fix a character $\lambda$ of the Bernstein center $\mathfrak {z}(G{{(F)}})$.

Then the space of distributions which are:

• (1) left $(H_1{{(F)}},\chi _1)$-equivariant,

• (2) right $(H_2{{(F)}},\chi _2)$-equivariant,

• (3) $(\mathfrak {z}(G{{(F)}}),\lambda )$-eigen,

is finite-dimensional. Moreover, this dimension is uniformly bounded when $\lambda$ varies.

Note that Theorems B and 1.9(i) imply that the dimension of (the Zariski closure of) the wave front set of a distribution that satisfies (1–3) does not exceed $\dim G$. In many ways the wave front set replaces the singular support, in absence of the theory of differential operators (see, e.g., [2, 3, 7, 8]). Thus, in order to prove Conjecture G, it is left to prove analogs of Theorems 1.7 and 3.13 for the integral system of equations (1–3).

1.7 Structure of the paper

In § 2, we prove Theorem B using a theorem of Steinberg [31] concerning the Springer resolution.

In § 3, we prove an effective version of Theorem 1.7, and then adapt it to algebraic families. We also derive Theorem D.

In § 4, we derive Theorem E from Theorem B and § 3. We do that by embedding the multiplicity space into a certain space of relative characters.

In Appendix A, we prove Lemma 3.1 which computes the pullback of the D-module of distributions with respect to a closed embedding. We use this lemma in § 3.

2 Proof of Theorem B

It is enough to prove the theorem for a reductive group G defined over an algebraically closed field of characteristic 0. Since S includes the zero section of ${{T^*G \cong }}G \times \mathfrak {g}^*$, we have $\dim S \ge \dim G$. Thus, it is enough to prove that $\dim S \le \dim G$. Let $\mathcal {B}$ denote the flag variety of G and $\mathcal {N}\subset \mathfrak {g}^*$ denote the nilpotent cone. Since G is reductive, we can identify

$$\begin{aligned} T^*\mathcal {B}\cong \{ (B,X) {{\in \mathcal {B}\times \mathfrak {g}^*}} \, \vert \, X \in (\mathrm {Lie} B)^{\bot } \}. \end{aligned}$$

Recall the Springer resolution $\mu :T^*\mathcal {B}\rightarrow \mathcal {N}$ defined by $\mu (B,X)=X$ and consider the following diagram.

(2.1)

Here, $\alpha$ is defined by $\alpha (g,X)=(X,{{\mathrm{Ad}}}^*(g^{-1})X)$, and $\mathrm {Res}$ is the restriction. Passing to the fiber of $0\in \mathfrak {h}_1^* \times \mathfrak {h}_2^*$, we obtain the following diagram.

(2.2)

Here, ${\mathcal {N}}_{\mathfrak {h}_i}:=\mathcal {N}\cap \mathfrak {h}_i^{\bot }$ and $L_i:=\{ (B,X) \in T^*\mathcal {B}\, \vert \, X \in \mathfrak {h}_i^{\bot } \}$. We need to estimate $\dim S$. We do it using the following lemma.

Lemma 2.1

(See § 2.1 below) Let $\varphi _i:X_i\rightarrow Y$, $i=1,2$, be morphisms of algebraic varieties. Suppose that $\varphi _2$ is surjective. Then there exists $y\in Y$ such that

$$\begin{aligned} \dim X_1\le \dim X_2 + \dim \varphi _1^{-1}(y) - \dim \varphi _2^{-1}(y). \end{aligned}$$

By this lemma, applied to $\phi _1={\alpha }'$ and $\phi _2=\mu '$, it is enough to estimate the dimensions of $L_i$ and of the fibers of $\mu '$ and $\alpha '$.

Lemma 2.2

We have $\dim L_1=\dim L_2 = \dim \mathcal {B}.$

Proof

Since $H_i$ has finitely many orbits in $\mathcal {B}$, it is enough to show that $L_i$ is the union of the conormal bundles to the orbits of $H_i$ in $\mathcal {B}$. Let $B\in \mathcal {B}$, and $\mathfrak {b}=\mathrm {Lie} B$, and identify $T_B\mathcal {B}\cong \mathfrak {g}/\mathfrak {b}$. Then $T_B(H_i\cdot B)\cong \mathfrak {h}_i /(\mathfrak {b}\cap \mathfrak {h}_i)$ and the conormal space at B to the $H_i$-orbit of B is identified with $\mathfrak {b}^\bot \cap \mathfrak {h}_i^\bot$. $\square$

Let $(\eta , {{\mathrm{Ad}}}^*(g)\eta )\in \mathrm {Im}(\alpha ')$. The fiber $(\alpha ')^{-1}(\eta , {{\mathrm{Ad}}}^*(g)\eta )$ is isomorphic to the stabilizer $G_{\eta }$, and the dimension of the fiber $(\mu ')^{-1}(\eta , {{\mathrm{Ad}}}^*(g)\eta )$ is twice the dimension of the Springer fiber $\mu ^{-1}(\eta )$. Recall the following theorem of Steinberg (conjectured by Grothendieck):

Theorem 2.3

([31, Theorem 4.6])

$$\begin{aligned} \dim G_\eta -2\dim \mu ^{-1}(\eta )={{\mathrm{rk}}}G. \end{aligned}$$

Using Lemma 2.1, we obtain for some $(\eta , \mathrm{ad}^*(g)\eta )$:

$$\begin{aligned} \dim S\le & {} \dim (L_1\times L_2) + \dim (a')^{-1}(\eta , \mathrm{ad}^*(g)\eta ) - \dim (\mu ')^{-1}(\eta , \mathrm{ad}^*(g)\eta ) \\= & {} 2\dim \mathcal {B}+ \dim G_\eta -2\dim \mu ^{-1}(\eta )=2\dim \mathcal {B}+ {{\mathrm{rk}}}G = \dim G. \end{aligned}$$

2.1 Proof of Lemma 2.1

Recall that, for a dominant morphism $\varphi : X\rightarrow Y$ of irreducible varieties, there exists an open dense $U\subset Y$ such that $\dim X=\dim Y+\dim \varphi ^{-1}(y)$ for all $y\in U$ (see, e. g., [29, Theorem 1.8.3]). Let Z be an irreducible component of $X_1$ of maximal dimension and let $W\subset Y$ be the Zariski closure of $\varphi _1(Z)$. Since W is irreducible, there exists an open dense $U\subset W$ such that

$$\begin{aligned} {{\dim X_1 =}} \dim Z\le \dim W+\dim \varphi _1^{-1}(y) \end{aligned}$$

(2.3)

for all $y\in U$. Let $V\subset U$ be an open dense subset such that $\varphi _2^{-1}(V)$ intersects those and only those irreducible components $C_1,\ldots ,C_j$ of $\varphi _2^{-1}(W)$ that map dominantly to W. Note that $j>0$ since $\varphi _2$ is surjective. Moreover, without loss of generality, we may assume that for every $1\le i\le j$, all fibers over V of the restriction of $\varphi _2$ to $C_i$ are of the same dimension. Since one of these dimensions has to be equal to $\dim \varphi _2^{-1}(V)-\dim V$, we have that there is an $1\le i\le j$ such that, for all $y\in V$,

$$\begin{aligned} \dim V&=\dim C_i - \dim (\varphi _2|_{C_i})^{-1}(y) \nonumber \\&\le \dim \varphi _2^{-1}(V)-\dim \varphi _2^{-1}(y)\le \dim X_2-\dim \varphi _2^{-1}(y). \end{aligned}$$

(2.4)

Thanks to $\dim V=\dim W$, taking any $y\in V$, formulas (2.3) and (2.4) imply the statement.$\square$

3 Dimension of the space of solutions of a holonomic system

In this section, we prove an effective version of Theorem 1.7, and then adapt it to algebraic families. We also derive Theorem D.

3.1 Preliminaries

3.1.1 D-modules

In this section, we will use the theory of D-modules on algebraic varieties over an arbitrary field k of characteristic zero. We will now recall some facts and notions that we will use. For a good introduction to the algebraic theory of D-modules, we refer the reader to [9] and [10]. For a short overview, see [5, Appendix B].

By a D-module on a smooth algebraic variety X, we mean a quasi-coherent sheaf of right modules over the sheaf $D_X$ of algebras of algebraic differential operators. By a finitely generated D-module on a smooth algebraic variety X, we mean a coherent sheaf of right modules over the sheaf $D_X$. Denote the category of $D_X$-modules by $\mathcal {M}(D_X)$.

For a smooth affine variety V, we denote $D(V):=D_V(V)$. Note that the category $\mathcal {M}(D_V)$ of D-modules on V is equivalent to the category of D(V)-modules. We will thus identify these categories.

The algebra D(V) is equipped with a filtration which is called the geometric filtration and defined by the degree of differential operators. The associated graded algebra with respect to this fitration is the algebra $\mathcal {O}(T^*V)$ of regular functions on the total space of the cotangent bundle of V. This allows us to define the singular support of a finitely generated D-module M on V in the following way. Choose a good filtration on M, i.e., a filtration such that the associated graded module is a finitely-generated module over $\mathcal {O}(T^*V)$, and define the singular support SS(M) to be the support of this module. One can show that the singular support does not depend on the choice of a good filtration on M.

This definition easily extends to the non-affine case. A finitely generated D-module M on X is called smooth if SS(M) is the zero section of $T^*X$. This is equivalent to being coherent over $\mathcal {O}_X$ and to being coherent and locally free over $\mathcal {O}_X$. The Bernstein inequality states that for any non-zero finitely generated M, we have $\dim SS(M) \ge \dim X$. If the equality holds, then M is called holonomic.

For a closed embedding $i:X \rightarrow Y$ of smooth affine algebraic varieties, define the functor $i^!:\mathcal {M}(D_Y)\rightarrow \mathcal {M}(D_X)$ by $i^!(M):=\{m\in M \, \vert \, I_Xm=0\},$ where $I_X\subset \mathcal {O}(Y)$ is the ideal of all functions that vanish on X. It has a left adjoint functor $i_*:\mathcal {M}(D_X) \rightarrow \mathcal {M}(D_Y)$, given by the tensor product with $i^!(D_Y)$. The functor $i_*$ is an equivalence of categories between $\mathcal {M}(D_X)$ and the category of $D_Y$-modules supported in X. Both $i_*$ and $i^!$ map holonomic modules to holonomic ones.

If V is an affine space, then the algebra D(V) has an additional filtration, called the Bernstein filtration. It is defined by $\deg (\partial /\partial x_i)=\deg (x_i)=1,$ where $x_i$ are the coordinates in V. This gives rise to the notion of Bernstein’s singular support, which we will denote $SS_b(M)\subset T^*V\cong V\oplus V^*$. It is known that $\dim SS(M)=\dim SS_b(M)$.

We will also use the theory of analytic D-modules. By an analytic D-module on a smooth complex analytic manifold X, we mean a coherent sheaf of right modules over the sheaf $D_X^{\mathrm {An}}$ of algebras of differential operators with analytic coefficients. All of the above notions and statements, except for those concerning the Bernstein filtration, have analytic counterparts. In addition, all smooth analytic D-modules of the same rank are isomorphic.

3.1.2 Distributions

We will use the theory of distributions on differentiable manifolds and the theory of tempered distributions on real algebraic manifolds, see e.g., [4, 19]. For an algebraic manifold X defined over $\mathbb {R}$, we denote the space of real valued distributions on $X(\mathbb {R})$ by $\mathcal {D}'(X(\mathbb {R}),\mathbb {R}):=(C_c^{\infty }(X(\mathbb {R}),\mathbb {R}))^*$ and the space of real valued tempered distributions (a.k.a. Schwartz distributions) by $\mathcal {S}^*(X{{(\mathbb {R})}},\mathbb {R}):=(\mathcal {S}(X{{(\mathbb {R}),\mathbb {R}}}))^*$. Similarly, denote by $\mathcal {D}'(X(\mathbb {R})):=\mathcal {D}'(X(\mathbb {R}),\mathbb {C})$ the space of complex valued distributions on $X(\mathbb {R})$ and by $\mathcal {S}^*(X(\mathbb {R})):=\mathcal {S}^*(X(\mathbb {R}),\mathbb {C})$ the space of complex valued tempered distributions on $X(\mathbb {R})$. Also, for an algebraic bundle $\mathcal {E}$ (complex or real) over X, we denote $\mathcal {D}'(X{{(\mathbb {R})}},\mathcal {E}):=(C_c^{\infty }(X{{(\mathbb {R})}},\mathcal {E}))^*$ and $\mathcal {S}^*(X{{(\mathbb {R})}},\mathcal {E}):=(\mathcal {S}(X{{(\mathbb {R})}},\mathcal {E}))^*$.

The algebra D(X) acts on the spaces $\mathcal {D}'(X{{(\mathbb {R}),\mathbb {R}}})$ and $\mathcal {S}^*(X{{(\mathbb {R}),\mathbb {R}}})$. Thus, for affine X we can consider these spaces as $D_{X}$-modules and $\mathcal {D}'(X{{(\mathbb {R}),\mathbb {R}}})$ also as an analytic $D_X$-module. Similarly, we will regard the spaces $\mathcal {D}'(X(\mathbb {R}))$ and $\mathcal {S}^*(X(\mathbb {R}))$ as $D_{X_{\mathbb {C}}}$-modules, where $X_{\mathbb {C}}$ denotes the complexification of X. For non-affine X, define the $D_X$-module $\mathcal {S}^*_{X,\mathbb {R}}$ by $\mathcal {S}^*_{X,\mathbb {R}}(U):=\mathcal {S}^*(U(\mathbb {R}),\mathbb {R})$. It is easy to see that this sheaf is quasi-coherent. Denote by $\mathcal {S}^*_{X}$ the $D_{X_{\mathbb {C}}}$-module obtained by complexification of $\mathcal {S}^*_{X,\mathbb {R}}$.

We define the singular support of a tempered distribution to be the singular support of the D-module it generates. It is well known that this definition is equivalent to Definition 1.3. We say that a distribution is holonomic if it generates a holonomic D-module.

Lemma 3.1

(See Appendix A). Let $i:X \rightarrow Y$ be a closed embedding of smooth affine algebraic varieties defined over $\mathbb {R}$. Then

$$\begin{aligned} \mathcal {D}'(X{{(}}\mathbb {R}))\cong i^!(\mathcal {D}'(Y{{(}}\mathbb {R}))) \text { and }\mathcal {S}^*(X{{(}}\mathbb {R}))\cong i^!(\mathcal {S}^*(Y{{(}}\mathbb {R}))). \end{aligned}$$

Lemma 3.2

Let M be a smooth ${{D(\mathbb {A}^n_{\mathbb {C}} )}}$-module of rank r. Embed the space $\mathrm {An}(\mathbb {C}^n)$ of analytic functions on $\mathbb {C}^n$ into $\mathcal {D}'(\mathbb {R}^n)$ using the Lebesgue measure. Then

$$\begin{aligned} {\begin{matrix} {{\mathrm{Hom}}}_{{{D(\mathbb {A}^n_{\mathbb {C}} )}}}(M,\mathcal {D}'(\mathbb {R}^n)) &{}={{\mathrm{Hom}}}_{{{D(\mathbb {A}^n_{\mathbb {C}} )}}}(M,\mathrm {An}(\mathbb {C}^n))\qquad \text {and }\\ r &{}= \dim {{\mathrm{Hom}}}_{{{D(\mathbb {A}^n_{\mathbb {C}} )}}}(M,\mathcal {D}'(\mathbb {R}^n)). \end{matrix}} \end{aligned}$$

Proof

Let $M_{\mathrm {An}}:=M \otimes _{\mathcal {O}(\mathbb {C}^n)}\mathrm {An}(\mathbb {C}^n)$ and ${{D_{\mathrm {An}}(\mathbb {A}^n_{\mathbb {C}} )}}:={{D(\mathbb {A}^n_{\mathbb {C}} )}} \otimes _{\mathcal {O}(\mathbb {C}^n)}\mathrm {An}(\mathbb {C}^n)$ be the analytizations of M and ${{D(\mathbb {A}^n_{\mathbb {C}} )}}$. Then

$$\begin{aligned}{{\mathrm{Hom}}}_{{{D(\mathbb {A}^n_{\mathbb {C}} )}}}(M,\mathcal {D}'(\mathbb {R}^n)) \cong {{\mathrm{Hom}}}_{{{D_{\mathrm {An}}(\mathbb {A}^n_{\mathbb {C}} )}}}(M_{\mathrm {An}},\mathcal {D}'(\mathbb {R}^n)).\end{aligned}$$

Since $M_{\mathrm {An}}$ is also smooth, $M_{\mathrm {An}} \cong \mathrm {An}(\mathbb {C}^n)^r$. Thus it is left to prove that

$$\begin{aligned}{{\mathrm{Hom}}}_{{{D_{\mathrm {An}}(\mathbb {A}^n_{\mathbb {C}} )}}}(\mathrm {An}(\mathbb {C}^n),\mathcal {D}'(\mathbb {R}^n))={{\mathrm{Hom}}}_{{{D_{\mathrm {An}}(\mathbb {A}^n_{\mathbb {C}} )}}}(\mathrm {An}(\mathbb {C}^n),\mathrm {An}(\mathbb {C}^n))\end{aligned}$$

and the latter space is one-dimensional. This follows from the fact that a distribution with vanishing partial derivatives is a multiple of the Lebesgue measure.$\square$

Corollary 3.3

If a distribution generates a smooth D-module, then it is analytic.

3.1.3 Lie algebra actions

Definition 3.4

Let X be an algebraic manifold defined over a field k and $\mathfrak {g}$ a Lie algebra over k.

• (i) An action of $\mathfrak {g}$ on X is a Lie algebra map from $\mathfrak {g}$ to the algebra of algebraic vector fields on X.

• (ii) Assume that X is affine, fix an action of $\mathfrak {g}$ on X and let $\mathcal {E}$ be an algebraic vector bundle on X. Let M be the space of global regular (algebraic) sections of $\mathcal {E}$. An action of $\mathfrak {g}$ on $\mathcal {E}$ is a linear map $T:\mathfrak {g}\rightarrow {{\mathrm{End}}}_{k}(M)$ such that, for any ${\alpha }\in \mathfrak {g}, \, f\in \mathcal {O}(X), \, v \in M$, we have

  $$\begin{aligned} T({\alpha })(fv)=({\alpha }f) v+fT({\alpha })v. \end{aligned}$$

• (iii) The definition above extends to non-affine X in a straightforward way.

3.1.4 Weil representation

Definition 3.5

Let V be a finite-dimensional real vector space. Let $\omega$ be the standard symplectic form on $V \oplus V^*$. Denote by $p_V:V \oplus V^* \rightarrow V$ and $p_{V^*}:V \oplus V^*\rightarrow V^*$ the natural projections. Define an action of the symplectic group ${{{{\mathrm{Sp}}}(V \oplus V^*)}}$ on the algebra D(V) by

$$\begin{aligned} {\begin{matrix} (\partial _v)^g &{}:=\pi (g)(\partial _v):=p_{V^*}(g(v,0)) + \partial _{p_{V}(g(v,0))}, \\ w^g &{}:=\pi (g)w:=p_{V^*}(g(0,w)) + \partial _{p_{V}(g(0,w))} \end{matrix}} \end{aligned}$$

where $v\in V, \, w\in V^*, \, \partial _v$ denotes the derivative in the direction of v, and elements of $V^*$ are viewed as linear polynomials and thus differential operators of order zero. For a D(V)-module M and an element $g\in {{\mathrm{Sp}}}(V \oplus V^*)$, we will denote by $M^g$ the D(V)-module obtained by twisting the action of D(V) by $\pi (g)$.

Since the above action of ${{\mathrm{Sp}}}(V \oplus V^*)$ preserves the Bernstein filtration on D(V), the following lemma holds.

Lemma 3.6

For a finitely generated D(V)-module M and $g\in {{\mathrm{Sp}}}(V \oplus V^*)$ we have $SS_{b}(M^g)=gSS_{b}(M)$.

Theorem 3.7

([33]). There exists a two-folded cover $p\!:\!\widetilde{{{\mathrm{Sp}}}}(V \oplus V^*)\rightarrow {{\mathrm{Sp}}}(V \oplus V^*){{(}}\mathbb {R})$ and a representation $\Pi$ of $\widetilde{{{\mathrm{Sp}}}}(V \oplus V^*)$ on the space $\mathcal {S}^*(V)$ of tempered distributions on V such that, for any $\alpha \in D(V)$, $g\in \widetilde{{{\mathrm{Sp}}}}(V \oplus V^*)$, $\xi \in \mathcal {S}^*(V)$, we have

$$\begin{aligned}\Pi (g)(\xi \alpha )=(\Pi (g)\xi )\alpha ^{p(g)}.\end{aligned}$$

Corollary 3.8

We have an isomorphism of D(V)-modules $\mathcal {S}^*(V)^g \cong \mathcal {S}^*(V)$ for any $g\in {{\mathrm{Sp}}}(V \oplus V^*){{(}}\mathbb {R})$.

In fact, this corollary can be derived directly from the Stone–von Neumann theorem.

3.1.5 Flat morphisms

Lemma 3.9

Let $\phi :X \rightarrow Y$ be a proper morphism of algebraic varieties defined over a field k and $\mathcal {M}$ be a coherent sheaf on X. Then there exists an open dense $U\subset Y$ such that $\mathcal {M}|_{\phi ^{-1}(U)}$ is flat over U.

Proof

By [15, Théorème II.3.I], the set V of scheme-theoretic points $x\in X$ for which $\mathcal {M}$ is $\phi$-flat at x is open in X. Since $\phi$ is proper, the set $Z:=\phi (X{\setminus }{V})$ is closed in Y. Note that $\mathcal {M}$ is flat over $U:=X{\setminus }{Z}$, since $\phi ^{-1}(U)\subset V$. Moreover, U contains the generic points of the irreducible components of Y. Hence $U\subset Y$ is dense. $\square$

Lemma 3.10

(See, e.g., [28, Corollary on p. 50]). Let $\phi :X \rightarrow Y$ be a proper morphism of algebraic varieties defined over a field k and, $\mathcal {M}$ be a coherent sheaf on X that is flat over Y. For a point $y\in Y$, let $\mathcal {M}_y$ denote the pullback of $\mathcal {M}$ to $\phi ^{-1}(y)$. Then the function

$$\begin{aligned} y \mapsto \chi (\mathcal {M}_y)=\sum _{i=0}^{\infty }(-1)^i\dim _{k(y)} {\text {H}}^i(\mathcal {M}_y) \end{aligned}$$

is locally constant.

Corollary 3.11

Let Y be an algebraic variety defined over a field k and let $\mathcal {M}$ be a coherent sheaf on $Y \times \mathbb {P}^n$. Then there exists an open dense $U\subset Y$ such that the Hilbert polynomial³ of $\mathcal {M}_y$ does not depend on y as long as $y\in U$.

3.2 Dimension of the space of solutions of a holonomic system

Definition 3.12

Let k be a field of characteristic zero.

• (i) Let M be a finitely generated D-module over an affine space $\mathbb {A}_k^n$ over k. Let $F^i$ be a good filtration on M with respect to the Bernstein filtration on the ring $D_{{{\mathbb {A}_k^n}}}$. Let p be the corresponding Hilbert polynomial of M, i.e., $p(i)=\dim F^i$ for large enough i. Let ${{d}}$ be the degree of p and $a_{{{d}}}$ be the leading coefficient of p. Define the Bernstein degree of M to be $\deg _b(M):={{d}}!a_{{{d}}}$. It is well-known that ${{d}}$ and $a_{{{d}}}$ do not depend on the choice of a good filtration $F^i$.

• (ii) Let M be a finitely generated D-module over a smooth algebraic variety X defined over k. Let $X=\bigcup _{i=1}^l U_i$ be an open affine cover of X and let $\phi _i:U_i \hookrightarrow {{\mathbb {A}_k^{n_i}}}$ be closed embeddings. Denote

  $$\begin{aligned} \deg _{\{(U_i,\phi _i)\}}(M):=\sum _{i=1}^l \deg _b((\phi _i)_*(M{{|_{U_{i}}}})). \end{aligned}$$

  Define the global degree of M by $\deg (M):=\min \deg _{\{(U_i,\phi _i)\}}(M)$, where the minimum is taken over the set of all possible affine covers and embeddings.

In this subsection, we prove

Theorem 3.13

Let X be an algebraic manifold defined over $\mathbb {R}$. Let M be a holonomic right $D_{{{X_{\mathbb {C}}}}}$-module. Then $\dim {{\mathrm{Hom}}}(M,\mathcal {S}^*(X)) \le \deg (M)$.

We will need the following geometric lemmas.

Lemma 3.14

Let V be a vector space over an algebraically closed field, let $L\subset V$ be a subspace and $C\subset V$ be a closed conic algebraic subvariety such that $L\cap C = \{0\}$. Then the projection $p:C \rightarrow V/L$ is a finite map.

Proof

By induction, it is enough to prove the case $\dim L =1$. Choose coordinates $x_1,\dots ,x_n$ on V such that the coordinates $x_1,\dots ,x_{n-1}$ vanish on L. Let p be a homogeneous polynomial that vanishes on C but not on L. Write $p=\sum _{i=1}^{{{d}}} g_ix_n^i$, where each $g_i$ is a homogeneous polynomial of degree ${{d}}-i$ in $x_1,\dots ,x_{n-1}$. Then $x_n|_C$ satisfies a monic polynomial equation with coefficients in $\mathcal {O}(V/L)$.$\square$

Lemma 3.15

Let W be a 2n-dimensional real symplectic vector space, and $C\subset {{W_{\mathbb {C}}}}$ be a closed conic subvariety of dimension n. Then there exists a real Lagrangian subspace $L\subset W$ such that ${{L_{\mathbb {C}}\cap C}} = \{0\}$.

Proof

Let $\mathcal {L}$ denote the variety of all Lagrangian subspaces of W. Note that $\dim \mathcal {L}= n(n+1)/2$. Let $P(C)\subset \mathbb {P}(W_{{{\mathbb {C}}}})$ be the projectivizations of C and $W_{\mathbb {C}}$. Consider the configuration space

$$\begin{aligned} X:=\{(x,L)\in P(C) \times \mathcal {L}\, \vert \, x \subset L\}. \end{aligned}$$

Since $\mathcal {L}$ is smooth, irreducible, and has a real point, it is enough to show that ${{\overline{p(X)}}} \ne \mathcal {L}$ where $p:X \rightarrow \mathcal {L}$ is the projection. Let $q:X\rightarrow P(C)$ be the other projection. Note that $\dim q^{-1}(x)=n(n-1)/2$ for any $x \in P(C)$. Thus

$$\begin{aligned} \dim X = n(n-1)/2+n-1<n(n+1)/2=\dim \mathcal {L}, \end{aligned}$$

and thus $p:X \rightarrow \mathcal {L}_{{{\mathbb {C}}}}$ cannot be dominant.$\square$

Corollary 3.16

Let V be a real vector space of dimension n. Consider the standard symplectic form on $V\oplus V^*$. Let $C\subset V_{{{\mathbb {C}}}}\oplus V_{{{\mathbb {C}}}}^*$ be a closed conic subvariety of dimension n, defined over $\mathbb {R}$. Let $p:V_{{{\mathbb {C}}}}\oplus V_{{{\mathbb {C}}}}^*\rightarrow V_{{{\mathbb {C}}}}$ denote the projection. Then there exists a linear symplectic automorphism $g \in {{\mathrm{Sp}}}(V\oplus V^*){{(\mathbb {R})}}$ such that $p|_{gC}$ is a finite map.

Proof

By Lemma 3.15 there exists a Lagrangian subspace $L\subset V \oplus V^*$ such that ${{L_{\mathbb {C}}\cap C}} = \{0\}$. Since the action of ${{\mathrm{Sp}}}(V\oplus V^*){{(\mathbb {R})}}$ on Lagrangian subspaces is transitive, there exists $g \in {{\mathrm{Sp}}}(V\oplus V^*){{(\mathbb {R})}}$ such that $V^*=gL$ and thus ${{gC\cap V_{\mathbb {C}}^*}}=\{0\}$. From Lemma 3.14 we get that $p|_{gC}$ is a finite map.

$\square$

Proof of Theorem 3.13

Let $X=\bigcup _{i=1}^l U_i$ be an open affine cover of X and let $\phi _i:U_i \hookrightarrow \mathbb {A}_{{{\mathbb {C}}}}^{n_i}$ be closed embeddings. Clearly

$$\begin{aligned} \dim {{\mathrm{Hom}}}(M,\mathcal {S}^*_{{{X}}})\le \sum _{i=1}^l \dim {{\mathrm{Hom}}}(M|_{{{(U_i)_{\mathbb {C}}}}},\mathcal {S}^*(U_i{{(\mathbb {R}))}}). \end{aligned}$$

By Lemma 3.1

$$\begin{aligned} {{\mathrm{Hom}}}(M|_{{{(U_i)_{\mathbb {C}}}}},\mathcal {S}^*(U_i{{(\mathbb {R})}}))&\cong {{\mathrm{Hom}}}(M|_{{{(U_i)_{\mathbb {C}}}}},\phi _i^!(\mathcal {S}^*(\mathbb {R}^{n_i}))\\&\cong {{\mathrm{Hom}}}((\phi _i)_*(M|_{{{(U_i)_{\mathbb {C}}}}}),\mathcal {S}^*(\mathbb {R}^{n_i})). \end{aligned}$$

Thus it is enough to show that for any holonomic D-module N on an affine space $\mathbb {A}^n_{{{\mathbb {C}}}}$ we have

$$\begin{aligned} \dim {{\mathrm{Hom}}}(N,\mathcal {S}^*(\mathbb {R}^{n})) \le \deg _b(N). \end{aligned}$$

Let $C\subset \mathbb {A}^{2n}_{{{\mathbb {C}}}}$ be the singular support of N with respect to the Bernstein filtration. By Corollary 3.16, there exists $g \in {{\mathrm{Sp}}}_{2n}{{(\mathbb {R})}}$ such that $p|_{gC}$ is a finite map, where $p:\mathbb {A}_{{{\mathbb {C}}}}^{2n}\rightarrow \mathbb {A}_{{{\mathbb {C}}}}^n$ is the projection on the first n coordinates. By Corollary 3.8 we have

$$\begin{aligned} \dim {{\mathrm{Hom}}}(N,\mathcal {S}^*(\mathbb {R}^n)) = \dim {{\mathrm{Hom}}}(N^g,\mathcal {S}^*(\mathbb {R}^n)^g)=\dim {{\mathrm{Hom}}}(N^g,\mathcal {S}^*(\mathbb {R}^n)). \end{aligned}$$

By Lemma 3.6 we have $SS_b(N^g)=gC$. Let F be a good filtration on $N^g$ (with respect to the Bernstein filtration on $D(\mathbb {A}_{{{\mathbb {C}}}}^n)$). We see that ${{\mathrm{Gr}}}N^g$ is finitely generated over $\mathcal {O}(\mathbb {A}_{{{\mathbb {C}}}}^n)$, and thus so is $N^g$. Thus $N^g$ is a smooth D-module. Note that ${{\mathrm{rk}}}_{\mathcal {O}(\mathbb {A}^n)} N^g \le \deg _b N^g=\deg _b N$. By Lemma 3.2 $\dim {{\mathrm{Hom}}}(N^g,\mathcal {S}^*(\mathbb {R}^n)) \le {{\mathrm{rk}}}_{\mathcal {O}(\mathbb {A}_{{{\mathbb {C}}}}^n)} N^g$. $\square$

3.3 Families of D-modules

In this section we discuss families of D-modules on algebraic varieties over an arbitrary field k of zero characteristic.

Let $\phi :X \rightarrow Y$ be a map of algebraic varieties and $\mathcal {M}$ a quasi-coherent sheaf of $\mathcal {O}_X$-modules. For any $y\in Y$, denote by $\mathcal {M}_y$ the pullback of $\mathcal {M}$ to $\phi ^{-1}(y)$.

Definition 3.17

Let X, Y be smooth algebraic varieties.

• If X and Y are affine, we define the algebra D(X, Y) to be $D(X)\otimes _k \mathcal {O}(Y)$.

• Extending this definition we obtain a sheaf of algebras $D_{X,Y}$ on $X \times Y$.

• By a family of $D_X$-modules parameterized by Y, we mean a sheaf of right modules over the sheaf of algebras $D_{X,Y}$ on $X \times Y$ which is quasi-coherent as a sheaf of $\mathcal {O}_{X\times Y}$-modules.

• We call a family of $D_X$-modules parameterized by Y coherent if it is locally finitely generated as a $D_{X,Y}$-module.

• For a family $\mathcal {M}$ of $D_X$-modules parameterized by Y and a point $y\in Y$, we call $\mathcal {M}_y$ the specialization of $\mathcal {M}$ at y and consider it with the natural structure of a $D_X$-module.

• We say that a coherent family $\mathcal {M}$ is holonomic if every specialization is holonomic.

Theorem 3.18

Let X, Y be smooth algebraic varieties and let $\mathcal {M}$ be a family of $D_X$-modules parametrized by Y. Then $\deg \mathcal {M}_y$ is bounded when y ranges over the k-points of Y.

Proof

Without loss of generality, we can assume that $X=\mathbb {A}^n$ and Y is an affine variety, and prove that $\deg _b(\mathcal {M}_y)$ is bounded. We will prove this by induction on $\dim Y$.

The Bernstein filtration on $D(\mathbb {A}^n)$ gives rise to a filtration on $D(\mathbb {A}^n,Y)$. Choose a filtration F on $\mathcal {M}$ which is good with respect to this filtration and let $N:={{\mathrm{Gr}}}{\mathcal {M}}$, considered as a graded $\mathcal {O}(\mathbb {A}^{2n}\times Y)$-module. Associate to N a coherent sheaf $\mathcal {N}$ on $\mathbb {P}^{2n-1}\times Y$. Let $\mathcal {N}_y$ be the pullback of $\mathcal {N}$ under the embedding of $\mathbb {P}^{2n-1}$ into $\mathbb {P}^{2n-1}\times Y$ given by $x \mapsto (x,y)$. By definition, the Hilbert polynomial of $\mathcal {M}_y$ with respect to the filtration induced by F is the Hilbert polynomial of $\mathcal {N}_y$. By Corollary 3.11, there exists an open dense subset $U \subset Y$ such that the Hilbert polynomial of $\mathcal {N}_y$ does not depend on y as long as $y\in U$. By the induction hypothesis, $\deg _b(\mathcal {M}_y)$ is bounded on $Y {\setminus }U$, and thus bounded on Y. $\square$

For an application of this theorem we will need the following lemma.

Lemma 3.19

Let a real Lie algebra $\mathfrak {g}$ act on an algebraic manifold X defined over $\mathbb {R}$ and on a complex algebraic vector bundle $\mathcal {E}$ on X. Fix a natural number n and let Y be the variety of all representations of $\mathfrak {g}$ on $\mathbb {C}^n$. Then there exists a coherent family $\mathcal {M}$ of $D_X$-modules parameterized by Y such that, for any $\tau \in Y$, we have

• (1) ${{\mathrm{Hom}}}_{\mathfrak {g}}(\tau ,\mathcal {S}^*(X(\mathbb {R}),\mathcal {E}))={{\mathrm{Hom}}}_{D_X}(\mathcal {M}_{\tau },\mathcal {S}^*_X).$

• (2) The singular support of $\mathcal {M}_{\tau }$ (with respect to the geometric filtration) is included in

  $$\begin{aligned} \{(x,\phi ) \in T^*X \, \vert \, \forall \alpha \in \mathfrak {g}\text { we have } \langle \phi , \alpha (x)\rangle =0 \}. \end{aligned}$$

Proof

It is enough to prove the lemma for affine X. Let N be the coherent sheaf of the regular (algebraic) sections of $\mathcal {E}$ (considered as a sheaf of $\mathcal {O}_{{{X_{\mathbb {C}}}}}$-modules). Let $\mathcal {N}$ be the pullback of N to ${{X_{\mathbb {C}}\times Y}}$. Let $\mathcal {N}':=\mathcal {N}\otimes _{\mathcal {O}_{{{X_{\mathbb {C}}\times Y}}}}D_{{{X_{\mathbb {C}}, Y}}}\otimes _{\mathbb {C}}\mathbb {C}^n$, and $\mathcal {N}''\subset \mathcal {N}'$ be the $D_{{{X_{\mathbb {C}}, Y}}}$-submodule generated by elements of the form

$$\begin{aligned} \alpha n \otimes 1 \otimes v+ n \otimes \xi _{\alpha } \otimes v + n\otimes f_{\alpha }(v), \end{aligned}$$

where $\alpha \in \mathfrak {g}$, $\xi _{\alpha }$ is the vector field on X corresponding to $\alpha$, and $f_{\alpha }(v)\in D_{{{X_{\mathbb {C}}, Y_{\mathbb {C}}}}}\otimes _{\mathbb {C}}\mathbb {C}^n$ is the $\mathbb {C}^n$-valued regular function on ${{X_{\mathbb {C}}\times Y}}$ given by $f_{\alpha }(v)(x,\tau )=\tau (\alpha )v$. Then $\mathcal {M}:=\mathcal {N}'/\mathcal {N}''$ satisfies the requirements. $\square$

Theorems 3.13, 3.18, and Lemma 3.19 imply Theorem D.

4 Proof of Theorems A and E

In this section, we derive Theorems A and E from Theorem B and § 3. We do that by embedding the multiplicity space into a certain space of relative characters.

4.1 Preliminaries

For a reductive group G defined over $\mathbb {R}$, we denote by ${{\mathrm{Irr}}}(G{{(\mathbb {R})}})$ the collection of irreducible admissible smooth Fréchet  representations of $G{{(\mathbb {R})}}$ of moderate growth. We refer to [11, 32] for the background on these representations.

Theorem 4.1

(See [32, Theorem 4.2.1]) The center $\mathfrak {z}(\mathcal {U}(\mathfrak {g}))$ of the universal enveloping algebra of the complexified Lie algebra of G acts finitely on every admissible smooth Fréchet  representation $\pi$ of G of moderate growth. This means that there exists an ideal in $\mathfrak {z}(\mathcal {U}(\mathfrak {g}))$ of finite codimension that annihilates $\pi$.

Lemma 4.2

([1, Theorem 1.2 and Corollary 1.4]) For any reductive group G defined over $\mathbb {R}$, there exists an involution $\theta$ of G such that, for any $\pi \in {{\mathrm{Irr}}}(G{{(\mathbb {R})}})$, we have $\hat{\pi }\cong \pi ^{\theta }$.

Theorem 4.3

(Casselman embedding theorem, see [12, Proposition 8.23]) Let G be a reductive group G defined over $\mathbb {R}$ and P be a minimal parabolic subgroup of G. Let $\pi \in {{\mathrm{Irr}}}(G{{(\mathbb {R})}})$. Then there exist a finite-dimensional representation $\sigma$ of P and an epimorphism $\mathrm {Ind}_{P{{(\mathbb {R})}}}^{G{{(\mathbb {R})}}}(\sigma ) \twoheadrightarrow \pi$.

4.2 Proof of Theorem A and Proposition 1.4

Theorem A follows from Theorem B and Proposition 1.4.

Proof of Proposition 1.4

Let $\xi$ be a relative character of a smooth admissible Fréchet  representation $\pi$ of $G{{(\mathbb {R})}}$ of moderate growth with respect to a pair of subgroups $(H_1{{(\mathbb {R})}},H_2{{(\mathbb {R})}})$ and their characters $\chi _1,\chi _2$. By Theorem 4.1, there exists an ideal $I \subset \mathfrak {z}(\mathcal {U}(\mathfrak {g}))$ of finite codimension that annihilates $\pi$ and thus annihilates $\xi$. For any element $z \in \mathfrak {z}(\mathcal {U}(\mathfrak {g}))$, there exists a polynomial p such that $p(z)\in I$ and thus $p(z)\xi =0$. This implies that the symbol of any $z \in \mathfrak {z}(\mathcal {U}(\mathfrak {g}))$ of positive degree vanishes on the singular support of $\xi$. It is well-known that the joint zero-set of these symbols over each point $g\in G$ is the nilpotent cone $\mathcal {N}(\mathfrak {g}^*)$. Since $\xi$ is $({{\mathfrak {h}}}_1\times {{\mathfrak {h}}}_2, \chi _1\times \chi _2)$-equivariant, this implies that the singular support of $\xi$ lies in S. $\square$

4.3 Proof of Theorem E

Part (i) follows immediately from Theorem D and the Casselman embedding theorem. If G is quasisplit, then so does part (ii). For the proof of part (ii) in the general case, we will need the following lemma.

Lemma 4.4

Let G be a reductive group defined over $\mathbb {R}$ and let $H_1,H_2$ be spherical subgroups. Let $Y=\mathrm {Spec}(\mathfrak {z}(\mathcal {U}(\mathfrak {g}))){{\times Y_1\times Y_2}}$, where $Y_i$ is the variety of characters of $\mathfrak {h}_i=\mathrm {Lie\,}H_i$. For any $\lambda \in Y(\mathbb {C})$, define $U_{\lambda ,{{\chi _1,\chi _2}}}:=\mathcal {S}^*(G{{(\mathbb {R})}})^{{{\mathfrak {h}_1\times \mathfrak {h}_2,(\chi _1,\chi _2)}},(\mathfrak {z}(\mathcal {U}(\mathfrak {g})),\lambda )}$ to be the space of tempered distributions on G that are left $\chi _1$-equivariant with respect to $\mathfrak {h}_1$, right $\chi _2$-equivariant with respect to $\mathfrak {h}_2$ and are eigendistributions with respect to the action of $\mathfrak {z}(\mathcal {U}(\mathfrak {g}))$ with eigencharacter $\lambda$. Then $\dim U_{\lambda ,\chi _1,\chi _2}$ is bounded over $Y(\mathbb {C})$.

Proof

Let us construct a family of D(G)-modules $\mathcal {M}$ parameterized by Y. For any $\alpha \in \mathfrak {g}$, let $r_\alpha$ and $l_\alpha$ be the corresponding right and left invariant vector fields on G considered as elements in D(G, Y). For any $\beta \in \mathfrak {z}(\mathcal {U}(\mathfrak {g}))$, $\alpha _i\in \mathfrak {h}_i$, let $f_\beta ,g^i_{\alpha _i}$ be the functions on Y that send $(\mu ,\gamma _1,\gamma _2)\in Y$ to $\mu (\beta ), \gamma _i(\alpha _i)$ respectively. Also let $d_\beta$ be the differential operator on G corresponding to $\beta$, such that $d_{\beta }\xi =\beta \xi$ for any distribution $\xi$ on $G{{(\mathbb {R})}}$. We consider $d_\beta ,r_{\alpha _1},l_{\alpha _2},f_\beta ,g^{i}_{\alpha _i}$ as elements of D(G, Y). Let $I \subset D(G,Y)$ be the ideal generated by $r_{\alpha _1}{{-g^{1}_{\alpha _1}}}$, $l_{\alpha _2}{{-g^{2}_{\alpha _2}}}$ and $f_\beta {{-}} d_\beta$ where $\alpha _i\in \mathfrak {h}_i$ and $\beta \in \mathfrak {z}(\mathcal {U}(\mathfrak {g}))$. Define $\mathcal {M}:=D(G,Y)/I$.

It is easy to see that $U_{\lambda ,\chi _1,\chi _2} \cong {{\mathrm{Hom}}}(\mathcal {M}_{(\lambda ,\chi _1,\chi _2)},\mathcal {S}^*(G{{(\mathbb {R})}}))$. As in the proof of Proposition 1.4, the singular support of $\mathcal {M}_{(\lambda ,\chi _1,\chi _2)}$ lies in S, for any $\lambda ,\chi _1,\chi _2$. By Theorem B, $\mathcal {M}_{(\lambda ,\chi _1,\chi _2)}$ is holonomic and, therefore, $\mathcal {M}$ is holonomic. By Theorem 3.13, we have $\dim U_{\lambda ,\chi _1,\chi _2} \le \deg \mathcal {M}_{(\lambda ,\chi _1,\chi _2)}$. By Theorem 3.18, $\deg \mathcal {M}_{(\lambda ,\chi _1,\chi _2)}$ are bounded. $\square$

Proof of Theorem E(ii)

We choose an involution $\theta$ as in Lemma 4.2, let $H_1:=H, \, H_2:=\theta (H)$, and define the spaces $U_{\lambda }$ as in Lemma 4.4.

Now let $\pi \in {{\mathrm{Irr}}}(G{{(\mathbb {R})}})$ and let $\chi$ be a character of $\mathfrak {h}$ such that $(\pi ^*)^{\mathfrak {h},\chi }\ne 0$. Let ${\lambda }$ stand for the infinitesimal character of $\pi$. By Lemma 4.2, $(\hat{\pi }^*)^{d\theta (\mathfrak {h}),{{d\theta (\chi )}}}\ne 0$. Fix a non-zero $\phi \in (\hat{\pi }^*)^{{d\theta (\mathfrak {h}),{{d\theta (\chi )}}}}$. Then $\phi$ defines an embedding $(\pi ^*)^{\mathfrak {h},{{\chi }}} \hookrightarrow U_{\lambda ,{{\chi ,d\theta (\chi )}}}$ by $\psi \mapsto \xi _{\psi ,{\phi }}$, where $\xi _{\psi ,{\phi }}$ is the relative character, which is defined by $\xi _{\psi ,{\phi }}(f):=\langle \psi , \pi (f) \phi \rangle$. Thus, $\dim (\pi ^*)^{\mathfrak {h},{{\chi }}} \le \dim U_{\lambda ,{{\chi ,d\theta (\chi )}}}$, which is bounded by Lemma 4.4. $\square$

Appendix A: Proof of Lemma 3.1

For the proof, we will need the following standard lemmas. Let M be a smooth manifold and $N \subset M$ be a closed smooth submanifold.

Lemma A.1

Denote $I_N:=\{f \in \mathrm {C}_c^{\infty }(M) \, \vert f|_N=0\}$. Let $J\subset I_N$ be an ideal in $\mathrm {C}_c^{\infty }(M)$ such that

• (1) For any $x\in N$, the space $\{d_xf\,|\,f\in J\}$ is the conormal space to N in M at the point x.

• (2) For any $x \in M {\setminus } N$, there exists $f \in J$ such that $f(x)\ne 0$.

Then $J=I_N$.

Proof

Using the partition of unity, it is enough to show that for any $f\in I_N$ and $x \in M$, there exists $f' \in J$ such that f coincides with $f'$ in a neighborhood of x. For $x \notin N$ this is obvious, so we assume that $x \in N$. We prove the statement by induction on the codimension d of N in M. The base case $d=1$ follows, using the implicit function theorem, from the case $N=\mathbb {R}^{n-1}\subset \mathbb {R}^{n}=M$, which is obvious.

For the induction step, take an element $g\in J$ such that $d_xg\ne 0$. Let

$$\begin{aligned} Z:=\{y \in M \, \vert \, g(y)=0\}\quad \text { and }\quad U:=\{y\in M \, \vert \,d_yg\ne 0 \}. \end{aligned}$$

By the implicit function theorem, $U \cap Z$ is a closed submanifold of U. Choose $\rho \in \mathrm {C}_c^{\infty }(M)$ such that $\rho =1$ in a neighborhood of x and ${{\mathrm{Supp}}}(\rho ) \subset U$. Let $\bar{f}:=(\rho f)|_{U \cap Z}$. Let

$$\begin{aligned} \bar{J} := \{\alpha |_{U \cap Z} \vert \alpha \in J \text { and } {{\mathrm{Supp}}}\alpha \subset U\}. \end{aligned}$$

By the induction hypothesis, $\bar{f} \in \bar{J}.$ Thus, there exists $f'' \in J$ such that $f-f''$ vanishes in a neighborhood of x in Z. Now case $d=1$ implies that there exists $\alpha \in \mathrm {C}_c^{\infty }(M)$ such that $f-f''$ coincides with $\alpha g$ in a neighborhood of x. $\square$

Lemma A.2

The restriction $\mathrm {C}_c^{\infty }(M) \rightarrow \mathrm {C}_c^{\infty }(N)$ is an open map.

Proof

Let $K\subset M$ be a compact subset. It is easy to see that there exists a compact $K' \supset K$ such that the restriction map $\mathrm {C}^{\infty }_{K'}(M)\rightarrow \mathrm {C}^{\infty }_{K'\cap N}(N)$ is onto, using the partition of unity. By the Banach open map theorem this map is open. Thus, the restriction $\mathrm {C}_c^{\infty }(M) \rightarrow \mathrm {C}_c^{\infty }(N)$ is an open map. $\square$

Let Y be an affine algebraic manifold defined over $\mathbb {R}$ and X be a closed algebraic submanifold. Let $i:X \rightarrow Y$ denote the embedding.

Lemma A.3

Let $\xi$ be a distribution on $X{{(\mathbb {R})}}$ such that $i_*\xi$ is a tempered distribution. Then $\xi$ is a tempered distribution.

Proof

The map $i_*$ is dual to the pullback map $C_c^{\infty }(Y{{(\mathbb {R})}})\rightarrow C_c^{\infty }(X{{(\mathbb {R})}})$. This can be extended to a continuous map $i^*:\mathcal {S}(Y{{(\mathbb {R})}})\rightarrow \mathcal {S}(X{{(\mathbb {R})}})$ which is onto by [4, Theorem 4.6.1]. The Banach open map theorem implies that $i^*$ is an open map. It is easy to see that $i_{*}\xi :\mathcal {S}(Y{{(\mathbb {R})}})\rightarrow \mathbb {C}$ vanishes on $\mathrm {Ker}(i^*)$, and thus it gives rise to a continuous map $\mathcal {S}(X{{(\mathbb {R})}})\rightarrow \mathbb {C}$, which extends $\xi$. $\square$

Lemma A.4

Let $\xi$ be a complex valued distribution on $Y{{(\mathbb {R})}}$ such that $p\xi =0$ for any polynomial p on Y that vanishes on X. Then $\xi$ is a pushforward of a distribution on $X{{(\mathbb {R})}}$.

Proof

Let J(X) be the ideal of all polynomials on Y that vanish on X. Let $J :=J(X) \mathrm {C}_c^{\infty }(Y{{(\mathbb {R})}})$. By Lemma A.1 we have $J=I_{X{{(\mathbb {R})}}}$. Thus, $\xi$ vanishes on $I_{X{{(\mathbb {R})}}}$ and thus, by Lemma A.2, $\xi$ is a pushforward of a distribution on $X{{(\mathbb {R})}}$.

Lemma 3.1 follows from Lemmas A.3 and A.4 and the definition of $i^!$ for closed embedding of smooth affine varieties.