Rankin–Selberg local factors modulo $\ell$

Abstract

After extending the theory of Rankin–Selberg local factors to pairs of $\ell$-modular representations of Whittaker type, of general linear groups over a non-Archimedean local field, we study the reduction modulo $\ell$ of $\ell$-adic local factors and their relation to these $\ell$-modular local factors. While the $\ell$-modular local $\gamma$-factor we associate with such a pair turns out to always coincide with the reduction modulo $\ell$ of the $\ell$-adic $\gamma$-factor of any Whittaker lifts of this pair, the local L-factor exhibits a more interesting behaviour, always dividing the reduction modulo-$\ell$ of the $\ell$-adic L-factor of any Whittaker lifts, but with the possibility of a strict division occurring. We completely describe $\ell$-modular L-factors in the generic case and obtain two simple-to-state nice formulae: Let $\pi ,\pi '$ be generic $\ell$-modular representations; then, writing $\pi _b,\pi '_b$ for their banal parts, we have

$$\begin{aligned} L(X,\pi ,\pi ')=L(X,\pi _b,\pi _b'). \end{aligned}$$

Using this formula, we obtain the inductivity relations for local factors of generic representations. Secondly, we show that

$$\begin{aligned} L(X,\pi ,\pi ')=\mathop {\mathbf {GCD}}(r_{\ell }(L(X,\tau ,\tau '))), \end{aligned}$$

where the divisor is over all integral generic $\ell$-adic representations $\tau$ and $\tau '$ which contain $\pi$ and $\pi '$, respectively, as subquotients after reduction modulo $\ell$.

1 Introduction

Let F be a (locally compact) non-Archimedean local field of residual characteristic p and residual cardinality q, and let R be an algebraically closed field of characteristic $\ell$ prime to p or zero. In this article, following Jacquet–Piatetski-Shapiro–Shalika in [9] for complex representations, we associate local Rankin–Selberg integrals with pairs of R-representations of Whittaker type $\rho$ and $\rho '$ of ${\text {GL}}_n(F)$ and ${\text {GL}}_m(F)$ and show that they define L-factors $L(X,\rho ,\rho ')$ and satisfy a functional equation defining local $\gamma$-factors. The purpose of this article lies both in the future study of R-representations by these invariants and in the relationship between $\ell$-modular local factors and the reductions modulo $\ell$ of $\ell$-adic local factors (we quote our main theorems towards this goal at the end of this introduction).

The theory of $\ell$-modular representations of ${\text {GL}}_n(F)$ was developed by Vignéras in [20], culminating in her $\ell$-modular local Langlands correspondence for ${\text {GL}}_n(F)$ in [23], which is characterised initially on supercuspidal $\ell$-modular representations by compatibility with the $\ell$-adic local Langlands correspondence. The possibility of characterising such a correspondence with natural invariants forms part of the motivation for this work. Indeed, already for ${\text {GL}}_2(F)$ this is an interesting question, an answer given in this special case by Vignéras in [22].

We follow [9] to define local factors for R-representations of Whittaker type, after understanding the splitting of R-Haar measures required for this approach in Sect. 2.2. In Sect. 3, we show that an L-factor attached to $\ell$-adic representations of Whittaker type is equal to the inverse of a polynomial with coefficients in $\overline{\mathbb {Z}_{\ell }}$, allowing for a natural reduction modulo $\ell$ map on the set of $\ell$-adic L-factors. Furthermore, for $\ell$-modular representations $\pi$ and $\pi '$ of Whittaker type of ${\text {GL}}_n(F)$ and ${\text {GL}}_m(F)$, there exist $\ell$-adic representations $\tau$ and $\tau '$ of Whittaker type of ${\text {GL}}_n(F)$ and ${\text {GL}}_m(F)$ which stabilise natural $\overline{\mathbb {Z}_{\ell }}$-lattices $\Lambda$ and $\Lambda '$ in their respective Whittaker models such that the $\ell$-modular representations induced on $\Lambda \otimes _{\overline{\mathbb {Z}_{\ell }}}\overline{\mathbb {F}_{\ell }}$ and $\Lambda '\otimes _{\overline{\mathbb {Z}_{\ell }}}\overline{\mathbb {F}_{\ell }}$ are isomorphic to the Whittaker models of $\pi$ and $\pi '$. Our first main result is a comparison between the L-factors and local $\gamma$-factors defined by these two reduction modulo $\ell$ maps.

Theorem 3.13

• 1. The $\ell$-modular L-factor $L(X,\pi ,\pi ')$ divides the reduction modulo $\ell$ of the $\ell$-adic L-factor $L(X,\tau ,\tau ')$. Moreover, the division of L-factors may not be an equality.

• 2. Let $\theta$ be an $\ell$-adic character of F. The local $\gamma$-factor associated with $\pi ,\pi '$ and with the reduction modulo $\ell$ of $\theta$ is equal to the reduction modulo $\ell$ of the local $\gamma$-factor associated with $\tau ,\tau '$ and $\theta$.

Due to their nice behaviour with reduction modulo $\ell$, we obtain the inductivity relation of $\ell$-modular $\gamma$-factors of representations of Whittaker type (Theorem 4.1) by choosing appropriate lifts and reducing the $\ell$-adic inductivity relation of $\gamma$-factors, and from this we obtain a “division inductivity relation” for $\ell$-modular L-factors (Lemma 4.4). The remainder of the article is concerned with $\ell$-modular L-factors and their relation to $\ell$-adic L-factors. Our first main result in this direction is the complete answer for cuspidal $\ell$-modular representations.

Theorem 4.9

Let $\pi _1$ and $\pi _2$ be two cuspidal $\ell$-modular representations of $G_{n_1}$ and $G_{n_2}$. Then $L(X,\pi _1,\pi _2)$ is equal to 1, except in the following case: $\pi _1$ is banal (hence supercuspidal), and $\pi _2\simeq \chi \pi _1^\vee$ for some unramified character $\chi$ of $F^\times ~$ (in particular $n_1=n_2$). When  $\pi _1$ is banal, and $\pi _2\simeq \chi \pi _1^\vee$, let e be the common ramification index of $\pi _1$ and $\pi _2$ and we have

$$\begin{aligned} L(X,\pi _1,\pi _2)=\frac{1}{1-(\chi (\varpi _F)X)^{n/e}} \end{aligned}$$

and this factor is the reduction modulo $\ell$ of the L-factor of any cuspidal lifts of $\pi _1$ and $\pi _2$.

The proof of this theorem is diverse and uses the main result of [13] on test vectors for $\ell$-adic representations for the banal case, the division inductivity relation of L-factors for the cuspidal non-supercuspidal case, and a separate examination of the non-banal supercuspidal case by studying their lifts.

By restricting to pairs of banal generic representations, we obtain the inductivity relation of L-factors in this setting, and an explicit formula analogous to the $\ell$-adic case (Theorem 4.17). Banal $\ell$-modular representations were introduced and studied for inner forms of ${\text {GL}}_n$ by Mínguez and Sécherre in [16]. In our Preliminaries (Sect. 2.5), we show that a generic representation $\pi$ of ${\text {GL}}_n(F)$ can be written as a product of its nicely behaved banal part $\pi _b$ and its totally non-banal part $\pi _{tnb}$. Our third main result shows that for generic representations the L-factor depends only on the banal parts of the representations.

Theorem 4.19

Let $\pi =\pi _{b}\times \pi _{tnb}$ be a generic $\ell$-modular representation of ${\text {GL}}_n(F)$, and $\pi '=\pi _{b}'\times \pi _{tnb}'$ be a generic $\ell$-modular representation of ${\text {GL}}_{m}(F)$, then we have

$$\begin{aligned} L(X,\pi ,\pi ')= L(X,\pi _b,\pi _b'). \end{aligned}$$

From this theorem and the banal inductivity relation, we obtain the inductivity relation of L-factors for generic $\ell$-modular representations (Corollary 4.20).

Our fourth main result gives an equality between the local L-factor of a pair of generic $\ell$-modular representations we have defined via $\ell$-modular Rankin–Selberg integrals, and the greatest common divisor of the reductions modulo $\ell$ of certain $\ell$-adic L-factors.

Theorem 4.22

Let $\pi$ and $\pi '$ be two generic $\ell$-modular representations of ${\text {GL}}_n(F)$ and ${\text {GL}}_m(F)$, then

$$\begin{aligned} L(X,\pi ,\pi ')=\mathop {\mathbf {GCD}}(L(X,\tau ,\tau ')). \end{aligned}$$

where the divisor is over all integral generic $\ell$-adic representations $\tau$ of  ${\text {GL}}_n(F)$ and $\tau '$ of ${\text {GL}}_m(F)$ which contain $\pi$ and $\pi '$, respectively, as subquotients after reduction modulo $\ell$.

As we do not have an $\ell$-modular Langlands quotient theorem, which would allow us to associate with an irreducible representation of $G_n$, a unique representation with an injective Whittaker model lying above it, we restrict our attention in this article to representations of Whittaker type.

This work further develops the theory of $\ell$-modular local L-factors of Mínguez in [15]. In particular, we use his results on Tate L-factors modulo $\ell$. Recently, using our results from Sect. 2.2 on the splitting of Haar measures, Moss in [19] has studied Rankin–Selberg $\gamma$-factors in families over more general rings extending his earlier work on $\gamma$-factors for ${\text {GL}}_n\times {\text {GL}}_1$ convolutions [18].

2 Preliminaries

Before embarking on the study of local L-factors in positive characteristic, we introduce background on representations of the general linear group and extend the general theory. In particular, starting with results given in the standard reference [20], we show how integration behaves with respect to group decompositions. Indeed, this deserves checking as not all formulae follow from mimicking the proofs in the characteristic zero setting, due to the presence of compact open subgroups of measure zero. Additionally, in Sect. 2.5, we review the theory of $\ell$-adic and $\ell$-modular representations of Whittaker type and reduction modulo $\ell$, drawing on results originally in [21], while our exposition will be influenced by the recent generalisation to inner forms of general linear groups in [17]. In Theorem 2.26, we prove a technical result on lattices in representations of Whittaker type, in preparation for our results in the next section on the reduction modulo $\ell$ of $\ell$-adic local factors. In Sect. 2.6, we specialise to generic representations, and, in particular, notice that a generic $\ell$-modular representation has a banal part and a totally non-banal part, and prove the commutation of derivatives with reduction modulo $\ell$. Finally, in Sect. 2.7, we show that a non-banal supercuspidal $\ell$-modular representation has two lifts which are in different inertial classes.

2.1 Notations

Let F be a locally compact non-Archimedean local field of residual characteristic p with normalised absolute value | |. Let $\mathfrak {o}$ denote the ring of integers in F, $\mathfrak {p}=\varpi \mathfrak {o}$ the unique maximal ideal of $\mathfrak {o}$, and q the cardinality of $k=\mathfrak {o}/\mathfrak {p}$.

Let R be a commutative ring with identity of characteristic $\ell$ not equal to p. We suppose that R contains a square root of q, we fix such a choice.

Let

$$\begin{aligned} \mathcal {M}_{n,m}={\text {Mat}}(n,m,F),\quad \mathcal {M}_n={\text {Mat}}(n,n,F),\quad G_n={\text {GL}}_n(F), \end{aligned}$$

and $\eta$ be the row vector $(0,\ldots ,0,1) \in \mathcal {M}_{1,n}$. We write $\nu _R$ for the character $|~|\circ \det :G_n\rightarrow R^\times$, although we will often more simply write $\nu$ for $\nu _R$ when the coefficient ring is clear. For $k\in \mathbb {Z}$, let

$$\begin{aligned} G_n^k=\{g\in G_n: \nu _{\mathbb {C}}(g)=q^{-k}\}, \end{aligned}$$

and more generally $X^k=X\cap G_n^k$, for any $X\subset G_n$. Let $B_n$ be the Borel subgroup of upper triangular matrices, $A_n$ the diagonal torus, and $N_n$ the unipotent radical of $B_n$. We also set

$$\begin{aligned} Z_n=\{tI_n,\ t\in F^*\}, \end{aligned}$$

the centre of $G_n$.

We fix a non-trivial smooth character $\theta$ from $(F,+)$ to $R^\times$ and, by abuse of notation, we will denote by $\theta$ the character $x\mapsto \theta (\sum _{i=1}^{n-1} n_{i,i+1})$ of $N_n$. Notice that if $R=\overline{\mathbb {Q}_{\ell }}$, this character in fact takes values in $\overline{\mathbb {Z}_{\ell }}^\times$ because F is a union of pro-p subgroups.

If $\lambda$ is an ordered partition of n, we denote by $P_{\lambda }$ the standard parabolic subgroup of $G_n$ attached to it, by $M_{\lambda }$ the standard Levi factor of $P_\lambda$, and by $N_{\lambda }$ its unipotent radical. If $t+r=n$, we let

$$\begin{aligned} U_{t,r}=\left\{ \begin{pmatrix} I_t &{} x \\ &{} y \end{pmatrix}:x\in \mathcal {M}_{t,r},y\in N_r\right\} , \end{aligned}$$

and $H_{t,r}= G_tU_{t,r}$. By restriction, $\theta$ defines a character of $U_{t,r}$. We let $P_n=H_{n-1,1}$ denote the mirabolic subgroup of $G_n$.

We denote by $w_n$ the antidiagonal matrix of $G_n$ with ones on the antidiagonal, and if $n=r+t$, we denote by $w_{t,r}$ the matrix ${\text {diag}}(I_t,w_r)$. Notice that our notations are different from those of [10] for $U_{t,r}$, $H_{t,r}$, and $w_{t,r}$.

If $\Phi \in \mathcal {C}_c^\infty (F^n)$, we denote by $\widehat{\Phi }$ its Fourier transform with respect to the $\theta$-self-dual R-Haar measure dx on $F^n$ satisfying $dx(\mathfrak {o})=q^{-l/2}$, where the integer l satisfies that $\theta \mid _{\mathfrak {p}^l}$ is trivial, but $\theta \mid _{\mathfrak {p}^{l-1}}$ is non-trivial.

For G a locally profinite group, we let $\mathfrak {R}_R(G)$ denote the abelian category of smooth R-representations of G. All R-representations henceforth considered are assumed to be smooth.

Let $\overline{\mathbb {Q}_{\ell }}$ be an algebraic closure of the $\ell$-adic numbers, $\overline{\mathbb {Z}_{\ell }}$ its ring of integers, and $\overline{\mathbb {F}_{\ell }}$ its residue field, which is an algebraic closure of the finite field of $\ell$ elements. By an $\ell$-adic representation of G, we mean a representation of G on a $\overline{\mathbb {Q}_{\ell }}$-vector space, and by an $\ell$-modular representation of G, we mean a representation of G on a $\overline{\mathbb {F}_{\ell }}$-vector space. Suppose that G has a compact open subgroup of pro-order invertible in R, then for any closed subgroup K of G, the modulus character $\delta _K$ is defined in [20, I 2.6] (see also Sect. 2.2, where a different, but equivalent definition is given). Suppose, moreover, that for H a closed subgroup of G, the modulus characters $\delta _H$ and $\delta _G$ take values in $q^\mathbb {Z}$, and then we write ${\text {Ind}}_H^G$ for the functor of normalised smooth induction from $\mathfrak {R}_R(H)$ to $\mathfrak {R}_R(G)$ and write ${\text {ind}}_H^G$ for the functor of normalised smooth induction with compact support. By definition, if $\sigma$ is a smooth representation of H, then ${\text {Ind}}_H^G(\sigma )$ is the usual induction applied to $\delta _H^{1/2}\delta _G^{-1/2}\sigma$ where the square roots of the modulus characters are determined by the choice of $q^{1/2}$. We have a similar definition for ${\text {ind}}_H^G(\sigma )$.

We assume that our choice of square roots of q in $\overline{\mathbb {F}_{\ell }}$ and $\overline{\mathbb {Q}_{\ell }}$ are compatible in the sense that the former is the reduction modulo $\ell$ of the latter, which is chosen in $\overline{\mathbb {Z}_{\ell }}$.

Let Q be a parabolic subgroup of $G_n$ with Levi factor M. We write $\varvec{i}^{G_n}_Q$ for the functor of normalised parabolic induction from $\mathfrak {R}_R(M)$ to $\mathfrak {R}_R(G_n)$. If $\tau =\pi _1\otimes \dots \otimes \pi _r$ is a smooth R-representation of $M_{(m_1,\dots ,m_r)}$ with $\sum _i m_i=n$, we will use the product notation $\pi _1\times \dots \times \pi _r$ for the induced representation $\varvec{i}^{G_n}_{P_{(m_1,\dots ,m_r)}}(\tau )$. An irreducible R-representation of $G_n$ is called cuspidal if it is irreducible, and it does not appear as a subrepresentation of any parabolically induced representation. It is called supercuspidal, moreover, if it does not appear as a subquotient of any parabolically induced representation. For $\pi$ an R-representation with central character, for example an R-representation of $G_n$ parabolically induced from an irreducible R-representation, we denote its central character by $c_{\pi }$.

2.2 R-Haar measures

Let R be a commutative ring with identity of characteristic $\ell$, and let G be a locally profinite group which admits a compact open subgroup of pro-order invertible in R. We let

$$\begin{aligned} \mathcal {C}_c^{\infty }(G,R)=\{f:G\rightarrow R:f \text { is locally constant and compactly supported}\}, \end{aligned}$$

(we sometimes write this, more simply, as $\mathcal {C}_c^{\infty }(G)$ according to the context). A left (resp. right) R -Haar measure on G is a nonzero linear form on $\mathcal {C}_c^{\infty }(G,R)$ which is invariant under left (resp. right) translation by G. If $\mu$ is a left (or right) R-Haar measure on G and $f\in \mathcal {C}_c^{\infty }(G,R)$, we write

$$\begin{aligned} \mu (f)=\int _G f(g)\mu . \end{aligned}$$

By [20, I 2.4], for each compact open subgroup K of G of pro-order invertible in R there exists a unique left R-Haar measure $\mu$ such that $\mu (K)=1$. The volume $\mu (K')=\mu (\mathbf {1}_{K'})$ of a compact open subgroup $K'$ of G is equal to zero if and only if the pro-order of $K'$ is equal to zero in R. In the present work, the modulus character of G is the unique character $\delta _G:G\rightarrow R^\times$ such that, if $\mu$ is a left R-Haar measure on G, $\delta _G\mu$ is a right R-Haar measure on G. More generally, if H is a closed subgroup of G, we let $\delta ={\delta _{G}^{-1}}\mid _{H}\delta _H$, and

$$\begin{aligned} \mathcal {C}_c^{\infty }(H\backslash G,\delta , R) \end{aligned}$$

be the space of functions from G to R, fixed on the right by a compact open subgroup of G, compactly supported modulo H, and which transform by $\delta$ under H on the left (we sometimes write this as $\mathcal {C}_c^{\infty }(H\backslash G,\delta )$). For $f\in \mathcal {C}_c^{\infty }(G,R)$, we denote by $f^H$ the function in $\mathcal {C}_c^{\infty }(H\backslash G,\delta , R)$ defined by

$$\begin{aligned} f^H(g)=\int _{H} f(hg)\delta ^{-1}(h)dh, \end{aligned}$$

for dh a right R-Haar measure on H. It is proved in [20, I 2.8] that the map $f\mapsto f^H$ is surjective and that there is a unique, up to an invertible scalar, nonzero linear form $d_{H\backslash G}g$ on $\mathcal {C}_c^{\infty }(H\backslash G,\delta , R)$, which is right invariant under G. We call such a nonzero linear form on $\mathcal {C}_c^{\infty }(H\backslash G,\delta , R)$ a $\delta$-quasi-invariant quotient measure on $H\backslash G$ and, for $f\in \mathcal {C}_c^{\infty }(H\backslash G,\delta , R)$, we write

$$\begin{aligned} d_{H\backslash G}g(f)=\int _{H\backslash G} f(g)d_{H\backslash G}g. \end{aligned}$$

We can then define dg to be such that, for all $f\in \mathcal {C}_c^{\infty }(G,R)$, we have

$$\begin{aligned} \int _{G}f(g)dg=\int _{H\backslash G} f^H(g)d_{H\backslash G}g. \end{aligned}$$

Remark 2.1

Let H be a closed subgroup of G. Let $\mathcal {C}_{c,e}^\infty (H)$ denote the subspace of functions in $\mathcal {C}_c^\infty (H,\overline{\mathbb {Q}_{\ell }})$ which take integral values. Up to a correct normalisation of the $\overline{\mathbb {Q}_{\ell }}$-Haar measure dh on H, for all $f\in \mathcal {C}_{c,e}^\infty (H)$, the integral $\int _{H} f(h)dh$ belongs to $\overline{\mathbb {Z}_{\ell }}$. Suppose K is a closed subgroup of H, for which there is a $\delta$-quasi-invariant quotient measure $d_{K\backslash H}h$ on $K\backslash H$. We write $\mathcal {C}_{c,e}^\infty (K\backslash H,\delta )$ for the subspace of functions in $\mathcal {C}_{c}^\infty (K\backslash H,\delta )$ which take integral values. Similarly, up to correct normalisation of the quotient measure, the value of $\int _{K \backslash H} f(h)d_{K\backslash H}h$ belongs to $\overline{\mathbb {Z}_{\ell }}$ when $f\in \mathcal {C}_{c,e}^\infty (K\backslash H,\delta )$. Moreover, for all f in $\mathcal {C}_{c,e}^\infty (K\backslash H,\delta )$, we have

$$\begin{aligned} r_{\ell }\left( \int _{K \backslash H} f(h)d_{K\backslash H}h\right) =\int _{K \backslash H} r_{\ell }(f(h))d_{K\backslash H}h. \end{aligned}$$

We write $\mathcal {C}_{c,e}^\infty (F^n)$ for the subspace of functions in $\mathcal {C}_c^\infty (F^n)$ which take values in $\overline{\mathbb {Z}_{\ell }}$. Note that if $\Phi \in \mathcal {C}_{c,e}^\infty (F^n)$, then $\hat{\Phi }\in \mathcal {C}_{c,e}^\infty (F^n)$.

For the remainder of this section, let G denote a unimodular locally profinite group. Suppose that B is a closed subgroup of G, and K is a compact open subgroup of G such that $G=BK$.

Lemma 2.2

Let dg be an R-Haar measure on G. There exist a right R-Haar measure db on B and a right K-invariant measure dk on $(K\cap B)\backslash K$ such that, for all $f\in \mathcal {C}^\infty _c(G,R)$, we have

$$\begin{aligned} \int _{G}f(g)dg= \int _{(K\cap B)\backslash K} \int _{B}f(bk)\delta _B(b)^{-1}db dk.\end{aligned}$$

Proof

We observe first that the map $\phi \mapsto \phi \mid _{K}$ is a vector space isomorphism between $\mathcal {C}_c^{\infty }(B\backslash G,\delta _{B}, R)$ and $\mathcal {C}_c^{\infty }((K\cap B)\backslash K, R).$ It is injective because $G=BK$. To show surjectivity, we recall that the characteristic functions $\mathbf {1}_{(K\cap B)k \mathcal {U}}$, with $\mathcal {U}$ a compact subgroup of K of pro-order invertible in R and $k\in K$, span $\mathcal {C}_c^{\infty }((K\cap B)\backslash K, R)$. Moreover, the function $\mathbf {1}_{k \mathcal {U}}^B$ belongs to $\mathcal {C}_c^{\infty }(B\backslash G,\delta _{B}, R)$, and a computation shows ${\mathbf {1}_{k \mathcal {U}}^B}\mid _{K}=db(B\cap k \mathcal {U}k^{-1})\mathbf {1}_{(K\cap B)k \mathcal {U}}$. Surjectivity follows as $db(B\cap k \mathcal {U}k^{-1})$ is invertible in R. In particular, if dk is a right K-invariant measure on $\mathcal {C}_c^{\infty }((K\cap B)\backslash K, R)$, the map $\mu :\phi \mapsto dk(\phi \mid _{K})$ is a right G-invariant measure on $B\backslash G$. The result follows from the formula $\int _{G}f(g)dg=\mu (f^B)$. $\square$

Remark 2.3

Let $K_n={\text {GL}}_n(\mathfrak {o}_F)$. By the Iwasawa decomposition, we have $G_n=B_nK_n$. Let $\mu _{G_n}$ be an R-Haar measure on $G_n$. If $\ell =0$, or more generally $\ell \not \mid q-1$, then, for all $f\in \mathcal {C}_c^\infty (G_n,R)$, we have

$$\begin{aligned} \int _{G_n}f(g)dg=\int _{B_n}\int _{K_n}f(bk)dbdk, \end{aligned}$$

for good choices of a left R-Haar measure db on $B_n$ and an R-Haar measure dk on $K_n$. As noticed by Mínguez in [15], this is no longer true in general. More precisely, it is not true when $\ell \mid q-1$ as the restriction of an R-Haar measure on K to $\mathcal {C}_c^\infty ((K_n\cap B_n)\backslash K_n)$ is zero. That is why we use a right invariant measure on $K\cap B\backslash K$ in Lemma 2.2.

Let Q, L and N be closed subgroups of G such that $Q=LN$ and L normalises N. Suppose that there exists a compact open subgroup $\mathcal {U}$ of G of pro-order invertible in R such that $Q\cap \mathcal {U}=(N\cap \mathcal {U})(M\cap \mathcal {U})$. Let dl be an R-Haar measure on L and dn be an R-Haar measure on N.

Lemma 2.4

Let $f\in \mathcal {C}^\infty _c(Q,R)$. There exists a unique right R-Haar measure dq on Q such that

$$\begin{aligned} \int _Qf(q)dq=\int _L\int _Nf(nl)dldn. \end{aligned}$$

Proof

As L normalises N, we see that $\int _L\int _Nf(nl)dldn$ is a right Q-invariant linear form on $C_c^\infty (Q,R)$. But as $Q\cap \mathcal {U}=(N\cap \mathcal {U})(M\cap \mathcal {U})$, it is easy to see that $\int _L\int _N1_{Q\cap \mathcal {U}}(nl)dldn$ is nonzero. $\square$

Remark 2.5

A typical instance is when $G=G_n$, $Q=LN$ is a standard parabolic subgroup of $G_n$.

We have the following corollary to Lemmas 2.2 and 2.4.

Corollary 2.6

Let dg be an R-Haar measure on G. There exist an R-Haar measure da on $A_n$, and a right $K_n$-invariant measure dk on $(K_n\cap B_n)\backslash K_n$ such that, for all $f\in \mathcal {C}_c^\infty (N_n \backslash G_n, R)$, we have

$$\begin{aligned} \int _{N_n\backslash G_n}f(g)dg= \int _{(K_n\cap B_n)\backslash K_n}\int _{A_n} f(ak)\delta _{B_n}(a)^{-1}dadk.\end{aligned}$$

Proof

Let $f\in \mathcal {C}_c^\infty (N_n\backslash G_n,R)$, then $f=h^{N_n}$ for $h\in \mathcal {C}_c^\infty (G_n,R)$. By Lemmas 2.2 and 2.4,

$$\begin{aligned} \int _{G_n} h(g)dg=\int _{K_n\cap B_n\backslash K_n}\int _{ A_n } \int _{N_n}h(nak)\delta _{B_n}(a)^{-1}dn da dk, \end{aligned}$$

However,

$$\begin{aligned} \int _{G_n} h(g)dg=\int _{N_n\backslash G_n}f(g)dg \end{aligned}$$

and

$$\begin{aligned} \int _{N_n} h(nak)dn=h^{N_n}(ak)=f(ak). \end{aligned}$$

$\square$

From the Iwasawa decomposition, we also have $G_n=P_nZ_nK_n$. We use the following integration formula, which is proved in a similar fashion.

Corollary 2.7

Let dg be an R-Haar measure on G. There exist an R-Haar measure dz on $Z_n$, a $\delta$-quasi-invariant quotient measure dp on $N_n\backslash P_n$, and a right $K_n$-invariant measure dk on $(K_n\cap B_n)\backslash K_n$ such that, for all $f\in \mathcal {C}_c^\infty (N_n\backslash G_n,R)$,

$$\begin{aligned} \int _{N_n\backslash G_n}f(g)dg= \int _{(K_n\cap P_n)\backslash K_n}\int _{Z_n}\int _{N_n\backslash P_n} f(pzk)\det (p)^{-1}dpdzdk. \end{aligned}$$

Henceforth, equalities involving integrals will be true only up to the correct normalisation of measures.

2.3 Derivatives

Henceforth, we suppose that R is an algebraically closed field. Following [2], we define the following exact functors:

• 1. $\Psi ^+:\mathfrak {R}_R(G_{n-1})\rightarrow \mathfrak {R}_R(P_n)$, extension by the trivial representation twisted by $\nu ^{\frac{1}{2}}$.

• 2. $\Psi ^-: \mathfrak {R}_R(P_n)\rightarrow \mathfrak {R}_R(G_{n-1})$, the functor of $U_{n-1}$-coinvariants twisted by $\nu ^{-\frac{1}{2}}$.

• 3. $\Phi ^+:\mathfrak {R}_R(P_{n-1})\rightarrow \mathfrak {R}_R(P_n)$, the functor $\Phi ^+(X)={\text {ind}}_{P_{n-1}U_n}^{P_n}(X\otimes \theta )$.

• 4. $\Phi ^-: \mathfrak {R}_R(P_{n})\rightarrow \mathfrak {R}_R(P_{n-1})$, the functor of $(U_{n-1},\theta )$-coinvariants twisted by $\nu ^{-\frac{1}{2}}$.

• 5. $\Phi ^+_{nc}:\mathfrak {R}_R(P_{n-1})\rightarrow \mathfrak {R}_R(P_n)$, the functor $\Phi ^+_{nc}(X)={\text {Ind}}_{P_{n-1}U_n}^{P_n}(X\otimes \theta )$.

Theorem 2.8

([1, 2] (cf. [20, III 1.3]))

• 1. We have $\Psi ^{-}\Phi ^+=\Psi ^-\Phi _{nc}^+=\Phi ^-\Psi ^+=0$, and $\Psi ^-$, resp. $\Phi ^+$, resp. $\Phi ^-$, is left adjoint to $\Psi ^+$, resp. $\Phi ^-$, resp. $\Phi ^+_{nc}$.

• 2. The identity functor $\mathbf {1}:\mathfrak {R}_R(P_n)\rightarrow \mathfrak {R}_R(P_n)$ admits a filtration

  $$\begin{aligned} 0=T_n\subset T_{n-1}\subset \cdots \subset T_0=\mathbf {1} \end{aligned}$$

  such that $T_k=(\Phi ^+)^k(\Phi ^-)^k$ and $T_{k-1}/T_k=(\Phi ^+)^{k-1}\Psi ^+\Psi ^-(\Phi ^-)^{k-1}$.

Let $\tau$ be an R-representation of $P_n$. Let $\tau _{(k)}=(\Phi ^-)^{k-1}(\tau )$. The k th derivative $\tau ^{(k)}$ of $\tau$ is defined by $\tau ^{(k)}=\Psi ^-(\tau _{(k)})$. From the filtration property above, any smooth representation of $P_n$ has a highest nonzero derivative, we call it the highest derivative of $\tau$, it is a representation $\tau ^{(h)}$ of $G_{n-h}$ for $h\ge 1$. We recall the classification of irreducible R-representations of $P_n$.

Lemma 2.9

([1] (cf. [20, III 1.5])) Let $\tau$ be an irreducible R-representation of $P_n$. There exists a unique nonzero derivative of $\tau$. Furthermore, for k in $\{1,\ldots ,n\}$, if $\tau ^{(k)}\ne 0$ then $\tau =(\Phi ^+)^k\Psi ^+(\tau ^{(k)})$. Conversely, if $\rho$ is an irreducible R-representation of $G_{n-k}$, then $\pi =(\Phi ^+)^k\Psi ^+(\rho )$ is irreducible and $\rho$ is the kth derivative of $\pi$.

Let $\pi$ be an R-representation of $G_n$. The zeroth derivative $\pi ^{(0)}$ of $\pi$ is $\pi$. Let $\tau =\pi \mid _{P_n}$ and set $\pi _{(k)}=\tau _{(k)}$ for $k=0,1,\ldots ,n$. Define the k th derivative $\pi ^{(k)}$ of $\pi$ by $\pi ^{(k)}=\tau ^{(k)}$, for $k=1,\ldots ,n$.

Lemma 2.10

([1]) Let $\pi$ be an R-representation of finite length. Then the dimension of $\pi ^{(n)}$ is finite and equal to the dimension of ${\text {Hom}}_{N_n}(\pi ,\theta )$.

We remark once and for all that the functor

$$\begin{aligned} \pi \mapsto {\text {Hom}}_{N_n}(\pi ,\theta ) \end{aligned}$$

is exact from $\mathfrak {R}_R(G_n)$ to the category of R-vector spaces, because the functor $\pi \mapsto \pi ^{(n)}$ is exact and will use this fact a lot without mentioning it. We will also use several times the following proposition.

Proposition 2.11

([2, Proposition 3.7]) Let $\rho$ and $\rho '$ be R-representations of $G_n$ and $\tau$ and $\tau '$ be R-representations of $P_m$, we have

• 1. ${\text {Hom}}_{P_{n+1}}(\Psi ^+(\rho )\otimes \Psi ^+(\rho '),R)={\text {Hom}}_{G_n}(\rho \otimes \rho ',R)$;

• 2. ${\text {Hom}}_{G_n}(\Phi ^+(\tau )\otimes \Phi ^+(\tau '),R)={\text {Hom}}_{P_n}(\tau \otimes \tau ',R)$;

• 3. ${\text {Hom}}_{G_n}(\Psi ^+(\rho )\otimes \Phi ^+(\tau ),R)=\{0\}$.

The derivatives of a product are given by the Leibniz rule.

Lemma 2.12

([2] (cf. [20, III 1.10])) Suppose $\pi$ is an R-representation of $G_n$ and $\rho$ is an R-representation of $G_m$, then $(\pi \times \rho )^{(k)}$ has a filtration with successive quotients $\pi ^{(i)}\times \rho ^{(k-i)}$, for $0\leqslant i\leqslant k$.

It is well known to experts that the derivative functors commute with reduction modulo $\ell$. However, it seems that no proof appears in the literature. We will use this property in the sequel, but postpone its proof to Sect. 2.6.

Theorem 2.13

Let $\tau$ be an integral $\ell$-adic representation of finite length of $G_n$, then for $0\le k\le n$, one has $r_\ell (\tau ^{(k)})=[r_\ell (\tau )^{(k)}]$ (where the square brackets stand for the semi-simplification again).

Proof

See just after Corollary 2.29. $\square$

2.4 Parabolic induction and restriction, integral structures, and reduction modulo $\ell$

Let G be a direct product of the form $G_{m_1}\times \cdots \times G_{m_r}, with \,m_i \in \mathbb {Z}$. An $\ell$-adic representation $(\pi ,V)$ of G is called integral if it has finite length and if V contains a G-stable $\overline{\mathbb {Z}_{\ell }}$-lattice $\Lambda$. Such a lattice is called an integral structure, or lattice, in $\pi$. A character is integral if and only if it takes values in $\overline{\mathbb {Z}_{\ell }}$. By [20, II 4.12], a cuspidal representation is integral if and only if its central character is integral.

If $\pi$ is an integral $\ell$-adic representation with integral structure $\Lambda$, then $\pi$ defines an $\ell$-modular representation on the space $\Lambda \otimes _{\overline{\mathbb {Z}_{\ell }}}\overline{\mathbb {F}_{\ell }}$. By the Brauer–Nesbitt principle [24, Theorem 1], the semi-simplification, in the Grothendieck group of finite length $\ell$-modular representations, of $(\pi ,\Lambda \otimes _{\overline{\mathbb {Z}_{\ell }}}\overline{\mathbb {F}_{\ell }})$ is independent of the choice of integral structure in $\pi$ and we call this semisimple representation $r_{\ell }(\pi )$ the modulo  $\ell$ of $\pi$. We say that an $\ell$-modular representation $\pi$ lifts to an integral $\ell$-adic representation $\tau$ if $r_{\ell }(\tau )\simeq \pi$, and we will only really use this notion of lift when $\pi$ is irreducible.

Let H be a closed subgroup of G, $\sigma$ be an integral $\ell$-adic representation of H, and $\Lambda$ be an integral structure in $\sigma$. By [20, I 9.3], ${\text {ind}}_H^G(\Lambda )$ is an integral structure in ${\text {ind}}_H^G(\sigma )$. Suppose Q is a parabolic subgroup of G with Levi decomposition $Q=MN$, $\sigma$ is an integral representation of M, $\Lambda$ is an integral structure in $\sigma$. Then $\varvec{i}_Q^G(\Lambda )$ is an integral structure in $\varvec{i}_Q^G(\sigma )$. Moreover, we have $\varvec{i}_Q^G(\Lambda \otimes _{\overline{\mathbb {Z}_{\ell }}}\overline{\mathbb {F}_{\ell }})\simeq \varvec{i}_Q^G(\Lambda )\otimes _{\overline{\mathbb {Z}_{\ell }}}\overline{\mathbb {F}_{\ell }}$. Hence, parabolic induction commutes with reduction modulo $\ell$, in the sense that

$$\begin{aligned} r_{\ell }(\varvec{i}_Q^G(\sigma ))\simeq [\varvec{i}_Q^G(r_{\ell }(\sigma ))], \end{aligned}$$

where the square brackets indicate we take the semi-simplification of $\varvec{i}_Q^G(r_{\ell }(\sigma ))$.

If $\pi$ is an R-representation of G, and Q is a parabolic subgroup of G, then we denote by $\varvec{r}_Q^G(\pi )$ its normalised Jacquet module. In a similar way, if $\tau$ is an integral $\ell$-adic representation of G of finite length, it is a consequence of [6, Proposition 1.4, (i)] taking $R=\overline{\mathbb {Z}_\ell }$, that the image of a G-stable lattice of $\tau$ via the natural surjection on the Jacquet module $\varvec{r}_Q^G(\tau )$ is a (Q-stable) lattice. Thanks to the exactness properties of the Jacquet functor and [20, I 9.3], one then has the isomorphism

$$\begin{aligned} r_{\ell }(\varvec{r}_Q^G(\tau ))\simeq [\varvec{r}_Q^G(r_{\ell }(\tau ))]. \end{aligned}$$

2.5 Representations of Whittaker type

We now recall facts originally from [21], with our exposition and notation following [17, Section 7]. First, recall the following definition:

Definition 2.14

An irreducible R-representation of $G_n$ satisfying ${\text {Hom}}_{N_n}(\pi ,\theta )\ne 0$ is called generic.

A segment $\Delta =[a,b]_\rho$ is a sequence $(\nu ^a\rho ,\nu ^{a+1}\rho ,\cdots ,\nu ^b\rho )$ with $\rho$ a cuspidal R-representation of $G_m$ for some $m\geqslant 1$, and $a,b\in \mathbb {Z}$ with $a\leqslant b$. Its length is, by definition, $b-a+1$. Two segments  $\Delta =[a,b]_\rho$ and $\Delta '=[a',b']_{\rho '}$ are said to be equivalent if they have the same length, and $\nu ^a\rho \simeq \nu ^{a'}\rho '$. Hence, as noticed in [17, 7.2], the segment $[a,b]_\rho$ identifies with the cuspidal pair

$$\begin{aligned} r_\Delta =(M_{(m,\dots ,m)},\nu ^a\rho \otimes \nu ^{a+1}\rho \otimes \cdots \otimes \nu ^b\rho ), \end{aligned}$$

and the equivalence relation on segments is the restriction of the classical isomorphism equivalence relation on cuspidal pairs. To such a segment $\Delta$, in [17, Definition 7.5] the authors associate a certain quotient $\mathcal {L}(\Delta )$ of $I(\Delta )=\nu ^a\rho \times \nu ^{a+1}\rho \times \cdots \times \nu ^b\rho$. The representation $\mathcal {L}(\Delta )$ in fact determines $\Delta$, as its normalised Jacquet module with respect to the opposite of $N_{(m,\dots ,m)}$ is equal to $r_\Delta$ according to [17, Lemma 7.14]. The conclusion of this is that the objects $\Delta$, $\mathcal {L}(\Delta )$, and $r_\Delta$ determine one another, and hence we call $\mathcal {L}(\Delta )$ a segment.

We say that $\Delta$ precedes $\Delta '$ if we can extract from the sequence $(\nu ^a\rho , \ldots , \nu ^b\rho ,\nu ^{a'}\rho ',\ldots ,\nu ^{b'} \rho ')$ a subsequence which is a segment of length strictly larger than both the length of $\Delta$ and the length of $\Delta '$. We say $\Delta$ and $\Delta '$ are linked if $\Delta$ precedes $\Delta '$ or $\Delta '$ precedes $\Delta$.

Let $\rho$ be a cuspidal R-representation of $G_m$, we denote by $\mathbb {Z}_\rho$ the cuspidal line of $\rho$, that is the set

$$\begin{aligned} \mathbb {Z}_\rho =\{\nu ^r \rho ,r\in \mathbb {Z}\}, \end{aligned}$$

which when $\nu$ has finite order in R is perhaps more appropriately thought of as a circle. We define an element $o(\rho )$ in $\mathbb {N}\cup \{+\infty \}$ by the formula

$$\begin{aligned} o(\rho )=|\mathbb {Z}_\rho |. \end{aligned}$$

In [17, 5.2], a positive integer $e(\rho )$ is attached to $\rho$,

$$\begin{aligned} e(\rho )={\left\{ \begin{array}{ll} +\infty &{}\text { if }R=\overline{\mathbb {Q}_{\ell }};\\ |\{\nu ^r \rho ,r\in \mathbb {Z}\}|&{}\text { if }R=\overline{\mathbb {F}_{\ell }}\text { and } o(\rho )>1;\\ \ell &{}\text { if }R=\overline{\mathbb {F}_{\ell }}\text { and } o(\rho )=1. \end{array}\right. } \end{aligned}$$

An integer $f(\rho )$ is also defined in [17, 5.2] via type theory, the definition of which we will recall later, and following this reference we set $q(\rho )=q^{f(\rho )}$. If $o(\rho )=1$, then $e(\rho )$ is the order of $q(\rho )$. For integers $a\leqslant b$, we denote by $St(\rho ,[a,b])$ the generalised Steinberg representation associated with $\Delta =[a,b]_\rho$, i.e. the unique generic subquotient of

$$\begin{aligned} \nu ^a\rho \times \nu ^{a+1}\rho \times \cdots \times \nu ^b\rho . \end{aligned}$$

By [17, Remarque 8.14], the representation $St(\rho ,[a,b])$ is equal to the segment $\mathcal {L}(\Delta )$ if and only if its length $b-a+1<e(\rho )$. In this case, we say that $\mathcal {L}(\Delta )$ is a generic segment. As in [17], we will write $St(\rho ,k)$ for $St(\rho ,[0,k-1])$, for $k\geqslant 1$.

By [20, III 4.25 & 5.10], if $\rho$ is a cuspidal $\ell$-modular representation of $G_r$, then it is the reduction modulo $\ell$ of an integral cuspidal $\ell$-adic representation $\sigma$ of $G_r$, and the reduction modulo-$\ell$ of any integral cuspidal $\ell$-adic representation is still cuspidal.

Definition 2.15

We recall from [17, Remarque 8.15] that an irreducible R-representation $\pi$ of $G_n$ is called banal if no $\rho$ in the cuspidal support of $\pi$ is such that $\mathbb {Z}_\rho$ is included in this cuspidal support. Let $\rho$ be a R-cuspidal representation of $G_n$, take $a\leqslant b$ in $\mathbb {N}$, and set $\Delta =[a,b]_\rho$. The representation $\mathcal {L}(\Delta )$ is then banal if and only if the length $b-a+1$ of $\Delta$ is strictly less than $o(\rho )$, in which case we also say that $\Delta$ is banal.

Let $\Delta =[a,b]_\rho$ be a segment. We notice that if the representation $\mathcal {L}(\Delta )$ is banal, then it is generic, and that the generic segment $\mathcal {L}(\Delta )$ is non-banal if and only if $o(\rho )=1$, i.e. if and only if $\rho$ is non-banal in the sense of [16].

Proposition 2.16

Let $\mathcal {L}(\Delta )$ be an $\ell$-modular generic segment, with $\Delta =[a,b]_\rho$ and $\rho$ be a cuspidal representation of $G_m$ (hence $\mathcal {L}(\Delta )$ is a representation of $G_n$ with $n=m(b+1-a)$). Then its nonzero derivatives are of the form $\mathcal {L}(\Delta )^{(km)}$ for $k=0,\dots ,b-a+1$, and $\mathcal {L}(\Delta )^{(km)}=\mathcal {L}([a+k,b]_\rho )$. Moreover, if we denote by $\sigma$ a cuspidal lift of $\rho$, and set $D=[a,b]_\sigma$, then $\mathcal {L}(\Delta )$ is the reduction modulo $\ell$ of $\mathcal {L}(D)$.

Proof

For the first statements, by definition, the lth derivative of a $\mathcal {L}(\Delta )$ is a quotient of its Jacquet module with respect to the (unipotent subgroup of the) standard parabolic subgroup of type $(n-l,l)$; hence, according to [17, Proposition 7.25], the integer l is a multiple of m. Moreover, still according to the same proposition, and by definition of derivatives, we have

$$\begin{aligned} \mathcal {L}(\Delta )^{(km)}=(\Phi ^-)^{km}(\mathcal {L}([a,a+k-1]_\rho ))\otimes \mathcal {L}([a+k,b]_\rho )= \mathcal {L}([a+k,b]_\rho ) \end{aligned}$$

because $\mathcal {L}([a,a+k-1]_\rho )$ is still generic. Notice that this results holds for $\ell$-adic segments as well by [25, Proposition 9.6]. Now we move on to the statement concerning reduction modulo $\ell$. First, as reduction modulo $\ell$ commutes with derivatives, we know that $r_\ell (\mathcal {L}(D))$ contains a generic component. However, it must be a generic component of $r_\ell (I(D))$, which is the semi-simplification of $I(\Delta )$. As $I(\Delta )$ has a unique generic subquotient which is $\mathcal {L}(\Delta )$, we already know that $\mathcal {L}(\Delta )\subseteq r_\ell (\mathcal {L}(D))$. Hence, it suffices to prove that $r_\ell (\mathcal {L}(D))$ is irreducible. We do this by induction on the length of $\Delta$. If $\Delta$ is cuspidal (length 1), then there is nothing to prove. Now we do the induction step. If $r_\ell (\mathcal {L}(D))$ was reducible, then it would contain another submodule $\tau$ (which is degenerate, but it doesn’t matter), so we would have $\mathcal {L}(\Delta )\oplus \tau \subseteq r_\ell (\mathcal {L}(D))$. Notice that $\tau$ has the same cuspidal support as $\mathcal {L}(\Delta )$, in particular the integer h such that its highest derivative is $\tau ^{(h)}$ is a multiple of m (we recall that $m\geqslant 1$). Set $h=km$, in particular we have $\mathcal {L}(\Delta )^{(h)}\oplus \tau ^{(h)} \subseteq r_\ell (\mathcal {L}(D))^{(h)}$. Recall once again that derivatives commute with $r_\ell$; hence, $\mathcal {L}([a+k,b]_\rho )\oplus \tau ^{(h)}\subseteq r_\ell (\mathcal {L}(D^{(h)}))$, but

$$\begin{aligned} r_\ell (\mathcal {L}(D^{(h)}))=r_\ell (\mathcal {L}([a+k,b]_\sigma ))=\mathcal {L}([a+k,b]_\rho ), \end{aligned}$$

the last equality by induction hypothesis. This implies that $\tau ^{(h)}=0$, a contradiction. $\square$

Definition 2.17

An R-representation $\pi$ of $G_n$ is called of Whittaker type if it is parabolically induced from generic segments, i.e. if $\pi = \mathcal {L}(\Delta _1)\times \cdots \times \mathcal {L}(\Delta _t)$ with $\Delta _i$ generic segments, for $1\leqslant i\leqslant t$.

Let $\pi$ be a representation of Whittaker type. By Lemmas 2.10 and 2.12, the space ${\text {Hom}}_{N_n}(\pi ,\theta )$ is of dimension 1, and we denote by $W(\pi ,\theta )$ the Whittaker model of $\pi$, i.e. $W(\pi ,\theta )$ denotes the image of $\pi$ in ${\text {Ind}}_{N_n}^{G_n}(\theta )$. Note that a representation of Whittaker type may not be irreducible; however, it is of finite length. In fact, thanks to the results of Zelevinsky (cf. [25]) in the $\ell$-adic setting, and by [21, Theorem 5.7] (cf. [17, Theorem 9.10 and Corollary 9.12] for another proof) in the $\ell$-modular setting, the irreducible representations of Whittaker type of $G_n$ are exactly the generic representations.

If $\pi$ is a smooth representation of $G_n$, we denote by $\widetilde{\pi }$ the representation $g\mapsto \pi (^tg^{-1})$ of $G_n$. Let $\tau$ be an $\ell$-adic irreducible representation of $G_n$, then $\widetilde{\tau }\simeq \tau ^\vee$, by [7]. Hence, when $\mathcal {L}(\Delta )$ is an $\ell$-modular generic segment of $G_n$, it lifts to an $\ell$-adic segment $\mathcal {L}(D)$ according to Proposition 2.16, and as $\widetilde{\mathcal {L}(D)}\simeq \mathcal {L}(D)^\vee$, we deduce by reduction modulo $\ell$, cf. [20, I 9.7], that $\widetilde{\mathcal {L}(\Delta )}\simeq \mathcal {L}(\Delta )^\vee$.

Remark 2.18

Though we will not use it, it is worth noticing that this equality can be extended to all irreducible representations of $G_n$: both involutions $\pi \mapsto \pi ^\vee$ and $\pi \mapsto \widetilde{\pi }$ extend to the Grothendieck groups of $\ell$-adic and $\ell$-modular representations of finite length of $G_n$. Moreover, these involutions preserve integrality of $\ell$-adic representations, commute with reduction modulo $\ell$, and agree on the Grothendieck group of $\ell$-adic integral representations of finite length by [7]. Hence, they agree on the Grothendieck group of $\ell$-modular representations of finite length by surjectivity of the reduction modulo $\ell$ map on Grothendieck groups [17, Théorème 9.40]. They thus agree on all irreducible $\ell$-modular representations.

If $\pi =\mathcal {L}(\Delta _1)\times \dots \times \mathcal {L}(\Delta _t)$ is a representation of $G_n$ of Whittaker type, we have $\widetilde{\pi }=\widetilde{\mathcal {L}(\Delta _t)}\times \dots \times \widetilde{\mathcal {L}(\Delta _1)}= \mathcal {L}(\Delta _t)^\vee \times \dots \times \mathcal {L}(\Delta _1)^\vee ,$ and we deduce that $\widetilde{\pi }$ is also of Whittaker type. In order to state the functional equation for L-factors of representations of Whittaker type, we will need the following lemma whose proof follows from the discussion above and that $\widetilde{~}$ is an involution.

Lemma 2.19

Let $\pi$ be an $\ell$-modular representation of Whittaker type of $G_n$, then $\widetilde{\pi }$ is of Whittaker type and the map $W\mapsto \widetilde{W}$, where $\widetilde{W}(g)=W(w_n\, ^{t}g^{-1})$, is an R-vector space isomorphism between $W(\pi ,\theta )$ and $W(\widetilde{\pi },\theta ^{-1})$.

For the proofs to come, it will be convenient to choose certain “lifts” of $\ell$-modular representations of Whittaker type. Let us introduce the following notation first.

Definition 2.20

Let $\tau$ be an integral $\ell$-adic representation of Whittaker type. We denote by $W_e(\tau ,\theta )$ the subset of all functions in $W(\tau ,\theta )$ which take integral values.

Now we define “Whittaker lifts” of representations of Whittaker type. We will soon see that they exist.

Definition 2.21

Let $\pi$ be an $\ell$-modular representation of Whittaker type. An integral $\ell$-adic representation of Whittaker type $\tau$ is a Whittaker lift of $\pi$ if $W_e(\tau ,\theta )$ contains a sublattice $\mathcal {W}$ such that $W(\pi ,r_{\ell }(\theta ))=\mathcal {W}\otimes _{\overline{\mathbb {Z}_{\ell }}}\overline{\mathbb {F}_{\ell }}$.

Remark 2.22

Whittaker lifts are not strictly speaking lifts. However, if $\tau$ is a Whittaker lift of $\pi$ then we can lift functions in the Whittaker model of $\pi$ to integral functions in the Whittaker model of $\tau$.

We will use the following lemma to obtain integrality results on $\ell$-adic local factors.

Lemma 2.23

Let $\pi$ be an $\ell$-modular representation of $G_n$, and let $\tau$ be a Whittaker lift of $\pi$ with associated sublattice $\mathcal {W}\subset W_e(\tau ,\theta )$, then the $\overline{\mathbb {Z}_{\ell }}$-module $\widetilde{\mathcal {W}}=\{\widetilde{W}:W\in \mathcal {W}\}$ is a sublattice of $W_e(\widetilde{\tau },\theta ^{-1})$ such that $W(\widetilde{\pi },r_{\ell }(\theta ))=\widetilde{\mathcal {W}}\otimes _{\overline{\mathbb {Z}_{\ell }}}\overline{\mathbb {F}_{\ell }}$.

Proof

As $W\mapsto \widetilde{W}$ is a vector space isomorphism between $W(\tau ,\theta )$ and $W(\widetilde{\tau },\theta ^{-1})$ which restricts to a $\overline{\mathbb {Z}_{\ell }}$-module isomorphism between $W_e(\tau ,\theta )$ and $W_e(\widetilde{\tau },\theta ^{-1})$, the set $\widetilde{\mathcal {W}}$ is a sublattice of $W_e(\widetilde{\tau },\theta ^{-1})$. Moreover, as $W\mapsto \widetilde{W}$ is also a vector space isomorphism between $W(\pi ,r_\ell (\theta ))$ and $W(\widetilde{\pi },r_\ell (\theta ^{-1}))$, and because $r_\ell (\widetilde{W})=\widetilde{r_\ell (W)}$ for all $W\in W_e(\tau ,\theta )$, we deduce that $r_\ell (\mathcal {W})=W(\widetilde{\pi },r_\ell (\theta ^{-1}))$. $\square$

We also introduce “standard lifts” of representations of Whittaker type. We will see in Theorem 2.26 that they are Whittaker lifts.

Definition 2.24

If $\pi = \mathcal {L}(\Delta _1)\times \cdots \times \mathcal {L}(\Delta _t)$ is an $\ell$-modular representation of Whittaker type, we will call a standard lift of $\pi$ a representation $\tau =\mathcal {L}(D_1)\times \cdots \times \mathcal {L}(D_t)$, where $\mathcal {L}(D_i)$ lifts $\mathcal {L}(\Delta _i)$ as in Proposition 2.16.

Remark 2.25

Again, standard lifts are not strictly speaking lifts. However, as parabolic induction commutes with reduction modulo $\ell$, if $\tau$ is a standard lift of an $\ell$-modular representation $\pi$, then $r_\ell (\tau )$ is the semi-simplification of $\pi$.

We now show that standard lifts are Whittaker lifts.

Theorem 2.26

Let $\pi = \mathcal {L}(\Delta _1)\times \cdots \times \mathcal {L}(\Delta _t)$ be an $\ell$-modular representation of Whittaker type, and $\tau = \mathcal {L}(D_1)\times \cdots \times \mathcal {L}(D_t)$ be a standard lift of $\pi$, then it is a Whittaker lift.

Proof

If $\tau$ is generic, it is shown in [24, Theorem 2] that $W_e(\tau ,\theta )$ is an integral structure in $W(\tau ,\theta )$, such that $W(\pi ,r_{\ell }(\theta ))=W_e(\tau ,\theta )\otimes _{\overline{\mathbb {Z}_{\ell }}}\overline{\mathbb {F}_{\ell }}$. We use this result together with the properties of parabolic induction with respect to lattices, and a result from [4] about the explicit description of Whittaker functionals on induced representations.

For each $1\leqslant i\leqslant t$, each $\mathcal {L}(D_i)$ is generic and hence $W_e(\mathcal {L}(D_i),\theta )$ is an integral structure in $W(\mathcal {L}(D_i),\theta )$. By [20, I 9.3], the lattice $\mathfrak {L}=W_e(\mathcal {L}(D_1),\theta )\times \dots \times W_e(\mathcal {L}(D_t),\theta )$ is an integral structure in $\tau$. The space $\mathfrak {L}$ consists of all smooth functions from G to $L=W_e(\mathcal {L}(D_1),\theta )\otimes \dots \otimes W_e(\mathcal {L}(D_t),\theta )$, with the tensor product over $\overline{\mathbb {Z}_{\ell }}$, which transform on the left by $\mathcal {L}(D_1)\otimes \cdots \otimes \mathcal {L}(D_t)$. We recall that $W(\mathcal {L}(D),\theta )=W(\mathcal {L}(D_1),\theta )\otimes \dots \otimes W(\mathcal {L}(D_t),\theta )$, as well as $W(\mathcal {L}(\Delta ),\theta )=W(\mathcal {L}(\Delta _1),\theta )\otimes \dots \otimes W(\mathcal {L}(\Delta _t),\theta )$, is a representation of a standard parabolic subgroup $Q=MU$ of $G_n$, trivial on U.

A function f in $\tau =W(\mathcal {L}(D_1),\theta )\times \dots \times W(\mathcal {L}(D_t),\theta )$, by definition of parabolic induction, is a map from $G_n$ to $W(\mathcal {L}(D),\theta )$, i.e. for $g\in G_n$, $f(g)\in W(\mathcal {L}(D),\theta )$ identifies with a map from M to $\overline{\mathbb {Q}_{\ell }}$, so we can view f as a map of two variables from $G_n\times M$ to $\overline{\mathbb {Q}_{\ell }}$, and similarly, we can view the elements in $\pi =W(\mathcal {L}(\Delta _1),\theta )\times \dots \times W(\mathcal {L}(\Delta _t),\theta )$ as maps from $G_n\times M$ to $\overline{\mathbb {F}_{\ell }}$. In [4, Corollary 2.3], it is shown (for minimal parabolics, but their method works for general parabolics), that there is a Weyl element w in $G_n$, such that if one takes $f\in \tau$, then there is a compact open subgroup $U_f$ of U which satisfies that for any compact subgroup $U'$ of U containing $U_f$, the integral $\int _{U'} f(w u,1_M)\theta ^{-1}(u)du$ is independent from $U'$. We will write $\lambda (f)=\int _{U} f(w u, 1_M)\theta ^{-1}(u)du$. This assertion is also true for $\pi$ with the same proof, for the same choice of w, we write $\mu (h)=\int _{U} h(w u, 1_M)r_\ell (\theta )^{-1}(u)du$ for $h\in \pi$. Both $\lambda$ and $\mu$ are nonzero Whittaker functionals on $\tau$ and $\pi$, respectively, and $\lambda$ sends $\mathfrak {L}$ to $\overline{\mathbb {Z}_{\ell }}$ for a correct normalisation of du. We can, moreover, suppose, for correct normalisations of the $\ell$-adic and the $\ell$-modular Haar measures du, that $\mu =r_\ell (\lambda )$. The surjective map $w: \tau \mapsto W(\tau ,\theta )$ which takes f to $W_f$, defined by $W_f(g)=\lambda (\tau (g)f)$, sends $\mathfrak {L}$ into $W_e(\tau ,\theta )$. Similarly, for $h\in \pi$, if we set $W_h(g)=\mu (\pi (g)h)$, then the map $\pi \mapsto W(\pi ,r_\ell (\theta ))$, taking h to $W_h$, is surjective, and we have $r_\ell (W_f)=W_{r_\ell (f)}$. From this, we obtain that $\mathcal {W}=w(\mathfrak {L})$ is a sublattice of $W_e(\tau ,\theta )$ (see [20, I 9.3]), and $r_\ell (\mathcal {W})=W(\pi ,\theta )$. $\square$

2.6 Generic representations

It is stated (in different terms) in [23, 1.8.4] that if $\tau$ is an integral generic $\ell$-adic representation of $G_n$, then $r_\ell (\tau )$ has a unique generic component $J_{\ell }(\tau )$, and that the map $\tau \mapsto J_\ell (\tau )$ is a surjection from the set of classes of $\ell$-adic generic representations of $G_n$ to the set of classes of $\ell$-modular generic representations of $G_n$. We recall why this is true (it also follows at once from the fact that the derivatives commute with reduction modulo $\ell$, but we shall go in the other direction and use this result to prove that derivatives commute with reduction modulo $\ell$).

Lemma 2.27

Let $\tau$ be an $\ell$-adic integral generic representation of $G_n$, then $r_\ell (\tau )$ contains (with multiplicity one) a unique generic component $J_{\ell }(\tau )$. Moreover, if $r_1\times \dots \times r_t$ is a product of integral cuspidal $\ell$-adic representations such that $r_1\times \dots \times r_t \twoheadrightarrow \tau$, then setting $\rho _i=r_\ell (r_i)$, the representation $J_{\ell }(\tau )$ is the unique generic subquotient of $\rho _1\times \dots \times \rho _t$.

Proof

To prove that $r_\ell (\tau )$ contains a unique generic component, as in the start of the proof of Theorem 2.26 we call upon [24, Theorem 2], which tells us that $W_e(\tau ,\theta )$ is an integral structure in $\tau$. The representation $W_e(\tau ,\theta )\otimes _{\overline{\mathbb {Z}_{\ell }}}\overline{\mathbb {F}_{\ell }}$ is a nonzero $G_n$-submodule of $\mathcal {C}^\infty (N_n\backslash G_n,r_\ell (\theta ))$, and this implies that $r_\ell (\tau )$ contains at least one generic subquotient $J_{\ell }(\pi )$. Let $p: \sigma =r_1\times \dots \times r_t \twoheadrightarrow \tau$ be a surjection as in the statement (provided by considering the cuspidal support of $\tau$), then by multiplicity one of Whittaker functionals, p factors through a surjection $s:W(\sigma ,\theta ) \twoheadrightarrow \tau$ as $\tau$ is generic. By Theorem 2.26, there is a lattice $\mathcal {W}_\sigma \subset W_e(\sigma ,\theta )$ such that if we set $\rho _i=r_\ell (r_i)$, the $\overline{\mathbb {F}_{\ell }}$-module $\lambda =\mathcal {W}_\sigma \otimes \overline{\mathbb {F}_{\ell }}$ is equal to $W(\rho _1\times \dots \times \rho _t,r_\ell (\theta ))$. By [20, I 9.3 (vi)], the $\overline{\mathbb {Z}_{\ell }}$-module $\mathfrak {L}=s(\mathcal {W}_\sigma ) \subset \tau$ is a lattice in $\tau$, and $\mathfrak {L}\otimes \overline{\mathbb {F}_{\ell }}$ is a quotient of $\lambda$. By hypothesis $J_{\ell }(\pi )$ is an irreducible subquotient of $\mathfrak {L}\otimes \overline{\mathbb {F}_{\ell }}$, hence of $\lambda$, i.e. of $W(\rho _1\times \dots \times \rho _t,r_\ell (\theta ))$, and hence of $\rho _1\times \dots \times \rho _t$. In particular $J_{\ell }(\pi )$ is the unique generic subquotient of $\rho _1\times \dots \times \rho _t$, and this implies that $r_\ell (\tau )$ contains a unique generic subquotient. $\square$

We now prove that derivatives commute with reduction modulo $\ell$. Just as for Jacquet modules, this fact is not obvious, because it is not clear that the derivative of an integral structure is still free as a $\overline{\mathbb {Z}_{\ell }}$-module. First we recall some basic facts. Let $(\tau ,V)$ be an integral $\ell$-adic representation of finite length, with filtration $0\subset V_1\subset \dots \subset V_n=V$. Then it is a consequence of [20, I 9.3], that the subquotients $V_{i+1}/V_i$ are integral, and $r_\ell (V)=\bigoplus _i r_\ell (V_{i+1}/V_i)$.

Proposition 2.28

Let $\tau$ be an $\ell$-adic integral representation of $G_n$ of finite length, then we have $(\Phi ^-)^n r_\ell (\tau )=r_\ell ((\Phi ^-)^n \tau )$.

Proof

As $\Phi ^-$ is an exact functor, we can assume that $\tau$ is irreducible. In this case, our assertion amounts, by multiplicity one of Whittaker functionals, to show that $(\Phi ^-)^n r_\ell (\tau )\simeq \overline{\mathbb {F}_{\ell }}$ if $\tau$ is generic, and $(\Phi ^-)^n r_\ell (\tau )= \{0\}$ otherwise. If $\tau$ is generic, this follows from Lemma 2.27. If not, then, considering cuspidal supports again, one has $\tau \hookrightarrow \mu _1\times \dots \times \mu _r$, where the $\mu _i$ are cuspidal. We also have a surjection $s:\mu _1\times \dots \times \mu _r \twoheadrightarrow W(\mu _1\times \dots \times \mu _r,\theta )$, and denoting by $\tau '$ its kernel, we have that $\tau$ is contained in $\tau '$. In particular, $r_{\ell }(\tau )$ is contained in $r_{\ell }(\tau ')$. Moreover, we have $r_{\ell } (\mu _1\times \dots \times \mu _r)=r_\ell (\tau ')\oplus r_\ell (W(\mu _1\times \dots \times \mu _r,\theta ))$. On the other hand, setting $\rho _i=r_\ell (\mu _i)$, then $r_\ell (\mu _1\times \dots \times \mu _r)$ is the semi-simplification of $\rho _1\times \dots \times \rho _r$, in particular, it contains a unique generic component. But, according to Theorem 2.26, $r_\ell (W(\mu _1\times \dots \times \mu _r,\theta ))$ contains the semi-simplification of $W(\rho _1\times \dots \times \rho _r,\theta )$, and in particular it contains a generic component, namely the unique generic component of $r_\ell (\mu _1\times \dots \times \mu _r)$, so that $r_\ell (\tau ')$, hence $r_\ell (\tau )$, does not. Finally, we have $(\Phi ^-)^n (r_\ell (\tau ))=\{0\}$. $\square$

As an immediate corollary, we obtain the following.

Corollary 2.29

Let $\pi$ be an integral $\ell$-adic representation of $G_{n_1}\times G_{n_2}$ of finite length. Then $({\text {Id}}\otimes (\Phi ^-)^{n_2}) r_\ell (\pi )= r_\ell (({\text {Id}}\otimes (\Phi ^-)^{n_2})\pi )$.

From this corollary follows the proof of Theorem 2.13. Indeed, if we set $G=G_n$ and $P=P_{(n-k,k)}$, and $\tau$ as in the statement of Theorem 2.13, we have $\tau ^{(k)}=({\text {Id}}\otimes (\Phi ^-)^{k})\circ \varvec{r}_P^G(\tau )$ (cf. for example the proof of [14, Proposition 2.5]). But $\varvec{r}_P^G$ commutes with reduction modulo $\ell$, as we recalled in Sect. 2.4, and so does ${\text {Id}}\otimes (\Phi ^-)^{k}$, according to Corollary 2.29, and this completes the proof of Theorem 2.13.

We now recall [21, Theorem V.7] (cf. also [17, Theorem 9.10], and [25, Theorem 9.7] for the original proof when $R=\mathbb {C}$), which classifies generic representations.

Theorem 2.30

Let $\pi =\mathcal {L}(\Delta _1)\times \dots \times \mathcal {L}(\Delta _t)$ be an $\ell$-modular (or $\ell$-adic) representation of Whittaker type of $G_n$, then $\pi$ is irreducible if and only if the segments $\Delta _i$ and $\Delta _j$ are unlinked, for all $i,j\in \{1,\ldots , t\}$ with $i\ne j$. In this case, the product is commutative and the set $\{\Delta _1,\dots ,\Delta _t\}$ is determined by $\pi$.

First, we notice the following consequence.

Remark 2.31

If $\pi$ is an $\ell$-modular generic representation of $G_n$, as parabolic induction commutes with reduction modulo $\ell$, it is easy to see that it admits standard lifts which are generic (i.e. irreducible). In this case, one has $r_\ell (\tau )=\pi$ (because $\pi$ is irreducible), and $\tau$ is a true lift, which is generic. Any generic lift $\tau$ is such that $r_\ell (\tau )=J_\ell (\tau )=\pi$. In fact, thanks to Theorem 2.13, any lift of $\pi$ (which is necessarily irreducible), is generic.

It can be shown that any irreducible $\ell$-modular representation $\pi$ of $G_n$ can be written uniquely as a product $\pi _b\times \pi _{tnb}$, where $\pi _b$ has a banal cuspidal support, whereas $\pi _{tnb}$ has a non-banal cuspidal support. Notice that in general, the representation $\pi _b$ does not need to be banal (for example a segment with banal cuspidal support, but the length of which is too large). However, as a consequence of Theorem 2.30, we observe that for a generic representation $\pi$ of $G_n$, the representation $\pi _b$ is a product of banal segments (and $\pi _{tnb}$ is obviously a product of non-banal segments). We claim that $\pi _b$ is a banal representation in the sense of Definition 2.15. It is in fact the largest banal subproduct of $\pi$, this claim following at once from the next proposition. Similarly $\pi _{tnb}$ is non-banal, and has no non-trivial banal subproduct. For this reason we will call $\pi _b$ the banal part of $\pi$, and we will call $\pi _{tnb}$ the totally non-banal part of $\pi$ when $\pi$ is generic.

Proposition 2.32

Let $\rho$ be a cuspidal $\ell$-modular representation of $G_r$, and $\pi$ be a generic representation of $G_n$ whose cuspidal support is contained in $\mathbb {Z}_\rho$. Then $\pi$ is banal if and only if $\rho$ is banal.

Proof

If $o(\rho )=1$, then both $\pi$ and $\rho$ are non-banal, so we suppose that $o(\rho )\ne 1$, in which case $e(\rho )=|\mathbb {Z}_\rho |\geqslant 2$. It is convenient to think of $\mathbb {Z}_\rho$ as a circle, with $e(\rho )$ vertices drawn on it. If $\rho$ is non-banal, then it is obvious that $\pi$ is non-banal. Conversely suppose that $\pi$ is non-banal, but $\rho$ is banal. As $\pi$ is generic, it is a product of unlinked generic segments. By hypothesis, the cuspidal support of $\pi$ is a subset of $\mathbb {Z}_\rho$, this means that if one takes two segments occurring in $\pi$, they are either included in one another, or disjoint and not juxtaposed. As $\pi$ is non-banal, all the vertices on the circle $\mathbb {Z}_\rho$ must be covered by the union of all segments occurring in $\pi$. This implies that either one segment is of length $\geqslant e(\rho )$, or that two segments are linked. Both cases are impossible since $\pi$ is generic, a contradiction. $\square$

2.7 Lifting supercuspidal representations

The aim of this subsection is to show that a non-banal supercuspidal $\ell$-modular representation has two lifts which are not isomorphic after twisting by an unramified character. A fact that we will use to compute L-factors of non-banal supercuspidal $\ell$-modular representations. To prove this, we use the Bushnell–Kutzko construction to reduce the problem to finite group theory.

We first recall the Green classification of all cuspidal $\ell$-adic representations of ${\text {GL}}_m(k_F)$, where $k_F$ is a finite field of characteristic p, and the associated results of James on reduction modulo $\ell$. Let $k_E$ be a finite extension of $k_F$ of degree m. The group ${\text {Hom}}(k_E^\times ,\overline{\mathbb {Q}_{\ell }}^\times )$ is cyclic of order $|k_E^\times |$, and is the direct product of its $\ell$-singular part ${\text {Hom}}(k_E^\times ,\overline{\mathbb {Q}_{\ell }}^\times )_s$ and its $\ell$-regular part ${\text {Hom}}(k_E^\times ,\overline{\mathbb {Q}_{\ell }}^\times )_r$. The map $r_{\ell }$ is a surjective morphism

$$\begin{aligned} {\text {Hom}}(k_E^\times ,\overline{\mathbb {Q}_{\ell }}^\times )\rightarrow {\text {Hom}}(k_E^\times ,\overline{\mathbb {F}_{\ell }}^\times ) \end{aligned}$$

with kernel ${\text {Hom}}(k_E^\times ,\overline{\mathbb {Q}_{\ell }}^\times )_s$. Let $R=\overline{\mathbb {Q}_{\ell }}$ or $\overline{\mathbb {F}_{\ell }}$, a character $\chi \in {\text {Hom}}(k_E^\times ,R^\times )$ is called $(k_E/k_F)$-regular if its ${\text {Gal}}(k_E/k_F)$-orbit is of cardinality $|{\text {Gal}}(k_E/k_F)|$, i.e. if $\chi$ does not factor through the norm of any proper intermediate extension. If we write $G_\chi$ for the fixator of $\chi$ in ${\text {Gal}}(k_E/k_F)$, it is equivalent to say that $G_\chi =\{1\}$.

Theorem 2.33

([8, 12]) Let $k_F$ be a finite field, and $k_E$ be an extension of $k_F$ of degree m.

• 1. There is a surjective map

  $$\begin{aligned} \mu&\mapsto \sigma (\mu ) \end{aligned}$$

  from the set of $(k_E/k_F)$-regular characters of $k_E^\times ~$ to the set isomorphism classes of cuspidal $\ell$-adic representations of ${\text {GL}}_m(k_F)$, such that the preimage of $\sigma (\mu )$ is the ${\text {Gal}}(k_E/k_F)$-orbit of $\mu$.

• 2. The reduction $r_\ell (\sigma (\mu ))$ is cuspidal, and $r_\ell (\sigma (\mu ))=r_\ell (\sigma (\mu '))$ if and only if the $\ell$-regular parts of $\mu$ and $\mu '$, as elements of the group ${\text {Hom}}(k_E^\times ,\overline{\mathbb {Q}_{\ell }}^\times )$, are conjugate under ${\text {Gal}}(k_E/k_F)$. Moreover, $r_\ell (\sigma (\mu ))$ is supercuspidal if and only if the $\ell$-regular part of $\mu$ is $(k_E/k_F)$-regular. Finally, any cuspidal $\ell$-modular representation of ${\text {GL}}_m(k_F)$ lifts to a cuspidal $\ell$-adic representation $\sigma (\mu )$.

Let $\chi \in {\text {Hom}}(k^\times ,\overline{\mathbb {Q}_{\ell }})$, $\chi _r$ be the $\ell$-regular part of $\chi$, and $\chi _s$ be the singular part which has order a power of $\ell$, so that we can write $\chi =\chi _s\chi _r$ uniquely. As $G_\chi =G_{\chi _s}\cap G_{\chi _r}$, we notice the following obvious fact:

Lemma 2.34

The $\chi$ character is $(k_E/k_F)$-regular if and only if $G_{\chi _s}\cap G_{\chi _r}=\{1\}$.

We now make a first step towards computing the L-factors of non-banal supercuspidal representations. We continue with the notations of Theorem 2.33.

Proposition 2.35

Let $\overline{\sigma }$ be a cuspidal $\ell$-modular representation of ${\text {GL}}_m(k_F)$, and $\sigma (\chi )$ be a cuspidal lift of $\overline{\sigma }$. Then if the generators of the cyclic $\ell$-group ${\text {Hom}}(k_E^\times ,\overline{\mathbb {Q}_{\ell }}^\times )_s$ are not in the same $G_{\chi _r}$-orbit, then $\overline{\sigma }$ has two non-isomorphic cuspidal lifts. This implies that if $\ell$ divides $|k_E^\times |$ and $\overline{\sigma }$ is supercuspidal, then it has two non-isomorphic lifts.

Proof

Let $\sigma (\chi )$ be a cuspidal representation lifting $\overline{\sigma }$ and write $\chi =\chi _s\chi _r$. As $\chi$ is regular, one has $G_{\chi _s}\cap G_{\chi _r}=\{1\}$. But if $\mu$ is any generator of ${\text {Hom}}(k_E^\times ,\overline{\mathbb {Q}_{\ell }}^\times )_s$, one has $G_\mu \subset G_{\chi _s}$, so that $\mu \chi _r$ is regular as well. Moreover, $\sigma (\mu \chi _r)$ lifts $\overline{\sigma }$. But if two generators $\mu$ and $\mu '$ of ${\text {Hom}}(k_E^\times ,\overline{\mathbb {Q}_{\ell }}^\times )_s$ are in different $G_{\chi _r}$-orbits, then clearly $\chi _1=\mu \chi _r$ and $\chi _2=\mu '\chi _r$ are in different ${\text {Gal}}(k_E/k_F)$-orbits, and $\sigma (\chi _1)$ and  $\sigma (\chi _2)$ are two non-isomorphic lifts of $\overline{\sigma }$. Now if $\overline{\sigma }$ is supercuspidal, then the character $\chi _r$ is $(k_E/k_F)$-regular, hence $G_{\chi _r}$ is trivial. Hence, our second assertion follows when $|{\text {Hom}}(k_E^\times ,\overline{\mathbb {Q}_{\ell }}^\times )_s|\geqslant 3$, because in this case ${\text {Hom}}(k_E^\times ,\overline{\mathbb {Q}_{\ell }}^\times )_s$ has at least 2 generators. Now when $|{\text {Hom}}(k_E^\times ,\overline{\mathbb {Q}_{\ell }}^\times )_s|=2$, the generator $\mu$ of ${\text {Hom}}(k_E^\times ,\overline{\mathbb {Q}_{\ell }}^\times )_s$ is the only element of order 2 in ${\text {Hom}}(k_E^\times ,\overline{\mathbb {Q}_{\ell }}^\times )$; hence, it is fixed by ${\text {Gal}}(k_E/k_F)$. In particular $\mu \chi _r$ is regular as well, but in a different ${\text {Gal}}(k_E/k_F)$-orbit, and we conclude as before. $\square$

Remark 2.36

Notice that in contrast to Proposition 2.35 a non-banal cuspidal non-supercuspidal representation may only have one lift, for example for any $\ell \ne 2$, ${\text {GL}}_2(\mathbb {F}_2)$ has only one isomorphism class of cuspidal $\ell$-adic representation $\sigma$, which corresponds to unique ${\text {Gal}}(\mathbb {F}_4/\mathbb {F}_2)$-orbit any non-trivial characters of $\mathbb {F}_4^\times \simeq \mathbb {Z}/3\mathbb {Z}$. Hence, any cuspidal $\ell$-modular representation of ${\text {GL}}_2(\mathbb {F}_2)$ has exactly one lift. The only $\ell$ which divides $\mathbb {F}_4^\times ~$ is 3, in which case $r_3(\sigma )$ is cuspidal non-supercuspidal.

In [20, Chapitre III], the Bushnell–Kutzko construction of all cuspidal $\ell$-adic representations of $G_n$, is extended to $\ell$-modular representations. Let $R=\overline{\mathbb {Q}_{\ell }}$ or $\overline{\mathbb {F}_{\ell }}$, and $\pi$ be a cuspidal R-representation of $G_n$.

• 1. By [3, Chapter 6], [20, Chapitre III], the representation $\pi$ contains a maximal extended simple type $(\mathbf {J},\Lambda )$, ($\mathbf {J}$ is a compact mod-centre open subgroup of $G_n$ and $\Lambda$ is an irreducible R-representation of $\mathbf {J}$ of finite dimension) and we have $\pi \simeq {\text {ind}}_{\mathbf {J}}^{G_n}(\Lambda )$. Moreover, any representation compactly induced from an extended maximal simple type is cuspidal and a cuspidal representation determines a maximal simple type $(\mathbf {J},\Lambda )$ up to conjugation in $G_n$ (the so-called intertwining implies conjugacy property).

• 2. Let $(\mathbf {J},\Lambda )$ be an extended maximal simple type for $\pi$. The group $\mathbf {J}$ is defined via a field extension $E=F[\beta ]$ of F in $\mathcal {M}_n$ such that $\mathbf {J}=E^\times \ltimes J$ with J the maximal compact subgroup of $\mathbf {J}$. The restriction $\lambda$ of $\Lambda$ to J decomposes (non-uniquely) as $\lambda = \kappa \otimes \sigma$, with $\kappa$ a $\beta$-extension, and $\sigma$ an irreducible cuspidal R-representation of $J/J^1\simeq G_m(k_E)$. The pair $(J,\lambda )$ is called a maximal simple type. By [17, Lemmas 6.1 and 6.8], the representation $\pi$ is supercuspidal if and only if $\sigma$ is supercuspidal.

• 3. The ramification index e of $\pi$, is the ramification index of the extension E / F above, which is well defined as intertwining of extended maximal simple types implies conjugacy.

• 4. Let $\pi$ be a cuspidal $\ell$-modular representation and e be the ramification index of $\pi$. By [17, Lemme 5.3 and Section 6], the representation $\pi$ is banal if and only if $q^{n/e}\not \equiv 1[\ell ]$ and, a banal cuspidal $\ell$-modular representation is necessarily supercuspidal.

• 5. Let $\pi$ be a cuspidal $\ell$-modular representation. If $\pi$ contains an extended maximal simple type $(\mathbf {J},\Lambda )$, then one can lift $(\mathbf {J},\Lambda )$ to an integral $\ell$-adic maximal extended type $(\mathbf {J},\widetilde{\Lambda })$, so that the representation $\tau ={\text {ind}}_\mathbf{{J}}^{G_n} {\widetilde{\Lambda }}$ is a cuspidal lift of $\pi$ and all lifts are of this form. Moreover, if $\lambda =\Lambda \mid _J=\kappa \otimes \sigma$, then $\kappa$ lifts to a $\beta$-extension $\widetilde{\kappa }$ and $\sigma$ lifts to a cuspidal representation $\widetilde{\sigma }$, so that $(J,\widetilde{\kappa }\otimes \widetilde{\sigma })$ is a maximal simple type in $(\mathbf {J},\widetilde{\Lambda })$

Remark 2.37

For a cuspidal R-representation $\pi$ of $G_n$, the integer $f(\pi )$ we mentioned in Sect. 2.5, is defined by $f(\pi )=n/e$ where e is the ramification index of $\pi$.

We can now prove the result on lifting supercuspidal $\ell$-modular representations of $G_n$ we need later:

Proposition 2.38

Let $\pi$ be a non-banal supercuspidal $\ell$-modular representation of $G_n$. There exists two lifts of $\pi$ which are not isomorphic by twisting by unramified characters.

Proof

Let $(J,\kappa \otimes \sigma )$ be a simple type contained $\pi$, then $\sigma$ is a supercuspidal representation of $J/J^1\simeq {\text {GL}}_m(k_E)$. By Proposition 2.35, it has two non-isomorphic cuspidal lifts $\tau$ and $\tau '$. Let $\widetilde{\kappa }$ be a $\beta$-extension lifting $\kappa$ such that $(J,\widetilde{\kappa }\otimes \tau )$ and $(J,\widetilde{\kappa }\otimes \tau ')$ are simple types for two cuspidal lifts of $\pi$ which we write as $\rho$ and $\rho '$. It follows from [17, Corollaire 5.5], that $\rho$ and $\rho '$ are not isomorphic by twisting by unramified characters, as [ibid.] gives an equivalent condition for cuspidal representations such as these to be isomorphic after twisting by an unramified character: that $\mathbf {K}(\rho )={\text {Hom}}_{J^1}(\widetilde{\kappa },\rho )$ and $\mathbf {K}(\rho ')={\text {Hom}}_{J^1}(\widetilde{\kappa },\rho ')$ are isomorphic as representations of $G_m(k_E)$; and by [17, Lemme 5.3] we have $\mathbf {K}(\rho )=\tau$ and $\mathbf {K}(\rho ')=\tau '$. $\square$

3 Rankin–Selberg local factors for representations of Whittaker type

The theory of derivatives [1, 2] being valid in positive characteristic (see Sect. 2.3) and equipped with the theory of R-Haar measures (see Sect. 2.2), means we can now safely follow [9] to define L-factors and $\gamma$-factors. However, as mentioned in the introduction as we do not have a Langlands quotient theorem at our disposal, we restrict our attention to representations of Whittaker type (see Sect. 2.5). Of independent interest, we give a different shorter proof than [ibid.] on multiplicity one of $\gamma$-factors in Proposition 3.7, relying on derivatives rather than invariant distributions. In the main result of the section, Theorem 3.13, we show that the $\ell$-modular $\gamma$-factor of a pair of representations always equals the reduction modulo $\ell$ of the $\ell$-adic $\gamma$-factor of any Whittaker lifts, and that the L-factor of this pair divides the reduction modulo $\ell$ of the $\ell$-adic L-factor of any Whittaker lifts.

3.1 Definition of the L-factors

We first recall the asymptotics of Whittaker functions obtained in [10, Proposition 2.2]. For $1\leqslant i\leqslant n$, we write $Z_i$ for subgroup $\{{\text {diag}}(tI_i,I_{n-i}),t\in F^\times \}$ of $G_n$. The diagonal torus $A_n$ of $G_n$ is the direct product $Z_1\times \cdots \times Z_n$.

Lemma 3.1

Let $\pi$ be an R-representation of Whittaker type of $G_n$. For each $1\leqslant i\leqslant n-1$, there is a finite family $X_i(\pi )$ of characters of $Z_i$, such that if W is a Whittaker function in $W(\pi ,\theta )$, then its restriction $W(z_1,\dots ,z_{n-1})$ to $A_{n-1}=Z_1\times \cdots \times Z_{n-1}$ is a sum of functions of the form

$$\begin{aligned} \Phi (z)\prod _{i=1}^{n-1}\chi _i(z_i)v(z_i)^{m_i} \end{aligned}$$

for $\chi _i\in X_i(\pi )$, integers $m_i\geqslant 0$, and $\Phi \in \mathcal {C}_c^\infty (F^{n-1})$.

The proof of Jacquet–Piatetski-Shapiro–Shalika in [ibid.] applies mutatis mutandis for $\ell$-modular representations.

Remark 3.2

For $1\leqslant i \leqslant n-1$, we can take $X_i(\pi )$ to be the family of central characters (restricted to $Z_i$) of the irreducible components of the (non-normalised) Jacquet module $\pi _{N_{i,n-i}}$. We denote by $X_n(\pi )$ the singleton $\{\omega _\pi \}$. We denote by $E_i(\pi )$, the family of central characters (restricted to $Z_i$) of the irreducible components of the normalised Jacquet module $\pi _{N_{i,n-i}}$, for $1\leqslant i\leqslant n-1$, and let $E_n(\pi )=X_n(\pi )$. The family $E_i(\pi )$ is obtained from $X_i(\pi )$ by multiplication by an unramified character of $Z_i$, in particular, if $R=\overline{\mathbb {Q}_{\ell }}$, the characters in $E_i(\pi )$ are integral if and only if those in $X_i(\pi )$ are integral.

Proposition 3.3

Let $\pi$ be an R-representation of Whittaker type of $G_n$, and $\pi '$ an R-representation of Whittaker type of $G_m$, for $m\leqslant n$.

• The case $n=m$. Let $W\in W(\pi ,\theta )$, $W'\in W(\pi ',\theta ^{-1})$, and $\Phi \in \mathcal {C}_c^\infty (F^n)$. Then, for every $k\in \mathbb {Z}$, the coefficient

  $$\begin{aligned}c_k(W,W',\Phi )=\int _{N_n\backslash G_n^k} W(g)W'(g)\Phi (\eta g)dg \end{aligned}$$

  is well defined, and vanishes for $k<<0$.

• The case $m\leqslant n-1$. For $0\leqslant j \leqslant n-m-1$, let $W\in W(\pi ,\theta )$, and $W'\in W(\pi ',\theta ^{-1})$. Then for every $k\in \mathbb {Z}$, the coefficient

  $$\begin{aligned} c_k(W,W';j)=\int _{\mathcal {M}_{j,m}}\int _{N_m\backslash G_m^k} W\begin{pmatrix} g &{} &{} \\ x &{} I_j &{} \\ &{} &{} I_{n-m-j}\end{pmatrix} W'(g)dgdx \end{aligned}$$

  is well defined, and vanishes for $k<<0$. When $m=n-1$, we will simply write $c_k(W,W')$ for $c_k(W,W';0)$.

Proof

The only thing to check is that the coefficients in the statement are well defined, i.e. finite sums, and zero for k negative enough. This is a consequence of Iwasawa decomposition together with Corollary 2.6, and Lemma 3.1 for the case $m\geqslant n-1$, and, in the case $m\leqslant n-2$, that the map $W\small {\begin{pmatrix} g &{} &{} \\ x &{} I_j &{} \\ &{} &{} I_{n-m-j}\end{pmatrix}}$ has compact support with respect to x, independently of g, by [9, Lemma 6.2]. $\square$

Following Proposition 3.3, we now can define our Rankin–Selberg formal series.

Definition 3.4

• The case $n=m$. Under the same notation as Proposition 3.3, we define the following formal Laurent series

  $$\begin{aligned} I(X,W,W',\Phi )=\sum _{k\in \mathbb {Z}} c_k(W,W',\Phi )X^k \in R((X)). \end{aligned}$$

• The case $m\leqslant n-1$. Under the same notation as Proposition 3.3, we define the following formal Laurent series

  $$\begin{aligned} I(X,W,W';j)=\sum _{k\in \mathbb {Z}} c_k(W,W';j)q^{k(n-m)/2}X^k \in R((X)). \end{aligned}$$

  When $m=n-1$, we will simply write $I(X,W,W')$ for $I(X,W,W';0)$.

The L-factors we study are defined by the following theorem.

Theorem 3.5

Let $\pi$ be an R-representation of Whittaker type of $G_n$, and $\pi '$ an R-representation of Whittaker type of $G_m$, for $1\leqslant m\leqslant n$.

• If $n=m$, the R-submodule spanned by the Laurent series $I(X,W,W',\Phi )$ as W varies in $W(\pi ,\theta )$,  $W'$ varies in $W(\pi ',\theta ^{-1})$, and $\Phi$ varies in $\mathcal {C}_c^\infty (F^{n})$, is a fractional ideal $I(\pi ,\pi ')$ of $R[X^{\pm 1}]$, and it has a unique generator which is an Euler factor $L(X,\pi ,\pi ')$.

• If $1\leqslant m\leqslant n-1$, fix j between 0 and $n-m-1$. The R-submodule spanned by the Laurent series $I(X,W,W';j)$ as W varies in $W(\pi ,\theta )$, $W'$ varies in $W(\pi ',\theta ^{-1})$, is a fractional ideal $I(\pi ,\pi ')$ of $R[X^{\pm 1}]$, is independent of j, and it has a unique generator which is an Euler factor $L(X,\pi ,\pi ')$.

Proof

We treat the case $m\leqslant n-2$, the case $m\geqslant n-1$ is totally similar. First we want to prove that our formal series in fact belong to R(X). In this case, the coefficient $c_k(W,W';j)$ is equal to

$$\begin{aligned} \int _{(K_m\cap B_m)\backslash K_m}\int _{A_m^k}\int _{\mathcal {M}_{j,m}} W\begin{pmatrix} ask &{} &{} \\ x &{} I_j &{} \\ &{} &{} I_{n-m-j}\end{pmatrix} W'(ask)\delta _{B_m}(a)^{-1}dxdadk, \end{aligned}$$

which, by smoothness of W and $W'$, we can write as a finite sum

$$\begin{aligned} \sum _i \int _{A_m^k} W_i\begin{pmatrix} a &{} \\ &{} I_{n-m}\end{pmatrix} W'_i(a)\delta _{B_m}(a)^{-1}da, \end{aligned}$$

with functions $W_i\in W(\pi ,\theta )$ and $W'_i\in W(\pi ',\theta ^{-1})$. For W and $W'$ in $W(\pi ,\theta )$, let

$$\begin{aligned} b_k(W,W')=\int _{A_m^k} W\begin{pmatrix} a &{} \\ &{} I_{n-m}\end{pmatrix} W'(a)\delta _{B_m}(a)^{-1}da, \end{aligned}$$

it is thus enough to check that

$$\begin{aligned} J(W,W')=\sum _{k\in \mathbb {Z}} b_k(W,W')q^{(n-m)/2}X^k \end{aligned}$$

belongs to R(X). Following the proof of [9] we see that, by Lemma 3.1, they belong to $\frac{1}{P(X)}R[X^{\pm 1}]$, where P(X) is a suitable power of the product over the unramified characters  $\chi _i$ in $E_i(\pi )$ for $1\leqslant i \leqslant n$ and the unramified characters $\mu _j$ in $E_j(\pi ')$ for $1\leqslant j \leqslant m$ of the Tate L-factors $L(X,\chi _i\mu _j)$. By [15], this factor is equal to 1 if $R=\overline{\mathbb {F}_{\ell }}$, and $q \equiv 1[\ell ]$, and is equal to $1/(1-\chi _i\mu _j(\varpi )X)$ otherwise. The other properties follow immediately from [9]. $\square$

The proof of Theorem 3.5 implies the following corollary.

Corollary 3.6

• If $\pi$ and $\pi '$ are $\ell$-adic representations of Whittaker type of $G_n$ and $G_m$, then $1/L(X,\pi ,\pi ')\in \overline{\mathbb {Z}_{\ell }}[X]$.

• If $\pi$ and $\pi '$ are $\ell$-modular representations of Whittaker type of $G_n$ and $G_m$, and $q\equiv 1 [\ell ]$, then $L(X,\pi ,\pi ')=1$.

Proof

By our assertion at the end of the proof of Theorem 3.5, the polynomial $\mathcal {Q}=1/L(X,\pi ,\pi ')$ divides (in $R[X^{\pm 1}]$, hence in R[X]) a power of the product $\mathcal {P}$ of the polynomials $1/L(X,\chi _i\mu _j)$ over the set of unramified characters $\chi _i$ in $E_i(\pi )$ for $1\leqslant i \leqslant n$ and unramified characters $\mu _j$ in $E_j(\pi ')$ for $1\leqslant j \leqslant m$. We already noticed that $\mathcal {P}$ must be 1 if $R=\overline{\mathbb {F}_{\ell }}$ and $q\equiv 1[\ell ]$, which proves our assertion in this case. In general, $\mathcal {P}$ belongs to $\overline{\mathbb {Z}_{\ell }}[X]$, with constant term 1, as so do the polynomials $1/L(X,\chi _i\mu _j)$. Let $\mathcal {A}$ be the quotient $\mathcal {P}/\mathcal {Q}$ in R[X]. We have $\mathcal {P}=\mathcal {A}\mathcal {Q}$ in R[X], with $\mathcal {P}(0)=\mathcal {Q}(0)=1$ (notice that the constant term of $\mathcal {Q}$ is 1 by definition of an Euler factor), and $\mathcal {P}\in \overline{\mathbb {Z}_{\ell }}[X]$. This equality in fact takes place in K[X], for K a finite extension (spanned by the coefficients of $\mathcal {P}$, $\mathcal {A}$, and $\mathcal {Q}$) of $\mathbb {Q}_\ell$ such that $\mathcal {P}\in \mathfrak {o}_K[X]$ (where $\mathfrak {o}_K=\overline{\mathbb {Z}_{\ell }}\cap K$ is the ring of integers of K). As $\mathfrak {o}_K[X]$ is a unique factorisation domain, the fact that $\mathcal {P}(0)=\mathcal {Q}(0)=\mathcal {A}(0)=1$ implies that $\mathcal {A}$ and $\mathcal {Q}$ are in fact in $\overline{\mathbb {Z}_{\ell }}[X]$. $\square$

3.2 The functional equation

We have defined Rankin–Selberg L-factors of pairs of representations of Whittaker type, we now need to show that these satisfy a local functional equation. By identifying $F^n$ with $\mathcal {M}_{1,n}$, the space $\mathcal {C}_c^\infty (F^n)$ provides a smooth representation $G_n$ by right translation, which we denote by $\rho$. We also denote by $\rho$ the action by right translation of $G_n$ on any space of functions. For $a\in R[X^{\pm 1}]$, we denote by $\chi _a$ the character in ${\text {Hom}}(G_n,R[X^{\pm 1}]^\times )$ defined by $\chi _a:g\mapsto a^{v(det(g))}$. In particular, we have $\nu =\chi _{q^{1/2}}$ is the absolute value of the determinant.

Let $\pi$ be an R-representation of Whittaker type of $G_n$, and $\pi '$ be an R-representation of Whittaker type of $G_m$. If $m=n$, we write

$$\begin{aligned} D(\pi ,\pi ',\mathcal {C}_c^\infty (F^n))={\text {Hom}}_{G_n}(\pi \otimes \pi '\otimes \mathcal {C}_c^\infty (F^n),\chi _X), \end{aligned}$$

for the space of R-linear maps, $\mathfrak {L}:\pi \times \pi ' \times \mathcal {C}_c^\infty (F^n)\rightarrow R[X^{\pm 1}]$, satisfying

$$\begin{aligned} \mathfrak {L}(\rho (h)W,\rho (h)W',\rho (h)\Phi )=X^k\mathfrak {L}(W,W',\Phi ) \end{aligned}$$

for all $W\in W(\pi ,\theta )$, $W'\in W(\pi ',\theta ^{-1})$, $\Phi \in \mathcal {C}_c^\infty (F^n)$, and $h\in G_n^k.$ If $m\leqslant n-1$, we write

$$\begin{aligned} D(\pi ,\pi ')={\text {Hom}}_{G_{m}U_{m+1,n-m-1}}(\pi \otimes \pi ',\chi _{q^{(n-m)/2}X}\otimes \theta ), \end{aligned}$$

for the space of R-linear maps, $\mathfrak {L}:\pi \times \pi \rightarrow R[X^{\pm 1}]$, satisfying

$$\begin{aligned} \mathfrak {L}(\rho (h)W,\rho (h)W')=q^{k(n-m)/2}X^k\mathfrak {L}(W,W',\Phi ), \quad \mathfrak {L}(\rho (u)W,W')\!=\!\theta (u)\mathfrak {L}(W,W') \end{aligned}$$

for all $W\in W(\pi ,\theta )$, $W'\in W(\pi ',\theta ^{-1})$, $h\in G_m^k$, and $u\in U_{m+1,n-m-1}$. We denote by $\mathcal {C}_{c,0}^\infty (F^n)$ the subspace of $\mathcal {C}_{c}^\infty (F^n)$ which is the kernel of the evaluation map $Ev_0:\Phi \mapsto \Phi (0)$.

Proposition 3.7

The spaces $D(\pi ,\pi ',\mathcal {C}_c^\infty (F^n))$ and $D(\pi ,\pi ')$ are free $R[X^{\pm 1}]$-modules of rank 1.

The proof in the complex case of Jacquet–Piatetski-Shapiro–Shalika in [9] is long. Some results obtained in the complex case [ibid.] using invariant distributions can be obtained quicker using derivatives which is how we proceed.

Proof

We start with the case $n=m$. The map

$$\begin{aligned} (W,W',\Phi )\mapsto I(X,W,W',\Phi )/L(X,\pi ,\pi ') \end{aligned}$$

is a nonzero element of $D(\pi ,\pi ',\mathcal {C}_c^\infty (F^n))$; hence, we only need to show that $D(\pi ,\pi ',\mathcal {C}_c^\infty (F^n))$ is a free $R[X^{\pm 1}]$-module of rank at most 1. $\square$

We have an exact sequence of representations of $G_n$

$$\begin{aligned} 0\rightarrow \mathcal {C}_{c,0}^\infty (F^n) \rightarrow \mathcal {C}_c^\infty (F^n) \rightarrow \mathbf {1}\rightarrow 0. \end{aligned}$$

We tensor this sequence by $\pi \otimes \pi '$ and, as $\pi \otimes \pi '$ is flat as an R-vector space, we obtain

$$\begin{aligned} 0\rightarrow \pi \otimes \pi ' \otimes \mathcal {C}_{c,0}^\infty (F^n) \rightarrow \pi \otimes \pi '\otimes \mathcal {C}_c^\infty (F^n) \rightarrow \pi \otimes \pi '\rightarrow 0. \end{aligned}$$

By considering central characters, it is clear that the space ${\text {Hom}}_{G_n}(\pi \otimes \pi ',\chi _X)=0$. Applying ${\text {Hom}}_{G_n}(~,\chi _X)$ which is left exact, we obtain that ${\text {Hom}}_{G_n}(\pi \otimes \pi ' \otimes \mathcal {C}_c^\infty (F^n),\chi _X)$ is a $R[X^{\pm 1}]$-submodule of ${\text {Hom}}_{G_n}(\pi \otimes \pi ' \otimes \mathcal {C}_{c,0}^\infty (F^n),\chi _X)$. Hence, it is sufficient to check that ${\text {Hom}}_{G_n}(\pi \otimes \pi ' \otimes \mathcal {C}_{c,0}^\infty (F^n),\chi _X)$ is of rank at most 1.

Now, we have an isomorphism between $\mathcal {C}_{c,0}^\infty (F^n)$ with $G_n$ acting via right translation and ${\text {ind}}_{P_n}^{G_n}(\delta _{P_n}^{1/2})$; hence, we have

$$\begin{aligned} {\text {Hom}}_{G_n}(\pi \otimes \pi ' \otimes \mathcal {C}_{c,0}^\infty (F^n),\chi _X)&\simeq {\text {Hom}}_{G_n}(\pi \otimes \pi ',\chi _X {\text {Ind}}_{P_n}^{G_n}(\delta _{P_n}^{-1/2})),\\&\simeq {\text {Hom}}_{P_n}(\pi \otimes \pi ',\chi _X \delta _{P_n}^{-1}) \end{aligned}$$

by Frobenius reciprocity. Now, by the theory of derivatives (see Sect. 2.3), $\pi$ and $\pi '$, as $P_n$-modules, are of finite length, with irreducible subquotients of the form $(\Phi ^+)^k\Psi ^+ (\rho )$, for $\rho$ an irreducible representation of $G_{n-k-1}$ and k between 0 and $n-1$. Moreover, $(\Phi ^+)^{n-1}\Psi ^+ (\mathbf {1})$ appears with multiplicity 1, as a submodule. By Proposition 2.11, the space

$$\begin{aligned}&{\text {Hom}}_{P_n}((\Phi ^+)^k\Psi ^+ (\rho ) \otimes (\Phi ^+)^j\Psi ^+ (\rho '),\chi _X \delta _{P_n}^{-1})\\&\quad = {\text {Hom}}_{P_n}((\Phi ^+)^k\Psi ^+ (\rho ) \otimes (\Phi ^+)^j\Psi ^+ (\chi _X^{-1} \delta _{P_n}\rho ') ,\mathbf {1}) \end{aligned}$$

is zero, except when $j=k$, in which case it is isomorphic to ${\text {Hom}}_{G_k}( \rho \otimes \rho ',\chi _X \nu ^{-1}))$. If $\rho$ and $\rho '$ are irreducible and $k\geqslant 1$, by considering central characters, the space ${\text {Hom}}_{G_k}( \rho \otimes \rho ',\chi _X \nu ^{-1})$ is zero. Thus ${\text {Hom}}_{P_n}(\pi \otimes \pi ',\chi _X \delta _{P_n}^{-1})$ is a $R[X^{\pm 1}]$-submodule of ${\text {Hom}}_{G_0}( \mathbf {1} \otimes \mathbf {1},\chi _X )\simeq R[X^{\pm 1}]$. This ends the proof in the case $n=m$, as $R[X^{\pm 1}]$ is principal.

We now consider the case $m\leqslant n-1$. Again, the space $D(\pi ,\pi ')$ is nonzero as it contains the map $(W,W')\mapsto I(X,W,W')/L(X,\pi ,\pi ')$, we will show that it injects into $R[X^{\pm 1}]$, which will prove the statement. Let L be in $D(\pi ,\pi ')$, by definition, the map L-factors through $\tau \times \pi '$, where $\tau$ is the quotient of $\pi$ by its subspace spanned by $\pi (u)W-\theta (u) W$ for $u\in U_{m+1,n-m-1}$ and $W\in W(\pi ,\theta )$. Hence, $\tau$ is nothing other than the space of $(\Phi ^+)^{n-m-1}(\pi )$. Taking into account the normalisation in the definition of the derivatives, we obtain the following injection:

$$\begin{aligned} {\text {Hom}}_{G_{m}U_{m+1,n-m-1}}(\pi \otimes \pi ',\chi _{q^{(n-m)/2}X}\otimes \theta )\hookrightarrow {\text {Hom}}_{G_m}((\Phi ^-)^{n-m-1}(\pi ) \otimes \pi ',\chi _{q^{1/2}X}). \end{aligned}$$

We next prove the following lemma.

Lemma 3.8

If $\sigma$ is an irreducible R-representation of $P_{m}$, then

$$\begin{aligned} {\text {Hom}}_{G_m}(\Phi ^+(\sigma )\otimes \pi ',\chi _{q^{1/2}X})\simeq {\text {Hom}}_{P_m}(\sigma \otimes \pi ',\chi _{X}). \end{aligned}$$

If $\sigma$ is an irreducible R-representation of $G_{m}$, then

$$\begin{aligned} {\text {Hom}}_{G_m}(\Psi ^+(\sigma )\otimes \pi ',\chi _{q^{1/2}X})\simeq {\text {Hom}}_{G_m}(\sigma \otimes \pi ',\chi _{X})=\{0\}. \end{aligned}$$

Proof of the Lemma

We first prove the second assertion. By definition of $\Psi ^+$, the $R[X^{\pm 1}]$-module ${\text {Hom}}_{G_m}(\Psi ^+(\sigma )\otimes \pi ',\chi _{q^{1/2}X})$ is equal to ${\text {Hom}}_{P_{m+1}}(\Psi ^+(\sigma )\otimes \Psi ^+(\pi '),\chi _{X}))$, which is itself isomorphic to ${\text {Hom}}_{G_m}(\sigma \otimes \pi ',\chi _{X})$ by Proposition 2.11. As $\sigma$ is irreducible, it has a central character, and thus ${\text {Hom}}_{G_m}(\sigma ,\chi _{X})=\{0\}$.

For the first assertion, as $P_{m+1}=G_m (P_m U_{m+1})$, we have an isomorphism $\Phi ^+(\sigma )\mid _{G_m}\simeq {\text {ind}}_{P_m}^{G_m}(\sigma )$ by Mackey theory (cf. [2, Theorem 5.2]). Hence, we obtain

$$\begin{aligned}&{\text {Hom}}_{G_m}(\Psi ^+(\sigma )\otimes \pi ',\chi _{q^{1/2}X})\\&\quad \simeq {\text {Hom}}_{G_m}({\text {ind}}_{P_m}^{G_m}(\sigma )\otimes \pi ',\chi _{q^{1/2}X})\simeq {\text {Hom}}_{P_m}(\sigma \otimes \pi ',\chi _{X}), \end{aligned}$$

the last isomorphism by Frobenius reciprocity. $\square$

Now, $(\Phi ^-)^{n-m-1}(\pi )$ is a $P_{m+1}$-module of finite length, and as $\pi$ is of Whittaker type, it contains $(\Phi ^+)^{m-1}\Psi ^+(\mathbf {1})$ as a submodule, the latter’s multiplicity being 1 as a composition factor. By the theory of derivatives, all of the other irreducible subquotients are either of the form $\Psi ^+(\sigma )$, with $\sigma$ an irreducible representation of $G_m$, or of the form $\Phi ^+(\sigma )$, with $\sigma$ an irreducible representation of $P_m$ of the form $(\Phi ^+)^{m-j-1}\Psi ^+(\sigma ')$, with $\sigma '$ a representation of $G_j$, for some $j\geqslant 1$. By Lemma 3.8, ${\text {Hom}}_{G_m}(\Psi ^+(\sigma )\otimes \pi ',\chi _{X})$ is zero. For all subquotients of the form $\Phi ^+(\sigma )$, we have ${\text {Hom}}_{G_m}(\Phi ^+(\sigma )\otimes \pi ',\chi _{q^{1/2}X})\simeq {\text {Hom}}_{P_m}(\sigma \otimes \pi ',\chi _{X})$ by Lemma 3.8.

Lemma 3.9

The $R[X^{\pm 1}]$-module ${\text {Hom}}_{P_m}(\sigma \otimes \pi ',\chi _{X})$ is zero if $\sigma$ is an irreducible R-representation of $P_m$ of the form $(\Phi ^+)^{m-j-1}\Psi ^+(\sigma ')$, with $\sigma '$ an R-representation of $G_j$, for some $j\geqslant 1$, whereas ${\text {Hom}}_{P_m}((\Phi ^+)^{m-1}\Psi ^+(\mathbf {1})\otimes \pi ',\chi _X)\hookrightarrow R[X^{\pm 1}]$.

Proof of the Lemma

As $\pi '$ is of Whittaker type, its restriction to $P_m$ is of finite length, with irreducible subquotients of the form  $(\Phi ^+)^{m-k-1}\Psi ^+(\mu )$, for $\mu$ an irreducible representation of $G_k$. Moreover, the representation $(\Phi ^+)^{m-k-1}\Psi ^+(\mathbf {1})$ occurs with multiplicity 1, and is a submodule. If $\sigma$ is an irreducible representation of $P_m$ of the form $(\Phi ^+)^{m-j-1}\Psi ^+(\sigma ')$, with $\sigma '$ a representation of $G_j$, for some $j\geqslant 1$, then ${\text {Hom}}_{P_m}((\Phi ^+)^{m-h-1}\Psi ^+(\sigma ')\otimes (\Phi ^+)^{m-k-1}\Psi ^+(\mu ),\chi _X)$ is zero by Proposition 2.11 if $j\ne k$ (in particular if $k=1$). Moreover, if $k=j$, by the same Proposition, we have ${\text {Hom}}_{P_m}((\Phi ^+)^{m-j-1}\Psi ^+(\sigma ')\otimes (\Phi ^+)^{m-k-1}\Psi ^+(\mu ),\chi _X)$ is isomorphic to ${\text {Hom}}_{G_j}(\sigma '\otimes \mu ,\chi _X)$, which is zero, by considering central characters. Hence, we have proved the first part of the lemma. If $\sigma =(\Phi ^+)^{m-1}\Psi ^+(\mathbf {1})$, reasoning as above, we see at once that ${\text {Hom}}_{P_m}((\Phi ^+)^{m-1}\Psi ^+(\mathbf {1})\otimes \pi ',\chi _X)$ injects into ${\text {Hom}}_{P_m}((\Phi ^+)^{m-1}\Psi ^+(\mathbf {1})\otimes (\Phi ^+)^{m-1}\Psi ^+(\mathbf {1}),\chi _X)\simeq {\text {Hom}}_{G_0}(\mathbf {1}\otimes \mathbf {1},\chi _X)$, the latter space being isomorphic to $R[X^{\pm 1}]$, and this completes the proof of the lemma. $\square$

All in all, we deduce that ${\text {Hom}}_{G_m}((\Phi ^-)^{n-m-1}(\pi ) \otimes \pi ',\chi _{q^{1/2}X})$ injects as a $R[X^{\pm 1}]$-submodule into ${\text {Hom}}_{P_m}((\Phi ^+)^{m-1}\Psi ^+(\mathbf {1})\otimes \pi ',\chi _{X})$, which itself injects into $R[X^{\pm 1}]$, and this ends the proof of the proposition. $\square$

Remark 3.10

Notice that all the injections defined in the proof of Proposition 3.7 are in fact isomorphisms. This could be viewed directly, or we can simply see that after composing all of them we obtain an isomorphism.

We are now in a position to state the local functional equation and define the Rankin–Selberg $\varepsilon$-factor of a pair of representations of Whittaker type. We recall that an invertible element of $R[X^{\pm 1}]$ is an element of the form $cX^k$, for c in $R^\times$, and k in $\mathbb {Z}$.

Corollary 3.11

Let $\pi$ be an R-representation of Whittaker type of $G_n$, and $\pi '$ be a representation of Whittaker type of $G_m$.

• 1. If $m=n$, there is an invertible element $\varepsilon (X,\pi ,\pi ',\theta )$ of the ring $R[X^{\pm 1}]$ such that for any $W\in W(\pi ,\theta )$, any $W'\in W(\pi ',\theta ^{-1})$, and any $\Phi$ in $\mathcal {C}_c^\infty (F^n)$, we have:

  $$\begin{aligned}\frac{I(q^{-1}X^{-1},\widetilde{W},\widetilde{W'},\widehat{\Phi })}{L(q^{-1}X^{-1},\widetilde{\pi },\widetilde{\pi '})}= c_{\pi '}(-1)^{m-1}\varepsilon (X,\pi ,\pi ',\theta )\frac{I(X,W,W',\Phi )}{L(X,\pi ,\pi ')}.\end{aligned}$$

• 2. If $m\leqslant n-1$, there is an invertible element $\varepsilon (X,\pi ,\pi ',\theta )$ of the ring $R[X^{\pm 1}]$ such that, for any $W\in W(\pi ,\theta )$, any $W'\in W(\pi ',\theta ^{-1})$, and any $0\leqslant j \leqslant n-m-1$, we have:

  $$\begin{aligned}&\frac{I(q^{-1}X^{-1},\rho (w_{m,n-m})\widetilde{W},\widetilde{W'};n-m-1-j)}{L(q^{-1}X^{-1},\widetilde{\pi },\widetilde{\pi '})}\\&\quad =c_{\pi '}(-1)^{m-1}\varepsilon (X,\pi ,\pi ',\theta ) \frac{I(X,W,W';j)}{L(X,\pi ,\pi ')}. \end{aligned}$$

Proof

It is a consequence of Proposition 3.7 if $n=m$, and if $m\leqslant n-1$ with $j=0$, as the functionals on both sides of the equality belong, respectively, to $D(\pi ,\pi ',\mathcal {C}_c^\infty (F^n))$ and $D(\pi ,\pi ')$. For $j\ne 0$, it follows from the case $j=0$ as in the complex setting, cf. [10]. $\square$

We call $\varepsilon (X,\pi ,\pi ',\theta )$ the local $\varepsilon$-factor associated with $\pi ,\pi '$, and $\theta$, and we write

$$\begin{aligned} \gamma (X,\pi ,\pi ',\theta )=\frac{\varepsilon (X,\pi ,\pi ',\theta )L(q^{-1}X^{-1},\widetilde{\pi },\widetilde{\pi '})}{L(X,\pi ,\pi ')}, \end{aligned}$$

for the local $\gamma$-factor associated with $\pi ,\pi '$, and $\theta$.

3.3 Compatibility with reduction modulo $\ell$

Let $\pi$ and $\pi '$ are integral $\ell$-adic representations of Whittaker type of $G_n$ and $G_m$. By Corollary 3.6, we already know that $L(X,\pi ,\pi ')$ is the inverse of a polynomial with integral coefficients, even without the integrality assumption. With the integrality assumption, we now consider the associated $\varepsilon$-factor.

Lemma 3.12

The factor $\varepsilon (X,\pi ,\pi ',\theta )$ is of the form $cX^k$, for c a unit in $\overline{\mathbb {Z}_{\ell }}$.

Proof

We only do the case $n=m$, the case $m\leqslant n-1$ follows mutatis mutandis. By Remark 2.1, whenever W, $W'$ and $\Phi$ have integral values, the Laurent series $I(X,W,W',\Phi )$ and $I(q^{-1}X^{-1},\widetilde{W},\widetilde{W'},\hat{\Phi })$ belong, respectively, to $\overline{\mathbb {Z}_{\ell }}((X))$ and $\overline{\mathbb {Z}_{\ell }}((X^{-1}))$. As the factors $L(X,\pi ,\pi ')$ and $L(q^{-1}X^{-1},\widetilde{\pi },\widetilde{\pi '})$ are the inverse of polynomials in $\overline{\mathbb {Z}_{\ell }}[X^{\pm 1}]$ with constant term 1, the quotient

$$\begin{aligned} I(q^{-1}X^{-1},\widetilde{W},\widetilde{W'},\hat{\Phi })/L(q^{-1}X^{-1},\widetilde{\pi },\widetilde{\pi '}) \end{aligned}$$

which belongs to $R[X^{\pm 1}]$, in fact belongs to $R[X^{\pm 1}]\cap \overline{\mathbb {Z}_{\ell }}((X^{-1}))=\overline{\mathbb {Z}_{\ell }}[X^{\pm 1}],$ and similarly for the quotient  $I(X,W,W',\Phi )/L(X,\pi ,\pi ')$. The functional equation then implies that the scalar c is in $\overline{\mathbb {Z}_{\ell }}$, and applying it twice shows that it is in $\overline{\mathbb {Z}_{\ell }}^\times$. $\square$

If P is an element of $\overline{\mathbb {Z}_{\ell }}[X]$ with nonzero reduction modulo $\ell$, we write $r_{\ell }(P^{-1})$ for $(r_{\ell }(P))^{-1}$. We now prove our first main result.

Let $\pi$ and $\pi '$ be $\ell$-modular representations of Whittaker type of $G_n$ and $G_m$. Let $\tau$ and $\tau '$ be Whittaker lifts of $\pi$ and $\pi '$ (respectively).

Theorem 3.13

We have

$$\begin{aligned} L(X,\pi ,\pi ')\mid r_{\ell }(L(X,\tau ,\tau ')), \end{aligned}$$

and

$$\begin{aligned} \gamma (X,\pi ,\pi ',r_{\ell }(\theta ))=r_{\ell }(\gamma (X,\tau ,\tau ',\theta )). \end{aligned}$$

Proof

We give the proof for $m\leqslant n-1$, and $j=0$, the other cases being similar. By definition, one can write $L(X,\pi ,\pi ')$ as a finite sum $\sum _i I(X,W_i,W_i')$, for $W_i \in W(\pi ,\theta )$ and $W_i' \in W(\pi ',\theta ^{-1})$. By Lemma 2.23, there are Whittaker functions $W_{i,e}\in W_e(\tau ,\mu )$ and $W_{i,e}'\in W_e(\tau ',\mu ^{-1})$ such that $\widetilde{W_{i,e}}\in W_e(\widetilde{\tau },\mu ^{-1})$ and $\widetilde{W_{i,e}'}\in W_e(\widetilde{\tau '},\mu )$ , and such that $W_i=r_{\ell }(W_{i,e})$, and $W_i'=r_{\ell }(W_{i,e}')$. By Remark 2.1, we have $L(X,\pi ,\pi ')=r_{\ell }(\sum _i I(X,W_{i,e},W_{i,e}'))$. As the sum $\sum _i I(X,W_{i,e},W_{i,e}')$ belongs to

$$\begin{aligned} L(X,\tau ,\tau ')\overline{\mathbb {Q}_{\ell }}[X^{\pm 1}]\cap \overline{\mathbb {Z}_{\ell }}((X))=L(X,\tau ,\tau ')\overline{\mathbb {Z}_{\ell }}[X^{\pm 1}], \end{aligned}$$

we obtain that $L(X,\pi ,\pi ')$ belongs to $r_{\ell }(L(X,\tau ,\tau '))\overline{\mathbb {F}_{\ell }}[X^{\pm 1}]$. This proves the first assertion. The equality for $\gamma$-factors follows the functional equation, and Remark 2.1. $\square$

As an immediate corollary of Theorem 3.13, of [9, Proposition 2.13] and of [11, Proposition 4.1, Remark 5.2], we obtain the stability of the local factors. We recall that the level $l(\chi )$ of a character $\chi$ of $F^\times$ is either zero if it is trivial on $\mathfrak {o}^\times$, or the smallest integer $n\geqslant 1$ such that $\chi$ trivial on $1+\mathfrak {p}^n$.

Corollary 3.14

Let $\pi$ and $\pi '$ be a pair of $\ell$-modular representations of Whittaker type of $G_n$ and $G_m$, respectively, then there exists $l_{\pi ,\pi '}\in \mathbb {N}$ such that

$$\begin{aligned} L(X,\pi \otimes \chi ,\pi ')=1, \end{aligned}$$

whenever $l(\chi )\geqslant l_{\pi ,\pi '}$. Moreover, if $\pi _1,\pi _2$ are $\ell$-modular representations of Whittaker type of $G_n$ with equal central characters, there exists $l_{\pi '}\in \mathbb {N}$ such that

$$\begin{aligned} \gamma (X,\pi _1\otimes \chi ,\pi ',\theta )=\gamma (X,\pi _2\otimes \chi ,\pi ',\theta ), \end{aligned}$$

whenever $l(\chi )\geqslant l_{\pi '}$.

Remark 3.15

As for the Godement–Jacquet L-functions (see [15]), we do not always have an equality between $r_{\ell }(L(X,\tau ,\tau '))$ and $L(X,\pi ,\pi ')$. For instance when $q\equiv 1[\ell ]$, we always have $L(X,\pi ,\pi ')=1$. But already, for unramified $\ell$-adic characters $\tau$ and $\tau '$ of $G_1$, we have

$$\begin{aligned} r_{\ell }(L(X,\tau ,\tau '))=1/(1-X). \end{aligned}$$

This will become completely transparent in the generic case by the end of the next section.

4 The inductivity relation and explicit computations

In Sect. 4.1, we obtain the inductivity relation of $\gamma$-factors of representations of Whittaker type by reduction modulo $\ell$. For L-factors, we restrict to pairs of generic representations and obtain again the inductivity relation, and an explicit formula, differing from the $\ell$-adic case due to the presence of non-banal representations. In fact, for generic representations we show that the L-factor only depends on the banal parts of the representations in Theorem 4.19. We also discuss precisely the relation with reduction modulo $\ell$, and obtain in Theorem 4.22 a nice interpretation of the $\ell$-modular L-factor of a pair of generic representations as the greatest common divisor of the L-factors of the pairs of generic $\ell$-adic representations containing our generic pair as subquotients on reduction modulo $\ell$. This also provides a coherent way of defining L-factors of pairs of irreducible $\ell$-modular representations.

4.1 Gamma factors of representations of Whittaker type

Let R be either $\overline{\mathbb {Q}_{\ell }}$ or $\overline{\mathbb {F}_{\ell }}$. Let $\pi _1,\pi _2$ be $\ell$-modular representations of Whittaker type and $\tau _1,\tau _2$ be standard lifts of $\pi _1,\pi _2$, respectively. For $i=1,2$, let $\tau _i'$ be a subquotient of $\tau _i$ of Whittaker type, and $\pi _i'$ be a subquotient of $\pi _i$ of Whittaker type. As the functor

$$\begin{aligned} \pi \mapsto {\text {Hom}}_{N_n}(\pi ,\theta ) \end{aligned}$$

is exact from $\mathfrak {R}_R(G_n)$ to the category of R-vector spaces, by multiplicity one of the Whittaker functional for Whittaker representations, we deduce that the (unique up to scaling) Whittaker functional on $\pi _1'$ is the restriction of that on $\pi _1$ (and similarly for the pairs $(\pi _2',\pi _2)$, $(\tau _1',\tau _1)$, and $(\tau _2',\tau _2)$).

This implies immediately the following inclusions of Whittaker models:

$$\begin{aligned} W(\tau _1',\theta )&\subseteq W(\tau _1,\theta ),&W(\tau _2',\theta ^{-1})&\subseteq W(\tau _2,\theta ^{-1}),\\ W(\pi _1',r_\ell (\theta ))&\subseteq W(\pi _1,r_\ell (\theta )),&W(\pi _2',r_\ell (\theta ^{-1}))&\subseteq W(\pi _2,r_\ell (\theta ^{-1})). \end{aligned}$$

These inclusions being equalities whenever the respective subquotients are actually quotients.

Applying $W\mapsto \widetilde{W}$ (in $\mathcal {C}^\infty (N_{n_i}\backslash G_{n_i})$) to the above inclusions, we obtain the inclusions:

$$\begin{aligned} W(\widetilde{\tau _1}',\theta ^{-1})&\subseteq W(\widetilde{\tau _1},\theta ^{-1}),&W(\widetilde{\tau _2}',\theta )&\subseteq W(\widetilde{\tau _2},\theta ),\\ W(\widetilde{\pi _1}',r_\ell (\theta ^{-1}))&\subseteq W(\widetilde{\pi _1},r_\ell (\theta ^{-1})),&W(\widetilde{\pi _2}',r_\ell (\theta ))&\subseteq W(\widetilde{\pi _2},r_\ell (\theta )). \end{aligned}$$

Thanks to all of the above inclusions of Whittaker models, the functional equation gives the equalities of gamma factors:

$$\begin{aligned} \gamma (X,\pi _1',\pi _2')&=\gamma (X,\pi _1,\pi _2),&\gamma (X,\tau _1',\tau _2')&=\gamma (X,\tau _1,\tau _2). \end{aligned}$$

Now, by Theorem 3.13, we have

$$\begin{aligned} r_\ell (\gamma (X,\tau _1,\tau _2))=\gamma (X,\pi _1,\pi _2), \end{aligned}$$

hence

$$\begin{aligned} r_\ell (\gamma (X,\tau _1',\tau _2'))=\gamma (X,\pi _1',\pi _2'). \end{aligned}$$

In particular, we can take $\tau _i'$ (resp. $\pi _i'$) to be the unique generic subquotient of $\tau _i$ (resp. $\pi _i$) for $i=1,2$.

The following theorem follows by reduction modulo $\ell$ (Theorem 3.13) of the inductivity relation of $\ell$-adic gamma factors in [9, 3.1].

Theorem 4.1

Let $\pi _1$ and $\pi _2$ and $\pi _3$ be $\ell$-modular representations of Whittaker type, of $G_{n_1}$, $G_{n_2}$ and $G_{n_3}$, respectively. Then

$$\begin{aligned} \gamma (X,\pi _1,\pi _2\times \pi _3)=\gamma (X,\pi _1,\pi _2)\gamma (X,\pi _1,\pi _3). \end{aligned}$$

Together with the discussion above, it has the following corollary:

Corollary 4.2

Let $\rho$ and $\rho '$ be cuspidal $\ell$-modular representations of $G_r$ and $G_{r'}$, respectively, and $a\leqslant b$ and $c\leqslant d$ in $\mathbb {Z}$. Then one has

$$\begin{aligned} \gamma (X,St(\rho ,[a,b]),St(\rho ',[c,d]))=\prod _{i=a}^b \prod _{j=c}^d \gamma (X,\nu ^i\rho ,\nu ^j\rho '). \end{aligned}$$

Proof

As $St(\rho ,[a,b])$ and $St(\rho ,[c,d])$ are the unique generic quotients of the induced representations $\nu ^a\rho \times \dots \times \nu ^b \rho$ and $\nu ^c\rho '\times \dots \times \nu ^d\rho '$, respectively, the assertion follows from Theorem 4.1 and the discussion preceding it. $\square$

4.2 Divisibility relations between L-factors of induced representations

In this subsection R can be $\overline{\mathbb {Q}_{\ell }}$ or $\overline{\mathbb {F}_{\ell }}$. We first start with a simple observation, which follows from the inclusions between Whittaker models discussed in the beginning of Sect. 4.1.

Lemma 4.3

Let $\pi$ (resp. $\tau$) be an R-representation of Whittaker type of $G_n$ (resp. $G_m$), and $\pi '$ (resp. $\tau '$) be a subquotient of Whittaker type (for example the unique generic subquotient) of $\pi$ (resp. $\tau$), then $L(X,\pi ',\tau ')$ divides $L(X,\pi ,\tau )$, and it is equal to it if both $\pi '$ and $\tau '$ are quotients of $\pi$ and $\tau$.

We also observe, as in [9, Section 7] (their proof being valid for $R=\overline{\mathbb {F}_{\ell }}$), thanks to the stability of L-factors under highly ramified twists (Corollary 3.14), that the inductivity relation of $\gamma$-factors (Theorem 4.1) implies the following result.

Lemma 4.4

Let $\pi _1$, $\pi _2$ and $\pi _3$ be R-representations of Whittaker type of $G_{n_1}$, $G_{n_2}$ and $G_{n_3}$, respectively, then $L(X,\pi _1\times \pi _2,\pi _3)$ divides $L(X,\pi _1,\pi _3)L(X,\pi _2,\pi _3)$.

We now recall [9, Proposition 9.1]. A small part of the proof of [ibid.] is given in a particular (but in fact very general) case, we give the very slight changes here, to obtain the general proof. We shall also give some useful corollaries.

Lemma 4.5

Let $\pi _1$ and $\pi _2$ be two R-representations of Whittaker type of $G_{n_1}$ and $G_{n_2}$, respectively, and let $\pi =\pi _1\times \pi _2$. Then for any pair $(W_2,\Phi )\in W(\pi _2,\theta )\times \mathcal {C}_c^\infty (F^{n_2},R)$, there is $W\in W(\pi ,\theta )$ such that for $g\in G_{n_2}$, we have

$$\begin{aligned} W({\text {diag}}(g,I_{n_1}))=W_2(g)\Phi (\eta g)\nu (g)^{n_1/2}. \end{aligned}$$

Proof

Because $W(\pi _1\times \pi _2,\theta )$ is equal to $W(W(\pi _1,\theta )\times W(\pi _2,\theta ),\theta )$, we can replace $\pi$ by $\pi '=W(\pi _1,\theta )\times W(\pi _2,\theta )$ to prove the statement. We write P for $P_{(n_1,n_2)}$, and U for its unipotent radical. The group U is isomorphic to $\mathcal {M}=\mathcal {M}(n_1,n_2,F)$ via the map

$$\begin{aligned} u:x\mapsto \begin{pmatrix} I_{n_1} &{} x \\ &{} I_{n_2} \end{pmatrix}. \end{aligned}$$

We can consider $W(\pi _1,\theta )\otimes W(\pi _2,\theta )$ as a P-module (with U acting trivially). We first recall the following standard fact on induced representations of locally profinite groups: the linear map $I:\mathcal {C}_c^\infty (G_n,R)\otimes W(\pi _1,\theta )\otimes W(\pi _2,\theta )\rightarrow \pi '$, defined by the formula

$$\begin{aligned} I(F\otimes W_1 \otimes W_2)(g)=\int _{P} F(pg) \delta _P^{-1/2}(p)\rho (p)^{-1} W_1\otimes W_2 dp, \end{aligned}$$

for $F\in \mathcal {C}_c^\infty (G_n,R)$, $W_1\in W(\pi _1,\theta )$ and $W_2\in W(\pi _2,\theta )$, is a well-defined surjection. Taking F with support in PwU, of the form $F(pwu(x))=\beta (p)\Psi (x)$ with $\beta$ the characteristic function of a small enough compact open subgroup of P (so that this subgroup has a nonzero volume, and fixes $W_1\otimes W_2$), and with $\Psi$ any function in  $\mathcal {C}_c(\mathcal {M},R)$, we obtain the map $f=I(F)$ in $\pi '$, which satisfies, up to nonzero scaling the relation:

$$\begin{aligned} f(w u(x))=\Psi (x) W_1\otimes W_2. \end{aligned}$$

Now the end of the proof is that of [9, Proposition 9.1]. $\square$

The following result also follows from the results of [9].

Lemma 4.6

Let m, n be positive integers, with $m \leqslant n$. If $\pi$ and $\pi '$ are two R-representations of $G_n$ of Whittaker type, and if W and $W'$ belong, respectively, to $W(\pi ,\theta )$ and $W(\pi ',\theta ^{-1})$, the integral

$$\begin{aligned}&I_{(n-m-1)}(X,W,W')\\&\quad =\sum _{k\in \mathbb {Z}}\left( \int _{N_n\backslash G_n^k} W({\text {diag}}(g,I_{n-m}))W'({\text {diag}}(g,I_{n-m})) dg\right) q^{k(n-m)}X^k \end{aligned}$$

belongs to the fractional ideal $I(\pi ,\pi ')$ (see Theorem 3.5).

Proof

Thanks to [9, Lemma 9.2], the proof of which is valid over R, we see that there are $W_0\in W(\pi ,\theta )$ and $W_0'\in W(\pi ',\theta ^{-1})$ such that $I_{(n-m-1)}(X,W,W')=I_{(0)}(X,W_0,W_0')$. Hence, it suffices to prove the lemma when $m=n-1$. In this case, for any $\Phi$ in $\mathcal {C}_c^\infty (F^n)$, we denote by $f_{\Phi }$ the function on F defined as $f_{\Phi }:t\mapsto \Phi (0,\dots ,0,t)$. Using the integration formula of Corollary 2.7, we can write

$$\begin{aligned} I(X,W,W',\Phi )=\int _{(K_n\cap P_n)\backslash K_n}I_{(0)}(X,\rho (k)W,\rho (k)W') I(X^n,c_\pi ,c_{\pi '},f_{\rho (k)\Phi })dk, \end{aligned}$$

where $\rho$ denotes right translation. Writing $K_{n,r}=1+\varpi ^r\mathcal {M}(n,\mathfrak {o})$, and taking $\Phi =\mathbf {1}_{K_{n,r}}$ for r large enough for W and $W'$ to be invariant under $K_{n,r}$, our integral reduces to a nonzero multiple of $I_{(0)}(X,W,W')$, and the result follows. $\square$

We now deduce from Lemmas 4.5 and 4.6, the following proposition, for the proof of which we refer to [5, Proposition 4.4].

Proposition 4.7

Let $\pi _1$ and $\pi _2$ be two R-representations of Whittaker type of $G_{n_1}$ and $G_{n_2}$, respectively, let $\pi =\pi _1\times \pi _2$, and put $n=n_1+n_2$. Let $\pi '$ be a representation of Whittaker type of $G_n$. Then $L(X,\pi _2,\pi ')^{-1}$ divides $L(X,\pi ,\pi ')^{-1}$ in R[X].

It has the following corollary, which we will use several times.

Corollary 4.8

Let $\rho$ be a cuspidal R-representation of $G_m$ ($m\geqslant 1$), and $a< b\leqslant c$ two positive integers such that $c-a+1< e(\rho )$, so that $St(\rho ,[a,b-1])$, $St(\rho ,[b,c])$ and $St(\rho ,[a,c])$ are generic segments. Then, for any $\ell$-modular representations $\pi$ and $\pi '$ of Whittaker type of $G_n$ and $G_{n'}$, respectively, we have

$$\begin{aligned} L(X,St(\rho ,[b,c])\times \pi ,\pi ')\mid L(X,St(\rho ,[a,c])\times \pi ,\pi ').\end{aligned}$$

Proof

Because $St(\rho ,[a,c])\times \pi$ is the generic quotient (remember that $c-a+1< e(\rho )$) of $St(\rho ,[a,b-1])\times St(\rho ,[b,c])\times \pi$, Lemma 4.3 gives us the relation

$$\begin{aligned} L(X,St(\rho ,[a,c])\times \pi ,\pi ')= L(X,St(\rho ,[a,b-1])\times St(\rho ,[b,c])\times \pi ,\pi '). \end{aligned}$$

The result now follows from Proposition 4.7. $\square$

4.3 L-factors of cuspidal representations

We are now going to give the classification of all cuspidal $\ell$-modular L-factors. We thus recall that according to [17, Theorem 6.14] (cf. [20, III 5.14]), any cuspidal representation $\pi$ of $G_n$ is of the form $\pi =St_r(\rho )=St(\rho ,e(\rho )\ell ^r)$ for $\rho$ a supercuspidal representation. In particular a banal cuspidal representation of $G_n$ is always supercuspidal.

Theorem 4.9

Let $\pi _1$ and $\pi _2$ be two cuspidal $\ell$-modular (or $\ell$-adic) representations of $G_{n_1}$ and $G_{n_2}$. Then $L(X,\pi _1,\pi _2)$ is equal to 1, except in the following case: $\pi _1$ is banal (hence supercuspidal), and $\pi _2\simeq \chi \pi _1^\vee$ for some unramified character $\chi$ of $F^\times ~$ (in particular $n_1=n_2$). When  $\pi _1$ is banal, and $\pi _2\simeq \chi \pi _1^\vee$, let $f=f(\pi _1)=f(\pi _2)$, we have

$$\begin{aligned} L(X,\pi _1,\pi _2)=\frac{1}{1-(\chi (\varpi _F)X)^{f}} \end{aligned}$$

and this factor is the reduction modulo $\ell$ of the L-factor of any cuspidal lifts of $\pi _1$ and $\pi _2$.

Proof

The case when $\pi _1$ banal and $\pi _2\simeq \chi \pi _1^\vee$ for some unramified character $\chi$ of $F^\times ~$ is a consequence of [13, Corollary 6.1]. Thus, it remains to consider the case where one of the representations is non-banal. By choosing lifts of $\pi _1$ and $\pi _2$, by Theorem 3.13, we are done if $\pi _2\not \simeq \chi \pi _1^\vee$, for some unramified character $\chi$ of $F^\times$. Assume first that $\pi _2$ is non-banal and supercuspidal. Thus, by Proposition 2.38 there are cuspidal lifts $\rho _2,\rho _2'$ of $\pi _2$ which are not isomorphic by twisting by an unramified character. In particular, if we let $\rho _1$ be a cuspidal lift of $\pi _1$, one of the L-factors $L(X,\rho _1,\rho _2)$ or $L(X,\rho _1,\rho _2')$ is trivial. But as $L(X,\pi _1,\pi _2)$ divides the reduction of both of these factors, the result follows.

Thus we can assume that $\pi _2$ is cuspidal non-supercuspidal and, as $\pi _2\simeq \chi \pi _1^\vee$ for some unramified character $\chi$ of $F^\times$, both of $\pi _1$ and $\pi _2$ are cuspidal non-supercuspidal. We write $\pi _1=St_{r_1}(\rho _1)$ and $\pi _2=St_{r_2}(\rho _2)$. We assume that $n_1=m_1 e(\rho _1)\ell ^{r_1}\ge n_2=m_2 e(\rho _2)\ell ^{r_2}$, where $\rho _1$ is a supercuspidal representation of $G_{m_1}$ and $\rho _2$ is a supercuspidal representation of $G_{m_2}$. Moreover, as neither of ${\text {St}}_{r_1}(\rho _1)$ or ${\text {St}}_{r_2}(\rho _2)$ is supercuspidal, we have $e(\rho _1)\ell ^{r_1}>1$ and $e(\rho _2)\ell ^{r_2}>1$ (and, in fact, we only really need to assume one of the two for the remainder of the proof). In this case, we have

$$\begin{aligned} L(X,St_{r_1}(\rho _1),St_{r_2}(\rho _2))\mid L(X,St_{r_1}(\rho _1),\rho _2\times \dots \times \nu ^{e(\rho _2)\ell ^{r_2}-1}\rho _2) \end{aligned}$$

because $St_{r_2}(\rho _2)$ is the generic subquotient of

$$\begin{aligned} \rho _2\times \dots \times \nu ^{e(\rho _2)\ell ^{r_2}-1}\rho _2, \end{aligned}$$

by Lemmas 4.3 and 4.4, we deduce that

$$\begin{aligned} L(X,St_{r_1}(\rho _1),St_{r_2}(\rho _2))\mid \prod _{i=0}^{e(\rho _2)l^{r_2}-1}L(X,St_{r_1}(\rho _1),\nu ^i\rho _2)). \end{aligned}$$

But each factor $L(X,St_{r_1}(\rho _1),\nu ^i\rho _2))$ is equal to 1 because $n_1>m_2$, hence the result. $\square$

4.4 L-factors of generic segments

By the definition of the ramification index and the lifting of cuspidal $\ell$-modular representations recalled in Sect. 2.7 we have the following proposition, which will allow us to study reduction modulo $\ell$ of banal L-factors of cuspidal representations.

Proposition 4.10

Let $\tau$ be an $\ell$-adic cuspidal representation of $G_n$ and $\rho$ be the reduction modulo $\ell$ of $\tau$, then $q(\tau )=q(\rho )$ and $f(\tau )=f(\rho )$.

We now move on to the case of generic segments. We notice that if $\rho$ is a cuspidal representation of $G_r$, the condition $k<e(\rho )$ implies that $St(\rho ,k)$ is non-banal if and only if $\rho$ is non-banal. Let $D_1=St(\tau _1,k_1)$ and $D_2=St(\tau _2,k_2)$ be generic segments of $G_{n_1}$ and $G_{n_2}$, with $n_1\geqslant n_2$. When $R=\overline{\mathbb {Q}_{\ell }}$, the following formula was proved in [9, Theorem 8.2]:

$$\begin{aligned} L(X,D_1,D_2)=\prod _{j=0}^{k_2-1} L(X,\nu ^{k_1-1}\tau _1,\nu ^j\tau _2). \end{aligned}$$

(1)

As a consequence, we obtain the following useful Lemma.

Lemma 4.11

Let $\rho$ be a cuspidal $\ell$-modular representation, and $St(\rho ,k_1)$ and $St(\rho ^\vee ,k_2)$ be generic segments, with $k_1\geqslant k_2$. Write P the set of poles of $L(X,St(\rho ,k_1),St(\rho ^\vee ,k_2)).$ Then $P^{f(\rho )}=\{x^{f(\rho )},\ x\in P\}$ is a subset of $\{q(\rho )^{k_1-1},\dots ,q(\rho )^{k_1+k_2-2}\}$.

Proof

Let $\tau$ be a cuspidal $\ell$-adic lift of $\rho$, so in particular $f(\rho )=f(\tau )$ and $q(\tau )=q(\rho )$ according to Proposition 4.10. We recall that  $L(X,St(\rho ,k_1),St(\rho ^\vee ,k_2))$ divides $r_\ell (L(X,St(\tau ,k_1),St(\tau ^\vee ,k_2)))$ according to Theorem 3.13, and this last factor is equal to $\prod _{j=0}^{k_2-1}r_\ell (L(q^{-k_1-j-1}X,\tau ,\tau ^\vee ))$, according to Eq. (1). Now, thanks to Theorem 4.9, we have

$$\begin{aligned} r_\ell (L(q^{-k_1-j-1}X,\tau ,\tau ^\vee ))=\frac{1}{1-q(\rho )^{-k_1-j-1}X^f}, \end{aligned}$$

and the result follows. $\square$

We now completely describe the $\ell$-modular L-factors for generic segments.

Theorem 4.12

Let $\mathcal {L}(\Delta _1)=St(\rho _1,k_1)$ and $\mathcal {L}(\Delta _2)=St(\rho _2,k_2)$ be two $\ell$-modular generic segments of $G_{n_1}$ and $G_{n_2}$, respectively, with $n_1\geqslant n_2$, then:

• 1. If $\mathcal {L}(\Delta _1)$ or $\mathcal {L}(\Delta _2)$ is non-banal, then

  $$\begin{aligned}L(X,\mathcal {L}(\Delta _1),\mathcal {L}(\Delta _2))=1. \end{aligned}$$

• 2. If $\mathcal {L}(\Delta _1)$ and $\mathcal {L}(\Delta _2)$ are both banal, then

  $$\begin{aligned} L(X,\mathcal {L}(\Delta _1),\mathcal {L}(\Delta _2))=\prod _{i=0}^{k_2-1} L(X,\nu ^{k_1-1}\rho _1,\nu ^i\rho _2). \end{aligned}$$

  In this case, if $\tau _1$ and $\tau _2$ are cuspidal lifts of $\rho _1$ and $\rho _2$, so that $\mathcal {L}(D_1)=St(\tau _1,k_1)$ and $\mathcal {L}(D_2)=St(\tau _2,k_2)$ are generic segments lifting $\mathcal {L}(\Delta _1)$ and $\mathcal {L}(\Delta _2)$, then

  $$\begin{aligned} L(X,\mathcal {L}(\Delta _1),\mathcal {L}(\Delta _2))=r_\ell (L(X,\mathcal {L}(D_1),\mathcal {L}(D_2)). \end{aligned}$$

Proof

First we notice that the last formula concerning reduction modulo $\ell$ follows from Part 2, Eq. (1) and the last part of Theorem 4.9. We also notice that according to Proposition 2.32, the segment $\mathcal {L}(\Delta _i)$ is banal if and only if $\rho _i$ is banal, for $i=1,2$. In either case, according to Lemmas 4.3 and 4.4, we have

$$\begin{aligned} L(X,\mathcal {L}(\Delta _1),\mathcal {L}(\Delta _2))\mid \prod _{i,j} L(X,\nu ^{i}\rho _1,\nu ^{j}\rho _2). \end{aligned}$$

If either segment is non-banal, each factor $L(X,\nu ^{i}\rho _1,\nu ^{j}\rho _2)$ is equal to 1 according to Theorem 4.9, and the first assertion follows. In fact, the same argument in the non-banal case shows that if $\rho _1^\vee$ and $\rho _2$ are not isomorphic up to twisting by an unramified character, then $L(X,\Delta _1,\Delta _2)=1$.

Hence, we are left with Part 2, and moreover, we can assume $\rho _2=\chi \rho _1^\vee$ for some unramified character $\chi$ of $F^\times$. Thanks to the relation $L(X,\pi ,\chi \pi ')=L(\chi (w_F)X,\pi ,\pi ')$ for any representations $\pi$ and $\pi '$ of Whittaker type, it is enough to prove the result when $\rho _2=\rho _1^\vee$. We thus set $\rho =\rho _1$ and $\rho ^\vee =\rho _2=\rho _1^\vee$. From now on, as we are dealing with banal representations, it is possible to follow the main idea of the proof of [9, Theorem 8.2] up to necessary adjustments (as some of the arguments in this reference depend on poles being in right and left half planes of $\mathbb {C}$). In particular we proceed by induction on $k_2$. We recall that because $\mathcal {L}(\Delta _1)$ and $\mathcal {L}(\Delta _2)$ are generic, we have the inequalities $k_i< e(\rho )$ for $i=1,2$, and because they are banal, we have $e(\rho )=o(\rho )\geqslant 2$.

If $k_2=1$, then $k_1\geqslant 1$. In this case, we proceed by induction on $k_1$. If $k_1=1$, our assertion follows from Theorem 4.9. If $k_1>1$, as $St(\rho ,k_1)$ is the generic subquotient of $\rho \times St(\nu \rho ,k_1-1)$, we have

$$\begin{aligned} L(X,St(\rho ,k_1),\rho ^\vee )\mid L(X,St(\nu \rho ,k_1-1),\rho ^\vee )L(X,\rho ,\rho ^\vee ) \end{aligned}$$

by Lemmas 4.3 and Proposition 4.4. Hence, there is $R\in \overline{\mathbb {F}_{\ell }}[X^{\pm 1}]$, such that

$$\begin{aligned} L(X,St(\rho ,k_1),\rho ^\vee )=R(X)L(X,St(\nu \rho ,k_1-1),\rho ^\vee )L(X,\rho ,\rho ^\vee ). \end{aligned}$$

(2)

We let $e=e(\rho )=o(\rho )$, $f=f(\rho )$, $P_1$ be the set of poles of $L(X,\rho ,\rho ^\vee )$, and $P_2$ be the set of poles of $L(X,St(\rho ,k_1),\rho ^\vee )$. By Lemma 4.11, one has $P_1^f\subseteq \{1\}$, whereas

$$\begin{aligned} P_2^f\subseteq \{q(\rho )^{k_1}\}\subseteq \{q(\rho ),\dots ,q(\rho )^{e-1}\}, \end{aligned}$$

and, in particular, $P_1^{f}\cap P_2^{f}$, hence $P_1\cap P_2$, is empty. Thus if we set $P(X)=R(X)L(X,\rho ,\rho ^\vee )$, then P must belong to $\overline{\mathbb {F}_{\ell }}[X^{\pm 1}]$. By our induction hypothesis, we have

$$\begin{aligned} L(X,St(\nu \rho ,k_1-1),\rho ^\vee )=L(X,\nu ^{k_1-1}\rho ,\rho ^\vee ), \end{aligned}$$

and we record the equation

$$\begin{aligned} L(X,St(\rho ,k_1),\rho ^\vee )=P(X)L(q^{1-k_1}X,\rho ,\rho ^\vee ). \end{aligned}$$

(3)

For the same reason, there is $Q\in \overline{\mathbb {F}_{\ell }}[X^{\pm 1}]$ such that

$$\begin{aligned} L(q^{-1}X^{-1},St(\rho ^\vee ,[1-k_1,0]),\rho )=Q(X)L(q^{-1}X^{-1},\rho ^\vee ,\rho ). \end{aligned}$$

(4)

Now, the relation

$$\begin{aligned} \gamma (X,St(\rho ,k_1),\rho ^\vee )=\prod _{i=0}^{k_1-1} \gamma (q^{-i}X,\rho ,\rho ^\vee ) \end{aligned}$$

implies that, up to units in $\overline{\mathbb {F}_{\ell }}[X^{\pm 1}]$, we have

$$\begin{aligned} \frac{L(q^{-1}X^{-1},St(\rho ^\vee ,[1-k_1,0]),\rho ))}{L(X,St(\rho ,k_1),\rho ^\vee )}= \prod _{i=0}^{k_1-1}\frac{L(q^{i-1}X^{-1},\rho ^\vee ,\rho )}{L(q^{-i}X,\rho ,\rho ^\vee )}. \end{aligned}$$

(5)

But, up to units in $\overline{\mathbb {F}_{\ell }}[X^{\pm 1}]$ again, we have

$$\begin{aligned} L(q^{i-1}X^{-1},\rho ^\vee ,\rho )=L(q^{1-i}X,\rho ^\vee ,\rho ), \end{aligned}$$

hence, up to units, Eq. (5) becomes

$$\begin{aligned} \frac{L(q^{-1}X^{-1},St(\rho ^\vee ,[1-k_1,0]),\rho ))}{L(X,St(\rho ,k_1),\rho ^\vee )}= \frac{L(qX,\rho ^\vee ,\rho )}{L(q^{1-k_1}X,\rho ,\rho ^\vee )}=\frac{L(q^{-1}X^{-1},\rho ^\vee ,\rho )}{L(q^{1-k_1}X,\rho ,\rho ^\vee )}. \end{aligned}$$

(6)

Comparing with Eqs. (3) and (4), we deduce that $P=Q$ up to units, and thus that P divides both polynomials

$$\begin{aligned} L(q^{-1}X^{-1},\rho ^\vee ,\rho )^{-1}=q(\rho )^{-1}X^{-f}-1,\text { and }L(q^{1-k_1}X,\rho ^\vee ,\rho )^{-1}=q(\rho )^{1-k_1}X^{f}-1. \end{aligned}$$

Hence, P divides $q(\rho )X^f-1$ and $q(\rho )X^f-q(\rho )^{k_1}$ in $\overline{\mathbb {F}_{\ell }}[X^{\pm 1}]$. As $q(\rho )^{k_1}\ne 1$, because $k_1<e(\rho )$, the polynomials $q(\rho )X^f-1$ and $q(\rho )X^f-q(\rho )^{k_1}$ are coprime; hence, P is a unit in $\overline{\mathbb {F}_{\ell }}[X^{\pm 1}]$ which must be equal to 1 as it is the quotient of two Euler factors, and the equality

$$\begin{aligned} L(X,St(\rho ,k_1),\rho ^\vee )=L(q^{1-k_1}X,\rho ,\rho ^\vee ) \end{aligned}$$

(7)

follows from Eq. (3).

Now we do the induction step, and suppose that $k_2>1$. By Lemmas 4.3 and 4.4 again, we know that there is $P\in \overline{\mathbb {F}_{\ell }}[X^{\pm 1}]$ such that

$$\begin{aligned} L(X,\mathcal {L}(\Delta _1),\mathcal {L}(\Delta _2))=P (X) \prod _{i=0}^{k_2-1} L(X,\mathcal {L}(\Delta _1),\nu ^i\rho ^\vee ), \end{aligned}$$

which by the previous case, gives the relation

$$\begin{aligned} L(X,\mathcal {L}(\Delta _1),\mathcal {L}(\Delta _2))=P (X) \prod _{i=0}^{k_2-1} L(X,\nu ^{k_1-1}\rho ,\nu ^i\rho ^\vee ) \end{aligned}$$

(8)

By the same argument, we obtain the existence of $Q\in \overline{\mathbb {F}_{\ell }}[X^{\pm 1}]$ such that

$$\begin{aligned} L(q^{-1}X^{-1},\mathcal {L}(\Delta _1)^\vee ,\mathcal {L}(\Delta _2)^\vee )=Q (X) \prod _{i=0}^{k_2-1} L(q^{-1}X^{-1},\rho ^\vee ,\nu ^{-i}\rho ). \end{aligned}$$

(9)

Up to units in $\overline{\mathbb {F}_{\ell }}[X^{\pm 1}]$, we have:

$$\begin{aligned} \gamma (X,\mathcal {L}(\Delta _1),\mathcal {L}(\Delta _2))= & {} \prod _{i=0}^{k_2-1}\gamma (X,\mathcal {L}(\Delta _1),\nu ^i\rho ^\vee )= \prod _{i=0}^{k_2-1}\frac{L(q^{-1}X^{-1},\mathcal {L}(\Delta _1)^\vee ,\nu ^{-i}\rho )}{L(X,\mathcal {L}(\Delta _1),\nu ^i\rho ^\vee )} \nonumber \\= & {} \prod _{i=0}^{k_2-1} \frac{L(q^{-1}X^{-1},\rho ^\vee ,\nu ^{-i}\rho )}{L(X,\nu ^{k_1-1}\rho ,\nu ^i\rho ^\vee )}. \end{aligned}$$

(10)

By Eqs. (8) and (9), we deduce that $P=Q$ up to units in $\overline{\mathbb {F}_{\ell }}[X^{\pm 1}]$, and in fact $P=Q$ as both must belong to $\overline{\mathbb {F}_{\ell }}[X]$ with constant term 1. On the other hand, by Lemma 4.8, we know that $L(X,\mathcal {L}(\Delta _1),St(\rho ,k_2-1))$ divides $L(X,\mathcal {L}(\Delta _1),\mathcal {L}(\Delta _2))$, and

$$\begin{aligned} L(X,\mathcal {L}(\Delta _1),St(\rho ,[1,k_2-1]))=\prod _{i=1}^{k_2-1} L(X,\nu ^{k_1-1}\rho ,\nu ^i\rho ^\vee ) \end{aligned}$$

by our induction hypothesis. This implies, by Eq. (8), that P divides $L(X,\nu ^{k_1-1}\rho ,\rho ^\vee )^{-1}$. Similarly, we find that P divides $L(q^{-1}X^{-1},\rho ^\vee ,\rho )^{-1}$. As we have already proved in the case $k_2=1$ that $L(X,\nu ^{k_1-1}\rho ,\rho ^\vee )^{-1}$ and $L(q^{-1}X^{-1},\rho ^\vee ,\rho )^{-1}$ are coprime, we deduce that $P=1$, and Eq. (8) gives the desired equality. $\square$

4.5 L-factors of banal generic representations

We recall the following fact. If $\rho$ is an $\ell$-adic cuspidal representation of $G_n$, with $f=f(\rho )$, then by [3, 6.2.5], the integer f is the order of the cyclic group of unramified characters of $F^\times$ fixing $\rho$. Notice that there is then only one subgroup of the group of unramified characters of $F^\times$ which has order f; hence, the condition  $\mu ^f=\mathbf {1}$ for an unramified character $\mu$ of $F^\times$ exactly means that $\mu$ fixes $\rho$.

Proposition 4.13

Let R be $\overline{\mathbb {Q}_{\ell }}$ or $\overline{\mathbb {F}_{\ell }}$, and let $\rho$, $\rho '$ and $\rho ''$ be cuspidal R-representations of $G_n$, such that $L(X,\rho ,\rho ')$ and $L(X,\rho ,\rho '')$ have a common pole, then $\rho '\simeq \rho ''$.

Proof

First, this implies that there are unramified characters $\chi '$ and $\chi ''$ of $F^\times$ such that $\rho '\simeq \chi '\rho ^\vee$ and $\rho '' \simeq \chi ''\rho ^\vee$, so $f(\rho )=f(\rho ')=f(\rho '')=f$. Now if x is this common pole, which is necessarily in $R^\times$, we have $1=x^f\chi ''(\varpi )^f=x^f\chi '(\varpi )^f$, i.e. $\chi ''(\varpi )^f=\chi '(\varpi )^f$. Calling $\mu$ the character $\chi ''\chi '^{-1}$, this implies that $\mu ^f=\mathbf {1}$, hence $\mu \rho '\simeq \rho '$ and the result follows. $\square$

Corollary 4.14

Let $\rho$ be a cuspidal R-representation of $G_m$, and $\pi _1$ and $\pi _2$ two R-representations of Whittaker type supported on two different cuspidal lines $\mathbb {Z}_{\rho _1}$ and $\mathbb {Z}_{\rho _2}$. Then $L(X,\rho ,\pi _1)$ and $L(X,\rho ,\pi _2)$ have no common pole.

Proof

It follows at once from Proposition 4.13 and Lemma 4.4. $\square$

Corollary 4.15

Let $\rho$ be a cuspidal R-representation of $G_m$, and $\pi _1$ and $\pi _2$ two R-representations of Whittaker type supported on two different cuspidal lines $\mathbb {Z}_{\rho _1}$ and $\mathbb {Z}_{\rho _2}$. Then $L(X,\rho ,\pi _1)$ and $L(q^{-1}X^{-1},\rho ^\vee ,\pi _2^\vee )$ have no common pole.

Proof

If $L(q^{-1}X^{-1},\rho ^\vee ,\pi _2^\vee )$ has a pole, by Lemma 4.4, it occurs as a pole of $L(q^{-1}X^{-1},\rho ^\vee ,\nu ^{k}\rho _2^\vee )$ for some $k\in \mathbb {Z}$. But, up to a unit of $R[X^{\pm 1}]$, the rational map  $L(q^{-1}X^{-1},\rho ^\vee ,\nu ^{k}\rho _2^\vee )$ is equal to $L(qX,\rho ,\nu ^{-k}\rho _2)=L(X,\rho ,\nu ^{-1-k}\rho _2)$. The result now follows from Corollary 4.14. $\square$

Lemma 4.16

Let $\rho$ be a cuspidal $\ell$-modular representation of $G_m$, and let $\pi =\mathcal {L}(\Delta _1)\times \mathcal {L}(\Delta _2)$ be a banal generic $\ell$-modular representation of $G_n$. If $\Delta _1=[a_1,b_1]_{\rho _1}$ and $\Delta _2=[a_2,b_2]_{\rho _2}$, then if $0 \leqslant k\leqslant \min (b_1-a_1,b_2-a_2)$, the factors $L(X,\rho ,\mathcal {L}(\Delta _1))$ and $L(q^{-1}X^{-1},\nu ^{-k}\rho ^\vee ,\mathcal {L}(\Delta _2)^\vee )$ have no common pole.

Proof

Suppose there is a common pole x. First by Corollary 4.15, $\rho _2$ and $\rho _1$ are on the same cuspidal line, in particular we can take $\rho _1=\rho _2=\tau$. If $\tau$ is not equal to $\rho ^\vee$ up to twisting by an unramified character or $\rho$ is non-banal, then both factors would be equal to 1 and we are done. Hence, $\rho$ is banal, $\tau =\chi \rho ^\vee$ for $\chi$ an unramified character of $F^\times$, and we set $f=f(\rho )$. Now

$$\begin{aligned} L(X,\rho ,\mathcal {L}(\Delta _1))=1/(1-(q^{-b_1}\chi (\varpi )X)^f)\end{aligned}$$

and

$$\begin{aligned}L(q^{-1}X^{-1},\rho ^\vee ,\nu ^{-k}\mathcal {L}(\Delta _2)^\vee )=1/(1-(q^{a_2+k-1}\chi (\varpi )^{-1}X^{-1})^f), \end{aligned}$$

setting $y=\chi (\varpi )x$, we get the relation $y^f=q^{f(a_2+k-1)}=q^{b_1f}$, i.e.

$$\begin{aligned} q(\rho )^{a_2}=q(\rho )^{b_1+1-k}. \end{aligned}$$

(11)

As $\pi$ is banal, we can always choose $a_1\leqslant b_1$ and $a_2\leqslant b_2$ such that, if $c=\max (b_1,b_2)$: either $a_1\leqslant a_2$ and $0\leqslant c-a_1<e-1$, or $a_2\leqslant a_1$ and $0\leqslant c-a_2<e-1$. In the first case, Eq. (11) implies $q(\rho )^{a_2-a_1}=q(\rho )^{b_1-a_1+1-k}$, but as $0\leqslant a_2-a_1< e-1$ and

$$\begin{aligned} 1\leqslant b_1-a_1+1-k\leqslant b_1-a_1+1<e, \end{aligned}$$

we deduce that $a_2-a_1=b_1-a_1+1-k$, i.e. $a_2=b_1+1-k$. In this case one would have $a_1\leqslant a_2\leqslant b_1+1$ and $b_2=b_1+1-k+b_2-a_2\geqslant b_1+1$, which is absurd as $\Delta _1$ and $\Delta _2$ would then be linked. In the second case, Eq. (11) implies $q(\rho )^{1-k+b_1-a_2}=1$. As $1\leqslant 1+[(b_1-a_1)-k]\leqslant 1+b_1-a_2-k \leqslant 1+b_1-a_2< e$, we deduce the equality $b_1-a_2= k-1$, which is absurd as $k\leqslant b_1-a_1\leqslant b_1-a_2$. This ends the proof. $\square$

We now prove the inductivity relation for banal generic representations.

Theorem 4.17

Let $\pi =\mathcal {L}(\Delta _1)\times \dots \times \mathcal {L}(\Delta _r)$ and $\pi '=\mathcal {L}(\Delta '_1)\times \dots \times \mathcal {L}(\Delta '_{r'})$ be two banal generic $\ell$-modular representations of $G_n$ and $G_{n'}$, respectively, then

$$\begin{aligned} L(X,\pi , \pi ')=\prod _{i,j} L(X,\mathcal {L}(\Delta _i),\mathcal {L}(\Delta '_j)). \end{aligned}$$

Proof

We proceed by induction on $n+n'$. If $n+n'=0$, then both sides of the equality are equal to 1. We now do the induction step. Thanks to Lemma 4.4, there is P and Q in R[X], with $P(0)=Q(0)=1$, such that

$$\begin{aligned} L(X,\pi , \pi ')=P\prod _{i,j} L(X,\mathcal {L}(\Delta _i),\mathcal {L}(\Delta '_j)), \end{aligned}$$

(12)

and

$$\begin{aligned} L(q^{-1}X^{-1},\pi ^\vee , \pi '^\vee )=Q\prod _{i,j} L(q^{-1}X^{-1},\mathcal {L}(\Delta _i)^\vee ,\mathcal {L}(\Delta _j)^\vee ). \end{aligned}$$

(13)

The inductivity relation of $\gamma$-factors (Theorem 4.1) gives, up to units of $R[X^{\pm 1}]$, the relation:

$$\begin{aligned} \frac{L(q^{-1}X^{-1},\pi ^\vee , \pi '^\vee )}{L(X,\pi ,\pi ')}= \frac{\prod _{i,j} L(q^{-1}X^{-1},\mathcal {L}(\Delta _i)^\vee ,\mathcal {L}(\Delta '_j)^\vee )}{\prod _{k,l} L(X,\mathcal {L}(\Delta _k),\mathcal {L}(\Delta '_l))}. \end{aligned}$$

(14)

In particular, Eqs. (12), (13) and (14) give $P=Q$ up to units of $R[X^{\pm 1}]$, i.e. $P=Q$ (as they are in R[X] with constant term equal to 1). It thus suffices to prove that they are equal to 1, which will follow from the fact that they have no common root. Notice that if $r=r'=1$ there is nothing to prove and P and Q are obviously equal to 1. If not, we can always assume that $r\geqslant 2$ as the L-function is by definition symmetric with respect to its two last variables. So we suppose that $r\geqslant 2$. In this case, we order the segments $\Delta _1,\dots ,\Delta _r$ and $\Delta '_1,\dots ,\Delta '_{r'}$ so that $\Delta _1$ has minimal length (the length being that of segments, not modules) amongst the $\Delta _i$ and, similarly, $\Delta '_1$ has minimal length amongst the $\Delta '_i$ , which is possible as $\pi$ and $\pi '$ are generic. We have $\mathcal {L}(\Delta _1)=St(\rho ,k)$, where $\rho$ is a cuspidal representation of $G_l$ for some positive integer l. We set $\mathcal {L}(\Delta _1)^{(l)}=St(\rho ,[1,k-1])$ (this is indeed the lth derivative of $\mathcal {L}(\Delta _1)$ by Proposition 2.16). Thanks to our ordering, the representation $\mathcal {L}(\Delta _1)^{(l)}\times \mathcal {L}(\Delta _2) \times \cdots \times \mathcal {L}(\Delta _r)$ is still generic. By induction hypothesis, we have

$$\begin{aligned}&L(X,\mathcal {L}(\Delta _1)^{(l)}\times \mathcal {L}(\Delta _2)\times \cdots \times \mathcal {L}(\Delta _r),\mathcal {L}(\Delta _1')\times \cdots \times \mathcal {L}(\Delta '_{r'})) \nonumber \\&\quad =\prod _{i=1}^{r'} L(X,\mathcal {L}(\Delta _1)^{(l)},\mathcal {L}(\Delta '_i))\prod _{j=2}^r\prod _{k=1}^{r'} L(X,\mathcal {L}(\Delta _j),\mathcal {L}(\Delta '_k)). \end{aligned}$$

(15)

Now by Corollary 4.8, we also know that $L(X,\mathcal {L}(\Delta _1)^{(l)}\times \mathcal {L}(\Delta _2)\times \dots \times \mathcal {L}(\Delta _r),\mathcal {L}(\Delta _1')\times \dots \times \mathcal {L}(\Delta '_{r'}))^{-1}$ divides $L(X,\pi , \pi ')^{-1}$. Hence, Eqs. (12) and (15) together imply that P divides

$$\begin{aligned} \prod _{i=1}^{r'} \frac{L(X,\mathcal {L}(\Delta _1)^{(l)},\mathcal {L}(\Delta '_i))}{L(X,\mathcal {L}(\Delta _1),\mathcal {L}(\Delta '_i)}. \end{aligned}$$

Write $\mathcal {L}(\Delta '_i)=St(\rho '_i,k_i)$, and let $\epsilon _i\in \{0,1\}$, be equal to 1 if and only if $\rho '_i$ is equal to $\rho ^\vee$ up to unramified twist and $k<k_i$. The above product, by Theorem 4.12, is equal to

$$\begin{aligned} \prod _{i=1}^{r'} L(X,\rho ,\mathcal {L}(\Delta '_i))^{-\epsilon _i}. \end{aligned}$$

Similarly, we show that Q divides

$$\begin{aligned} \prod _{i=1}^{r'} L(q^{-1}X^{-1},\nu ^{-k}\rho ,\mathcal {L}(\Delta '_i))^{-\epsilon _i}. \end{aligned}$$

In particular, if P and Q have a common root, there would be i and j in $\{ 1,\ldots ,r'\}$, with $k_i$ and $k_j$ both $>k$, such that $L(X,\rho ,\mathcal {L}(\Delta '_i))$ and $L(q^{-1}X^{-1},\nu ^{-k}\rho ,\mathcal {L}(\Delta '_j))$ would have a common pole x. This is absurd according to Lemma 4.16. This ends the proof. $\square$

We obtain the following corollary.

Theorem 4.18

Let $\pi$ and $\pi '$ be two $\ell$-modular banal generic representations of $G_n$, then $\pi$ and $\pi '$ admit generic lifts $\tau$ and $\tau '$, such that

$$\begin{aligned} L(X,\pi ,\pi ')=r_\ell (L(X,\tau ,\tau ')). \end{aligned}$$

Proof

Take $\tau$ and $\tau '$ two standard lifts which are generic, and apply Theorems 4.17 and 4.12. $\square$

4.6 L-factors of generic representations

We now obtain the inductivity relation for all generic representations. It follows at once from the results of the previous subsection and the following equality.

Theorem 4.19

Let $\pi =\pi _{b}\times \pi _{tnb}$ be a generic $\ell$-modular representation of $G_n$, and $\pi '=\pi _{b}'\times \pi _{tnb}'$ be a generic $\ell$-modular representation of $G_{n'}$, then we have

$$\begin{aligned} L(X,\pi ,\pi ')= L(X,\pi _b,\pi _b'). \end{aligned}$$

Proof

From Proposition 4.7, we know that $L(X,\pi _b,\pi _b')$ divides $L(X,\pi ,\pi ')$. Now from Lemma 4.4 we know that $L(X,\pi ,\pi ')$ divides $L(X,\pi _b,\pi _b')L(X,\pi _{tnb},\pi _b')L(X,\pi _b,\pi _{tnb}')L(X,\pi _{tnb},\pi _{tnb}')$. Hence, we only need to see that each factor $L(X,\pi _{tnb},\pi _b')$, $L(X,\pi _b,\pi _{tnb}')$ and $L(X,\pi _{tnb},\pi _{tnb}')$ is equal to one. This follows easily from Lemma 4.4 and Theorem 4.12. $\square$

An immediate corollary, which follows from the inductivity relation for L factors of $\ell$-modular banal generic representations (Theorem 4.17), is the inductivity relation for L factors of $\ell$-modular generic representations.

Corollary 4.20

Let $\pi =\mathcal {L}(\Delta _1)\times \dots \times \mathcal {L}(\Delta _r)$ and $\pi '=\mathcal {L}(\Delta _1')\times \dots \times \mathcal {L}(\Delta _t')$ be two $\ell$-modular generic representations, then

$$\begin{aligned} L(X,\pi ,\pi ')=\prod _{i,j}L(X,\mathcal {L}(\Delta _i),\mathcal {L}(\Delta _j')). \end{aligned}$$

As a consequence, we get a nice result on congruences of $\ell$-adic L-factors. We start with the particular case of segments.

Proposition 4.21

Let $\mathcal {L}(\Delta )$ and $\mathcal {L}(\Delta ')$ be two $\ell$-modular generic segments of $G_n$ and $G_m$, respectively. Then there are $\ell$-adic generic representations $\tau _1,\dots ,\tau _u$, and $\tau _1',\dots ,\tau _v'$ (with u and v at most 2) such that $J_\ell (\tau _i)=\mathcal {L}(\Delta )$ and $J_\ell (\tau _i')=\mathcal {L}(\Delta ')$, such that

$$\begin{aligned} L(X,\mathcal {L}(\Delta ),\mathcal {L}(\Delta '))=\mathop {\mathbf {GCD}}_{i,j}(L(X,\tau _i,\tau _j')). \end{aligned}$$

Proof

We write $\Delta =[a,b]_\rho$ and $\Delta '=[a',b']_{\rho '}$. In this case we already know the result if both segments are banal by Theorem 4.12 (one only needs one lift). If one segment is non-banal, for instance $\Delta$, then $\rho$ is non-banal as well (Proposition 2.32). We let $\mathcal {L}(D')$ be a segment lifting $\mathcal {L}(\Delta )$ in any case. If $\rho$ is supercuspidal, according to Proposition 2.38, $\rho$ has two cuspidal lifts $\sigma _1$ and $\sigma _2$ which are in different orbits under unramified twists. In particular, as the L factor of a pair of cuspidal representations which are not contragredient of one another up to unramified twist is equal to 1, using Lemmas 4.3 and 4.4 for $L(X,\mathcal {L}([a,b]_{\sigma _1}),\mathcal {L}(D'))$ and $L(X,\mathcal {L}([a,b]_{\sigma _2}),\mathcal {L}(D'))$, this implies that one of the factors is equal to 1, and we are done in this case (with $\tau _i=\mathcal {L}([a,b]_{\sigma _i})$, and $\tau _1'=\mathcal {L}(D')$). If $\rho$ is not supercuspidal, we can write it under the form $St_r(\mu )$ for $r\ge 2$ and some supercuspidal representation $\mu$. Let t be the positive integer such that $\rho$ is a representation of $G_t$, then $\mu$ is a representation of $G_d$ for $d= t/r < t$, the important point here being $d<t$. Then $\rho$ can be lifted to a cuspidal representation $\sigma _1$ of $G_t$, but we also have $\rho =J_\ell (St_r(\nu ))$, for $\nu$ a cuspidal lift of $\mu$. We let $\tau _1=\mathcal {L}([a,b]_{\sigma _1})$, and $\tau _2$ be the generic subquotient of $\nu ^a St_r(\nu )\times \dots \times \nu ^b St_r(\nu )$, then Lemma 2.27 implies that $\Delta =J_\ell (\tau _2)=J_\ell (\tau _1)$. As the L factor of a pair of cuspidal representations which are representations of different linear groups is equal to 1, by Lemmas 4.3 and 4.4 again, one of the factors $L(X,\tau _1,\mathcal {L}(D'))$ and $L(X,\tau _2,\mathcal {L}(D'))$ is equal to 1 and this ends the proof (with $\tau _1'=\mathcal {L}(D')$). $\square$

Theorem 4.22

Let $\pi$ and $\pi '$ be two generic $\ell$-modular representations of $G_n$ and $G_m$, then:

$$\begin{aligned}L(X,\pi ,\pi ')=\mathop {\mathbf {GCD}}(r_\ell (L(X,\tau ,\tau '))).\end{aligned}$$

where the divisor is over all integral generic $\ell$-adic representations $\tau$ of $G_n$ and $\tau '$ of $G_m$ which contain $\pi$ and $\pi '$, respectively, as subquotients after reduction modulo $\ell$.

Proof

We use notations of Lemma 2.27. Let $\tau$ and $\tau '$ be two $\ell$-adic generic representations of $G_n$, and let $\pi =J_\ell (\tau )$ and $\pi '=J_\ell (\tau ')$. We have a surjection $p: \sigma \twoheadrightarrow \tau$ (resp. $p':\sigma '\twoheadrightarrow \tau '~$), where $\sigma =r_1\times \dots \times r_t$ (resp. $r_1'\times \dots \times r_s'$) is a product of cuspidal representations, hence a representation of Whittaker type. By Lemma 4.3, we have $L(X,\tau ,\tau ')=L(X,\sigma ,\sigma ')$. We set $\rho _i=r_\ell (r_i)$ and $\rho _i'=r_\ell (r_i')$, so that Lemma 2.27 tells us that the representation $\pi$ (resp. $\pi '$) is the unique generic subquotient of $\rho _1\times \dots \times \rho _t$ (resp. $\rho _1' \times \dots \times \rho _s'$). By Lemma 4.3 again, the factor $L(X,\pi ,\pi ')$ divides

$$\begin{aligned} L(X,\rho _1\times \dots \times \rho _t,\rho _1' \times \dots \times \rho _s'), \end{aligned}$$

which in turn divides $r_\ell (L(X,\sigma ,\sigma '))=r_\ell (L(X,\tau ,\tau '))$ according to Theorem 3.13. This proves that $L(X,\pi ,\pi ')$ divides the greatest common divisor considered in the statement.

We now have to show that this greatest common divisor divides $L(X,\pi ,\pi ')$. We will thus find a finite number of $\ell$-adic generic representations $\tau _1,\dots ,\tau _u$ and $\tau _1',\dots ,\tau _v'$, with $\pi =J_\ell (\tau _i)$ for all i and $\pi '=J_\ell (\tau _j')$ for all j, such that $\mathop {\mathbf {GCD}}_{i,j}(r_\ell (L(X,\tau _i,\tau _j')))$ divides $L(X,\pi ,\pi ')$. We already know that this is true when $\pi$ and $\pi '$ are generic segments according to Proposition 4.21. In general, we write

$$\begin{aligned} \pi =\mathcal {L}(\Delta _{1})\times \dots \times \mathcal {L}(\Delta _r),\quad \pi '=\mathcal {L}(\Delta _1')\times \dots \times \mathcal {L}(\Delta _s'). \end{aligned}$$

According to Proposition 4.21, for each k between 1 and r (resp. each l between 1 and s) we can choose a finite number of generic representations $\tau _{i_k}^k$ (resp. $\tau _{j_l}^l$) such that $J_\ell (\tau _{i_k}^k)=\mathcal {L}(\Delta _k)$ (resp. $J_\ell ({\tau '}_{j_l}^l)=\mathcal {L}(\Delta _l')$), and

$$\begin{aligned} L(X,\mathcal {L}(\Delta _k),\mathcal {L}(\Delta _l'))=\mathop {\mathbf {GCD}}_{i_k,j_l}(r_\ell (L(X,\tau _{i_k}^k,{\tau '}_{j_l}^l))). \end{aligned}$$

We denote by $\tau _{i_1,\dots ,i_r}$ the unique generic subquotient of $\tau _{i_1}^1\times \dots \times \tau _{i_r}^r$, and ${\tau '}_{j_1,\dots ,j_s}$ the unique generic subquotient of ${\tau '}_{j_1}^1\times \dots \times {\tau '}_{j_s}^s$, so that $J_\ell (\tau _{i_1,\dots ,i_r})=\pi$, and $J_\ell (\tau _{i_1,\dots ,i_s}')=\pi '$. By Lemmas 4.3 and 4.4, we know that $L(X,\tau _{i_1,\dots ,i_r},\tau _{j_1,\dots ,j_s}')$ divides the factor $\prod _{k=1,l=1}^{r,s} L(X,\tau _{i_k}^k,{\tau '}_{i_l}^l).$ Hence

$$\begin{aligned} \underset{(i_1,\dots ,i_r),(j_1,\dots ,j_s)}{\mathop {\mathbf {GCD}}}\left( L(X,\tau _{i_1,\dots ,i_r},\tau _{j_1,\dots ,j_s}')\right) \mid \underset{(i_1,\dots ,i_r),(j_1,\dots ,j_s)}{\mathop {\mathbf {GCD}}}\left( \prod _{k=1,l=1}^{r,s} r_\ell (L(X,\tau _{i_k}^k,{\tau '}_{i_l}^l))\right) . \end{aligned}$$

This latter GCD is equal to

$$\begin{aligned} \prod _{k=1,l=1}^{r,s} \mathop {\mathbf {GCD}}_{i_k,j_l}\left( r_\ell (L(X,\tau _{i_k}^k,{\tau '}_{i_l}^l))\right) = \prod _{k=1,l=1}^{r,s} L(X,\mathcal {L}(\Delta _k),\mathcal {L}(\Delta _l'))=L(X,\pi ,\pi '), \end{aligned}$$

the last equality according to Corollary 4.20. This ends the proof. $\square$

Remark 4.23

Taking the unique generic subquotient in the reduction modulo $\ell$ of an integral generic $\ell$-adic representation is an instance of Vignéras’ $J_{\ell }$ map (cf. [23, 1.8.4]). As this map is defined on all integral irreducible $\ell$-adic representations of $G_n$, and is surjective onto all irreducible $\ell$-modular representations of $G_n$ [ibid.], it is tempting to define the L-factor $L(X,\pi ,\pi ')$, of pairs of irreducible (not necessarily generic) $\ell$-modular representations $\pi ,\pi '$, by a $\mathop {\mathbf {GCD}}$ of the reductions modulo $\ell$ of the $\ell$-adic L-factors $L(X,\tau ,\tau ')$ where $\tau ,\tau '$ vary over all integral $\ell$-adic representations such that $J_{\ell }(\tau )=\pi$ and $J_{\ell }(\tau ')=\pi '$. Interesting future questions revolve around determining whether this is a natural definition, perhaps, by finding an integral representation for this $\ell$-modular L-factor.