薛定谔方程是量子力学中最重要的方程,最近十几年来,对于与物理学密切相关的非线性薛定谔方程解的存在性和解的各种性质的研究,一直是全世界数学和物理工作者关注的焦点和研究热点.

考虑下面次临界薛定谔方程组

(1.1) $ \begin{eqnarray}\label{11} \left\{\begin{array}{ll} -\Delta u_1+\alpha_1 u_1 =\mu_1\, |u_1|^{p-1}u_1+\beta\, |u_1|^{\frac{p-3}{2}}u_1|u_2|^{\frac{p+1}{2}}, & x \in \Omega, \\ -\Delta u_2+\alpha_2 u_2 = \mu_2\, |u_2|^{p-1}u_2+\beta\, |u_1|^{\frac{p+1}{2}}\, |u_2|^{\frac{p-3}{2}}u_2 , &x \in \Omega, \\ u_1=u_2= 0, & x\in \partial \Omega, \end{array}\right. \end{eqnarray}$

其中$\Omega\subset\mathbb{R} ^N$是有界光滑区域, $N\geq 3$, $\alpha_i, \, \mu_i>0$, $i=1, \, 2$, $\beta < 0$, $1 < p < 2^*-1$, $2^*=\frac{2N}{N-2}$是临界$\textrm{Sobolev}$指数.

近年来,此类方程组由于在非线性力学和$\textrm{Bose-Einstein}$凝聚等物理中有着非常重要的作用,引起了广大数学和物理工作者的极大关注.大量杰出的研究成果不断涌现.下面简单介绍一下已有的重要结果,然后给出该文的研究问题.

大家关心的是方程组(1.1)的非平凡解的存在性及其相关性质,参见文献[1-6].下面简单回顾几个重要的研究成果.当$\Omega=\mathbb{R} ^N$, $N=2, \, 3$, $\alpha_i>0$, $i=1, \, 2$,数学上关于方程组(1.1)最早结果由$\textrm{Lin}$和Wei^[5]在$2005$年给出了基态解存在性.除了基态解,在过去十多年中,许多知名数学家对于多解性也做了深入细致的研究.文献[2-4, 6-8]中得到了方程(1.1)无穷多个正解、径向对称解等存在性.另外,解的各种性质也是研究的焦点.文献[3, 6, 9]研究了方程组(1.1)的正解的唯一性,正解的先验估计,在$\beta\rightarrow -\infty$时解的极限的光滑性,正解的渐近收敛行为及其相应正解产生的相位分离现象等.

前面已经指出,关于方程组(1.1)还有很多大家非常关心的重要问题没有解决.该文主要研究一些与方程组(1.1)密切相关的重要问题.从上面问题介绍中可知,方程组(1.1)正解研究方面已经取得了丰富成果.当$\Omega\subset\mathbb{R} ^N$是有界光滑区域, $N\geq 3$, $\alpha_i, \, \mu_i>0$, $i=1, \, 2$, $\beta < 0$, $1 < p < 2^*-1$时,参考文献[11]证明了无穷个变号解和半变号解的存在性,但是此时解的极限行为及其相位分离现象,却没有相应的研究结果.

该文中,定义$L^p(\Omega)$的范数为$|u|_p=\big(\int_{\Omega} |u|^p\, {\rm d}x\big)^{\frac{1}{p}}$, $H_0^1(\Omega)$的范数为

$\|u\|^2=\int_{\Omega}(|\nabla u|^2+ |u|^2)\, {\rm d}x, $

$C$表示常数(不同地方取值可以不一样).记$H:= H_0^1(\Omega)\times H_0^1(\Omega)$,其上的范数为

$\|(u_1, \, u_2)\|_{H}^2:=\|u_1\|_{\alpha_1}^2+\|u_2\|_{\alpha_2}^2, $

其中

$\|u_i\|_{\alpha_i}^2:=\int_{\Omega}(|\nabla u_i|^2+\alpha_i |u_i|^2)\, {\rm d}x, \quad i=1, \, 2.$

方程组(1.1)的解对应于$C^2$泛函$I_\beta:\, H\rightarrow \mathbb{R} $,

(2.1) $ \begin{eqnarray} \label{21} I_\beta(u_1, \, u_2) &:=& \frac{1}{2} (\| u_1\|_{\alpha_1}^2+\| u_2\|_{\alpha_2}^2)-\frac{1}{p+1}(\mu_1\, |u_1|_{p+1}^{p+1}+\mu_2\, |u_2|_{p+1}^{p+1})\\ &&+\frac{2|\beta|}{p+1} \int_{\Omega} |u_1|^{\frac{p+1}{2}}|u_2|^{\frac{p+1}{2}}\, {\rm d}x \end{eqnarray} $

的临界点.由于该文只关心非平凡解,记$\widetilde{H}:=\{(u_1, \, u_2)\in H:\, u_i\neq 0, \, i=1, \, 2 \}$.

定义2.1  若$u_1\not\equiv 0, \, u_2\not\equiv 0$,则称解$(u_1, \, u_2)$是非平凡解;若$(u_1, \, u_2)$形如$(u_1, \, 0)$或$(0, \, u_2)$,则称解$(u_1, \, u_2)$是半平凡解;若在$\Omega$上$u_i>0$对于$i=1, \, 2$都成立,则称解$(u_1, \, u_2)$是正解;若$u_1$和$u_2$都是变号的,则称解$(u_1, \, u_2)$是变号解;若$(u_1, \, u_2)$的一个分量是正的而另一个分量变号,则称解$(u_1, \, u_2)$是半变号解;若变号解$(u_1, \, u_2)$的能量泛函在所有变号解的泛函能量中是最小的,则称变号解$(u_1, \, u_2)$是最小能量变号解.

由文献[11]可知,对任意$\beta < 0$,存在$(u_{1, \beta}, \, u_{2, \beta})\in H$使得$(u_{1, \beta}, \, u_{2, \beta})$或是方程组(1.1)变号解或是方程组(1.1)半变号解.这一节我们研究当$\beta\rightarrow -\infty$时,方程组(1.1)解$(u_{1, \beta}, \, u_{2, \beta})$的渐近收敛行为.这节的主要结果如下：

定理2.1  存在$\textrm{Lipschitz}$连续的$(u_1, \, u_2)\in \widetilde{H}$,使得在子列的意义下满足:

(1)对任意的$0 < r < 1$,当$\beta\rightarrow -\infty$时,有$u_{1, \beta}\rightarrow u_1$, $u_{2, \beta}\rightarrow u_2$在$H_0^1(\Omega)\cap C^{0, r}(\overline{\Omega})$中强收敛;

(2)在开集$\{u_i\neq 0\}$, $i=1, \, 2$上,

(2.2) $\begin{eqnarray}\label{22}-\Delta u_i+\alpha_i \, u_i=\mu_i\, u_i^p, \qquad i=1, \, 2;\end{eqnarray}$

(3) $u_1\cdot u_2\equiv 0$,当$\beta\rightarrow -\infty$时,有

$|\beta|\, \int_{\Omega} u_{1, \beta}^{\frac{p+1}{2}}\, u_{2, \beta}^{\frac{p+1}{2}}\, {\rm d}x\rightarrow 0 .$

为了证明定理$2.1$,该文需要用文献[10]中介绍的向量指标定义恰当的极大极小值.考虑集合

$ {\cal F}=\{A\subset {\cal A}:\, A \hbox{是闭集}, \, \sigma_i(u_1, \, u_2)\in A, \, \forall (u_1, \, u_2)\in A, \, i=1, \, 2\} , $

其中${\cal A}:=\{(u_1, \, u_2)\in H:\, |u_1|_{p+1}=|u_2|_{p+1}=1 \}$,变换$\sigma_i:\, {\cal A}\rightarrow {\cal A}$, $i=1, \, 2$为$\sigma_1(u_1, \, u_2)=(-u_1, \, u_2)$, $\sigma_2(u_1, \, u_2)=(u_1, \, -u_2)$.对任意的$A\in {\cal F}$和$k_1, \, k_2\in {\mathbb N}$,考虑集合

$ \begin{eqnarray*} F_{(k_1, \, k_2)}(A)&=&\Big\{f=(f_1, \, f_2):\, A\rightarrow \prod\limits_{i=1}^2 \mathbb{R} ^{k_i-1}:\, f_i:\, A\rightarrow \mathbb{R} ^{k_i-1}\hbox{连续}, \\&&\ \ f_i(\sigma_i(u_1, \, u_2))=-f_i(u_1, \, u_2), \, f_i(\sigma_j(u_1, \, u_2))=f_i(u_1, \, u_2), \, i\neq j, \, i, j=1, 2 \Big\}, \end{eqnarray*}$

这里记$\mathbb{R} ^0:=\{0\}$.下面给出文献[10]中定义的向量指标.

定义2.2 (向量指标^[10])  令$A\in {\cal F}$并任取$k_1, \, k_2\in {\mathbb N}$.若对任意的$f\in F_{(k_1, \, k_2)}(A)$都存在$(u_1, \, u_2)\in A$满足$f(u_1, \, u_2)=(0, \, 0)$.则称$A$的向量指标$\gamma(A)\geq (k_1, \, k_2)$.记

$\Gamma^{(k_1, \, k_2)}:=\{A\in {\cal F}:\, \gamma(A)\geq (k_1, \, k_2)\}.$

引理2.1^[10]  (1)设$A_1\times A_2\subset {\cal A}$, $\eta_i:\, S^{k_i-1}\rightarrow A_i$是同胚,对任意$x\in S^{k_i-1}$, $\eta_i(-x)=-\eta_i(x)$, $i=1, \, 2$,则$A_1\times A_2 \in \Gamma^{(k_1, \, k_2)}$,其中$S^{k_i-1}=\{x\in \mathbb{R} ^{k_i}:\, |x|=1\}$.

(2)若$A \in\Gamma^{(k_1, \, k_2)}$,连续映射$\eta:\, A\rightarrow {\cal A}$满足$\eta \circ \sigma_i=\sigma_i\circ \eta, \, i=1, \, 2.$则有$\overline{\eta(A)}\in \Gamma^{(k_1, \, k_2)}$.

引理2.2  存在$A\in \Gamma^{(k_1, \, k_2)}$和正常数$ d^{k_1, k_2}$, $k_1, \, k_2\geq 2$,使得

$ \sup\limits_{(u_1, u_1)\in A} \sup\limits_{t, s\geq 0} I_\beta(tu_1, \, su_2)\leq d^{k_1, k_2}, $

其中$d^{k_1, k_2}$不依赖于$\beta < 0$的选取.

证  任取非空开集$\Omega_1, \, \Omega_2\subset \Omega$满足$\Omega_1\cap\Omega_2=\emptyset$.定义

$A_i:= \Big\{u_i\in \hbox{span} \{U^i_1, \, \cdots, U^i_{k_i}\}:\, U^i_j\in H_0^1(\Omega_i), \, U^i_j \, \hbox{线性无关, }\, 1\leq j\leq k_i, \, |u_i|_{p+1}=1 \Big\}, $

其中$i=1, \, 2$.显然存在从$S^{k_i-1}$到$A_i$的奇同胚, $i=1, \, 2$.则由引理$2.1(1)$可知, $A :=A_1\times A_2\in \Gamma^{(k_1, \, k_2)}$.对于任意的$(u_1, \, u_2)\in A$,由于$u_i\in H_0^1(\Omega_i)$,所以$u_1\cdot u_2\equiv 0$,则由于有限维线性空间的范数都是等价的,所以存在常数$d_{k_i}>0$使得$\| u_i\|_{\alpha_i}^2\leq d_{k_i}\, |u_i|_{p+1}^2$.由文献[11]可知存在常数$T_1\leq T_2$满足$0 < T_1\leq t_{u_1, u_2, \beta}, \, s_{u_1, u_2, \beta}\leq T_2 < +\infty$满足

$\sup\limits_{t, \, s\geq 0}\, I_\beta (tu_1, \, su_2)=I_\beta (t_{u_1, u_2, \beta}u_1, \, s_{u_1, u_2, \beta}u_2). $

记$\widehat{t}:=t_{u_1, u_2, \beta}, \, \widehat{s}:=s_{u_1, u_2, \beta}$.因此,

$\begin{eqnarray*} \sup\limits_{(u_1, u_1)\in A} \sup\limits_{t, s\geq 0} I_\beta(tu_1, \, su_2)&= &\Big(\frac{1}{2}-\frac{1}{p+1}\Big)\sup\limits_{(u_1, u_1)\in A}\, \Big(\widehat{t}^2\| u_1\|_{\alpha_1}^2+\widehat{s}^2\|u_2\|_{\alpha_2}^2\Big)\\ &\leq&\Big(\frac{1}{2}-\frac{1}{p+1}\Big) \sup\limits_{(u_1, u_1)\in A}\, \Big(\widehat{t}^2\, d_{k_1}\, |u_1|_{p+1}^2+\widehat{s}^2\, d_{k_2}\, |u_2|_{p+1}^2\Big) \\&\leq& d^{k_1, k_2} . \end{eqnarray*}$

故存在与$\beta < 0$无关的常数$d^{k_1, k_2}>0$使得$\sup\limits_{(u_1, u_1)\in A} \sup\limits_{t, s\geq 0} I_\beta(tu_1, \, su_2)\leq d^{k_1, k_2}$对任意$\beta < 0$都成立.

对任意$k_1, \, k_2\geq 1$和$0 < \delta < 2^{-\frac{1}{p+1}}$,定义

$d_{\beta, \delta}^{k_1, k_2}:=\inf\limits_{A\in \Gamma_\beta^{(k_1, k_2)}}\sup\limits_{A\backslash {\cal P}_\delta}\sup _{t, s\geq 0}\, I_\beta(tu_1, \, su_2), $

其中

$\Gamma_\beta^{(k_1, \, k_2)}:=\{A\in \Gamma^{(k_1, \, k_2)} :\, \sup\limits_{(u_1, u_1)\in A} \sup\limits_{t, s\geq 0} I_\beta(tu_1, \, su_2) < d^{k_1, k_2}+1\}, $

正锥${\cal P}_i:=\{(u_1, \, u_2)\in H:\, u_i\geq 0\}, \, i=1, \, 2$, ${\cal P}:=\bigcup\limits_{i=1}^2\big({\cal P}_i\cup -{\cal P}_i \big)$,

${\cal P}_\delta:=\{(u_1, \, u_2)\in H:\, \mbox{dist}_{p+1}((u_1, \, u_2), \, {\cal P}) <\delta\}, $

$ \mbox{dist}_{p+1} ((u_1, \, u_2), \, {\cal P}):=\min \{\mbox{dist}_{p+1} (u_i, \, \pm{\cal P}_i) , \, \mbox{dist}_{p+1} (u_i, \, \pm{\cal P}_i)\}, \quad i=1, \, 2, $

由于$\mbox{dist}_{p+1} (u_i, \, \pm{\cal P}_i) :=\inf\limits_{\omega\in \pm{\cal P}_i} |u_i-\omega |_{p+1}$,且$u_i=u_i^+-u_i^-$,可知$\mbox{dist}_{p+1} (u_i, \, \pm{\cal P}_i)=|u_i^\mp|_{p+1}$,其中$u^\pm:=\max \{0, \, \pm u\}$.引理$2.2$表明, $\Gamma_\beta^{(k_1, \, k_2)}\neq \emptyset$,所以$\Gamma_\beta^{(k_1, \, k_2)}$定义合理,并且$d_{\beta, \delta}^{k_1, k_2}\leq d^{k_1, k_2}$.

任意固定的$k_1, \, k_2\in {\mathbb N}$, $k_1\geq 2$, $k_2\geq 1$.由参考文献[11]可知,对任意$\beta < 0$,存在$0 < \delta_\beta < 2^{-\frac{1}{p+1}}$和$(u_{1, \beta}, \, u_{2, \beta})\in H$使得$(u_{1, \beta}, \, u_{2, \beta})$或是方程组(1.1)变号解或是方程组(1.1)半变号解且满足

(2.3) $\begin{eqnarray}\label{23} I_\beta(u_{1, \beta}, \, u_{2, \beta})= d_{\beta, \delta_\beta}^{k_1, k_2}\leq d^{k_1, k_2} <+\infty. \end{eqnarray} $

引理2.3  假设$k_1, \, k_2\geq 2$.则对于任意的$0 < \delta < 2^{-\frac{1}{p+1}}$和任意的$A \in\Gamma^{(k_1, \, k_2)}$,有$A\backslash{\cal P}_\delta \neq \emptyset$.

证  任取$A \in\Gamma^{(k_1, \, k_2)}$,定义$f=(f_1, \, f_2)$,

$ f_1(u_1, \, u_2)=\bigg(\int_{\Omega} |u_1|^pu_1\, {\rm d}x, \, 0, \, \cdots, \, 0\bigg), \quad f_2(u_1, \, u_2)=\bigg(\int_{\Omega} |u_2|^p u_2\, {\rm d}x, \, 0, \, \cdots, \, 0\bigg), $

则有$f_i:\, A\rightarrow \mathbb{R} ^{k_i-1}$连续,

$f_1(\sigma_1(u_1, \, u_2))=f_1(-u_1, \, u_2)=-\bigg(\int_{\Omega} |u_1|^pu_1\, {\rm d}x, \, 0, \, \cdots, \, 0\bigg)=-f_1(u_1, \, u_2), $

$f_1(\sigma_2(u_1, \, u_2))=f_1(u_1, \, -u_2)=\bigg(\int_{\Omega} |u_1|^pu_1\, {\rm d}x, \, 0, \, \cdots, \, 0\bigg)=f_1(u_1, \, u_2), $

同理,

$ f_2(\sigma_1(u_1, \, u_2))=f_2(u_1, \, u_2), f_2(\sigma_2(u_1, \, u_2))=-f_2(u_1, \, u_2). $

则由$F_{(k_1, \, k_2)}(A)$定义可知$f\in F_{(k_1, \, k_2)}(A)$,所以存在$(\widetilde{u}_1, \, \widetilde{u}_2)\in A$使得$f(\widetilde{u}_1, \, \widetilde{u}_2)=(0, \, 0)$.由于$A\in {\cal A}$,则可推出

$\int_{\Omega} (\widetilde{u}_1^+)^{p+1}\, {\rm d}x=\int_{\Omega} (\widetilde{u}_1^-)^{p+1}\, {\rm d}x=\frac{1}{2}, $

$\int_{\Omega} (\widetilde{u}_2^+)^{p+1}\, {\rm d}x=\int_{\Omega} (\widetilde{u}_2^-)^{p+1}\, {\rm d}x=\frac{1}{2}, $

即$\mbox{dist} ((\widetilde{u}_1, \, \widetilde{u}_2), \, {\cal P})=2^{-\frac{1}{p+1}}$.所以$(\widetilde{u}_1, \, \widetilde{u}_2)\in A \backslash{\cal P}_\delta$对任意的$0 < \delta < 2^{-\frac{1}{p+1}}$都成立.

定理2.1的证明  由于

$I_\beta(u_{1, \beta}, \, u_{2, \beta})= \Big(\frac{1}{2}-\frac{1}{p+1}\Big)\, \Big(\| u_{1, \beta}\|_{\alpha_1}^2+\|u_{2, \beta}\|_{\alpha_2}^2\Big), $

则由(2.3)式可知$(u_{1, \beta}, \, u_{2, \beta})$在$H$中一致有界.因为$H_0^1(\Omega)\hookrightarrow L^2(\Omega)$,则存在常数$C>0$使得

$| u_{i, \beta}|_2\leq C, \qquad i=1, \, 2.$

假设存在$\delta>0$,满足$u_{i, \beta}\in L^{2+\delta}(\Omega)$,方程(1.1)两端乘以$u_{i, \beta}\, |u_{i, \beta}|^\delta$,然后积分,可知存在常数$C>0$满足

$\begin{eqnarray*}&&\frac{1+\delta}{(1+\frac{\delta}{2})^2}\, \int_\Omega |\nabla |u_{i, \beta}|^{1+\frac{\delta}{2}} |^2\, {\rm d}x\\&\leq& \frac{1+\delta}{(1+\frac{\delta}{2})^2}\, \int_\Omega |\nabla |u_{i, \beta}|^{1+\frac{\delta}{2}} |^2\, {\rm d}x+|\beta|\, \int_\Omega|u_{i, \beta}|^{\frac{p-3}{2}}|u_{i, \beta}|^{2+\delta}|u_{j, \beta}|^{\frac{p+1}{2}}\, {\rm d}x\\&\leq &C\, \int_\Omega|u_{i, \beta}|^{2+\delta}\, {\rm d}x.\end{eqnarray*}$

因此

$\begin{eqnarray*}|u_{i, \beta}|_{2^*(1+\frac{\delta}{2})}&\leq&\Big(S^2\, \frac{(1+\frac{\delta}{2})^2}{1+\delta} \Big)^{\frac{1}{2+\delta}}\Big(C\, \int_\Omega|u_{i, \beta}|^{2+\delta}\, {\rm d}x\Big)^{\frac{1}{2+\delta}}\\&\leq& \Big(C\, \frac{(1+\frac{\delta}{2})^2}{1+\delta} \Big)^{\frac{1}{2+\delta}}|u_{i, \beta}|_{2+\delta}, \end{eqnarray*}$

其中$S$是$H_0^1(\Omega)\hookrightarrow L^{2^*}(\Omega)$的嵌入常数, $i\neq j, \, i, \, j=1, \, 2$.

令$\delta(1)=0$, $2+\delta(m+1)=2^*\frac{2+\delta(m)}{2}$.由于$\delta(m)\geq (\frac{2^*}{2})^{m-1}$,则有$\delta(m)\rightarrow +\infty$.则通过标准的$\textrm{Moser}$迭代可知

$\begin{eqnarray*}|u_{i, \beta}|_{2^*(1+\frac{\delta(m)}{2})}&\leq& \prod\limits_{m=1}^k\, \Big(C\, \frac{(1+\frac{\delta(m)}{2})^2}{1+\delta(m)} \Big)^{\frac{1}{2+\delta(m)}}|u_{i, \beta}|_{2}\\&\leq& {\rm e}^{\Big(\sum\limits_{j=1}^\infty \frac{1}{2+\delta(m)} \, \textrm{log}\, \Big(C\, \frac{(1+\frac{\delta(m)}{2})^2}{1+\delta(m)} \Big)\Big)}|u_{i, \beta}|_{2}, \end{eqnarray*}$

由于$\delta(m)\geq (\frac{2^*}{2})^{m-1}$,则有

$\sum\limits_{j=1}^\infty \frac{1}{2+\delta(m)} \log \Big(C\, \frac{(1+\frac{\delta(m)}{2})^2}{1+\delta(m)} \Big) <+\infty.$

由此可知$u_{i, \beta}$在$L^\infty(\Omega)$中对任意$\beta < 0$和$i=1, \, 2$都是一致有界的.则由椭圆正则性理论知道$u_{i, \beta}\in C(\overline{\Omega})\cap C^2(\Omega)$,所以定理$2.1$可由文献[12]的定理$1.1$和定理$1.2$推出.虽然文献[12]的结果是关于正解的,通过分别处理非平凡解的正部和负部,文献[12]所用讨论都能用于没有符号限制的解成立.

在这一节,我们研究当$\beta\rightarrow -\infty$时非平凡解的相位分离现象.这节的主要结果如下:

定理3.1  对于定理$2.1$中存在的$(u_1, \, u_2)\in \widetilde{H}$,使得在子列的意义下满足:

(1)若$k_1, \, k_2\geq 2$,则有$u_1$和$u_2$都是变号;

(2)若$(k_1, \, k_2)=(2, \, 2)$,则有$\{u_i\neq 0\}$恰有两个连通分支,且$u_i$是方程

(3.1) $\begin{eqnarray}\label{31} -\Delta u+\alpha_i \, u=\mu_i\, u^p, \qquad u\in H_0^1(\{u_i\neq 0\}) \end{eqnarray} $

的最小能量变号解, $i=1, \, 2$;

(3)若$k_1\geq2$且$k_2=1$,则有$u_1$变号, $\{u_1\neq 0\}$至多有$k_1$个连通分支, $u_2$在$\{u_2\neq 0\}$上是正的;

(4)若$(k_1, \, k_2)=(2, \, 1)$,则有$\{u_1\neq 0\}$恰有两个连通分支, $\{u_2\neq 0\}$连通且$u_i$是方程(3.1)的最小能量变号解, $i=1, \, 2$.

证   (1)首先考虑$k_1, \, k_2\geq 2$的情形.由于$u_{i, \beta}\in C(\overline{\Omega})\cap C^2(\Omega)$且$u_{i, \beta}$是变号解,所以存在$x^\pm_{i, \beta}\in \Omega$满足

$u_{i, \beta}(x^+_{i, \beta})=\max\limits_{x\in \Omega}u_{i, \beta}(x)>0\quad \mbox{且}\quad u_{i, \beta}(x^-_{i, \beta})=\min\limits_{x\in \Omega}u_{i, \beta}(x) <0, \quad i=1, \, 2.$

则有$\Delta u_{i, \beta}(x^+_{i, \beta})\leq 0$和$\Delta u_{i, \beta}(x^-_{i, \beta})\geq 0$.由于$(u_{1, \beta}, \, u_{2, \beta})$满足方程(1.1),对于$i\neq j, $$i, j=1, \, 2$,可得

$\begin{eqnarray*}&& -\Delta u_{i, \beta}(x^+_{i, \beta})+\alpha_i\, u_{i, \beta}(x^+_{i, \beta})\\ &=&\mu_i\, |u_{i, \beta}(x^+_{i, \beta})|^{p-1}\, u_{i, \beta}(x^+_{i, \beta})+ \beta \, |u_{i, \beta}(x^+_{i, \beta})|^{\frac{p-3}{2}}u_{i, \beta}(x^+_{i, \beta}) |u_{j, \beta}(x^+_{i, \beta})|^{\frac{p+1}{2}}, \end{eqnarray*}$

所以$\alpha_i\, u_{i, \beta}(x^+_{i, \beta})\leq \mu_i\, u_{i, \beta}(x^+_{i, \beta})^{p}$,即

$u_{i, \beta}(x^+_{i, \beta})\geq \Big(\frac{\alpha_i}{\mu_i}\Big)^{\frac{1}{p-1}}, \qquad \forall\quad \beta <0.$

同理

$\begin{eqnarray*}&& -\Delta u_{i, \beta}(x^-_{i, \beta})+\alpha_i\, u_{i, \beta}(x^-_{i, \beta})\\ &=&\mu_i\, |u_{i, \beta}(x^-_{i, \beta})|^{p-1}\, u_{i, \beta}(x^-_{i, \beta})+ \beta \, |u_{i, \beta}(x^-_{i, \beta})|^{\frac{p-3}{2}}u_{i, \beta}(x^-_{i, \beta}) |u_{j, \beta}(x^-_{i, \beta})|^{\frac{p+1}{2}}, \end{eqnarray*} $

所以$\alpha_i\, u_{i, \beta}(x^-_{i, \beta})\geq - \mu_i\, |u_{i, \beta}(x^-_{i, \beta})|^{p-1}u_{i, \beta}(x^-_{i, \beta})$,即

$ u_{i, \beta}(x^-_{i, \beta})\leq -\Big(\frac{\alpha_i}{\mu_i}\Big)^{\frac{1}{p-1}}, \qquad \forall\quad \beta <0.$

则由定理$2.1(1)$可知$u_1$和$u_2$变号,从而$\{u_i\neq 0\}$至少有两个连通分支, $i=1, \, 2$.

(2)若$(k_1, \, k_2)=(2, \, 2)$.假设$\{u_1\neq 0\}$至少有三个连通分支${\cal O}_1$, ${\cal O}_2$和${\cal O}_3$.假设在${\cal O}_1\cup{\cal O}_2$上有$u_1>0$,在${\cal O}_3$上有$u_1 < 0$.定义

$\omega^+:=u_1\, \chi_{{\cal O}_1}, \qquad \omega^-:=u_1\, \chi_{{\cal O}_2}, \qquad v:=u_1\, \chi_{{\cal O}_3}, $

其中

$\chi_{\Omega}(x) := \left\{\begin{array}{ll} 1, \quad& x\in \Omega, \\ 0, &x\in \mathbb{R} ^N\backslash \Omega, \end{array}\right.$

则有$\omega^\pm, \, v\in H^1_0(\Omega)\backslash \{0\}$,且有

(3.2) $\begin{eqnarray}\label{32} \|u_1 \|_{\alpha_1}^2>\|\omega^+ \|_{\alpha_1}^2+\|\omega^-\|_{\alpha_1}^2. \end{eqnarray}$

由于$I_\beta'(u_1, \, u_2)(\omega^\pm, \, 0)=0$和$I_\beta'(u_1, \, u_2)(0, \, u_{2}^\pm)=0$,可得

(3.3) $\begin{equation}\label{33} \|\omega^\pm\|_{\alpha_1}^2=\mu_1\, |\omega^\pm|_{p+1}^{p+1}-|\beta|\, \int_{\Omega} |\omega^\pm|^{\frac{p+1}{2}}|u_2|^{\frac{p+1}{2}}\, {\rm d}x, \end{equation}$

(3.4) $\begin{equation}\label{34} \|u_2^\pm\|_{\alpha_2}^2=\mu_2\, |u_2^\pm|_{p+1}^{p+1}-|\beta|\, \int_{\Omega} |u_1|^{\frac{p+1}{2}}|u_2^\pm|^{\frac{p+1}{2}}\, {\rm d}x. \end{equation}$

定义集合

${\cal N}_\beta:=\{(u_1, u_2)\in H:\, u_1, \, u_2\mbox{都变号}, \, I_\beta'(u_1, \, u_2)(u_1^\pm, \, 0)= I_\beta'(u_1, \, u_2)(0, \, u_2^\pm)=0\}, $

定义$d_\beta:=\inf\limits_{(u_1, u_2)\in {\cal N}_\beta}\, I_\beta(u_1, \, u_2)$,则$d_\beta\leq d_{\beta, \delta_\beta}^{2, 2}$,若证$ d_{\beta, \delta_\beta}^{2, 2}\leq d_\beta$,可得$d_\beta=d_{\beta, \delta_\beta}^{2, 2}$.由(3.3)和(3.4)式可得$(\omega^+-\omega^-, \, u_2)\in {\cal N}_\beta$对任意$\beta < 0$都成立,则$d_{\beta, \delta_\beta}^{2, 2}\leq I_\beta(\omega^+-\omega^-, \, u_2)$.下证$ d_{\beta, \delta_\beta}^{2, 2}\leq d_\beta$,事实上,选取任意的$(u_0, v_0)\in {\cal N}_\beta$满足$I_\beta(u_0, \, v_0) < d^{2, 2}+1$,定义

$A_1:=\{u\in \mbox{span}\{u_0^+, \, u_0^- \}:\, |u|_{p+1}=1\}, $

$A_2:=\{v\in \mbox{span}\{v_0^+, \, v_0^- \}:\, |v|_{p+1}=1\}, $

则$A:=A_1\times A_2\in \Gamma ^{(2, 2)}$.对于任意的$(u, \, v)\in A$,存在$l_i\in{\mathbb R}, i=1, 2, 3, 4$,满足

$u=l_1\, u_0^++l_2\, u_0^-, \qquad v=l_3\, v_0^++l_4\, v_0^-.$

所以

$\begin{eqnarray*}\sup\limits_{t, \, s\geq 0}\, I_\beta(tu, \, sv)&=& \sup\limits_{t, \, s\geq0}\, I_\beta\big(t(l_1\, u_0^++l_2\, u_0^-), \, s(l_3\, v_0^++l_4\, v_0^-)\big)\\&=&\sup\limits_{t, \, s\geq0}\, I_\beta\big(t\, |l_1|\, u_0^+-t\, |l_2|\, u_0^-, \, s\, |l_3|\, v_0^+-s\, |l_4|\, v_0^-\big)\\&\leq& I_\beta(u_0, \, v_0), \end{eqnarray*}$

即

$\sup\limits_{(u_1, u_2)\in A }\sup\limits_{t, \, s\geq 0}\, I_\beta(tu, \, sv)\leq I_\beta(u_0, \, v_0) <d^{2, 2}+1, $

则$A\in \Gamma ^{(2, 2)}_\lambda$和

$d^{2, 2}_{\beta, \delta_\beta}\leq \sup\limits_{A \setminus P_{\delta_\beta}} \sup\limits_{t, \, s\geq 0}\, I_\beta(tu, \, sv)\leq I_\beta(u_0, \, v_0), $

因此$d^{2, 2}_{\beta, \delta_\beta}\leq d_\beta$.所以由(3.2)式可得

(3.5) $\begin{eqnarray}\label{35}\Big(\frac{1}{2}-\frac{1}{p+1}\Big)\, (\|u_1 \|_{\alpha_1}^2+\|u_2\|_{\alpha_2}^2)&=&\lim\limits_{\beta\rightarrow -\infty}\Big(\frac{1}{2}-\frac{1}{p+1}\Big)\, (\|u_{1, \beta} \|_{\alpha_1}^2+\|u_{2, \beta}\|_{\alpha_2}^2)\\&=&\lim\limits_{\beta\rightarrow -\infty}I_\beta(u_{1, \beta}, \, u_{2, \beta}) =\lim\limits_{\beta\rightarrow -\infty}\, d_{\beta, \delta_\beta}^{2, 2}\\&\leq& \lim\limits_{\beta\rightarrow -\infty}I_\beta(\omega^+-\omega^-, \, u_2)\\&=&\Big(\frac{1}{2}-\frac{1}{p+1}\Big)\, (\|\omega^+ \|_{\alpha_1}^2+\|\omega^- \|_{\alpha_1}^2+\|u_2\|_{\alpha_2}^2)\\&<&\Big(\frac{1}{2}-\frac{1}{p+1}\Big)\, (\|u_1\|_{\alpha_1}^2+\|u_2\|_{\alpha_2}^2).\end{eqnarray}$

得到矛盾.所以$\{u_i\neq 0\}$恰有两个连通分支.

假设$v_i\in H_0^1(\{u_i\neq 0\})$是方程(3.1)的任意一个变号解, $i=1, \, 2$.则对任意的$\beta < 0$,有$(v_1, \, u_2), \, (u_1, \, v_2)\in {\cal N}_\beta$.于是,有

$\begin{eqnarray*}\Big (\frac{1}{2}-\frac{1}{p+1}\Big)\, (\|u_1 \|_{\alpha_1}^2+\|u_2\|_{\alpha_2}^2)&=&\lim\limits_{\beta\rightarrow -\infty}I_\beta(u_{1, \beta}, \, u_{2, \beta})\\&\leq& \lim\limits_{\beta\rightarrow -\infty}I_\beta(v_1, \, u_2)\\&=&\Big(\frac{1}{2}-\frac{1}{p+1}\Big)\, (\|v_1 \|_{\alpha_1}^2+\|u_2\|_{\alpha_2}^2)\end{eqnarray*}$

和

$\begin{eqnarray*}\Big (\frac{1}{2}-\frac{1}{p+1}\Big)\, (\|u_1 \|_{\alpha_1}^2+\|u_2\|_{\alpha_2}^2)&=&\lim\limits_{\beta\rightarrow -\infty}I_\beta(u_{1, \beta}, \, u_{2, \beta})\\ &\leq &\lim\limits_{\beta\rightarrow -\infty}I_\beta(u_1, \, v_2)\\ &=&\Big(\frac{1}{2}-\frac{1}{p+1}\Big)\, (\|u_1 \|_{\alpha_1}^2+\|v_2\|_{\alpha_2}^2), \end{eqnarray*}$

即$\|u_i\|_{\alpha_i}^2\leq \|v_i\|_{\alpha_i}^2$, $i=1, \, 2$,所以$(u_1, \, u_2)$是方程(3.1)的最小能量变号解.

(3)若$k_1\geq 2$且$k_2=1$.由参考文献[11]可知, $u_{1, \beta}$变号, $u_{2, \beta}$是正的,所以在$\{u_2\neq 0\}$上$u_2>0$.假设$\{u_1\neq 0\}$至少有$k_1+1$个连通分支.定义

$\begin{eqnarray*} {\cal M}_\beta:&=&\Big\{(u_1, u_2)\in H:\, u_1\mbox{变号}, \, \{u_1\neq 0\} \mbox{至少有} k_1+1 \mbox{个连通分支} {\cal O}_k, \, u_2\neq0, \, u_2\geq 0, \\&&\ \ I_\beta'(u_1, \, u_2)(u_1\, \chi_{{\cal O}_k}, \, 0)=I_\beta'(u_1, \, u_2)(0, \, u_2)=0, \, \forall 1\leq k\leq k_1 \Big\}.\end{eqnarray*}$

假设$(u_1, u_2)\in {\cal M}_\beta$,则$\{u_1\neq 0\}$至少有$k_1+1$个连通分支${\cal O}_k\, (1\leq k\leq k_1+1)$.对于$ 1\leq k\leq k_1$, $u_1\, \chi_{{\cal O}_k}\in H^1_0(\Omega)$,由$I_\beta'(u_1, \, u_2)(u_1\, \chi_{{\cal O}_k}, \, 0)=0, \, I_\beta'(u_1, \, u_2)(0, \, u_2)=0$可知,有

$\|u_1\, \chi_{{\cal O}_k}\|_{\alpha_1}^2=\mu_1\, |u_1\, \chi_{{\cal O}_k}|_{p+1}^{p+1}-|\beta|\, \int_{\Omega} |u_1\, \chi_{{\cal O}_k}|^{\frac{p+1}{2}}|u_2|^{\frac{p+1}{2}}\, {\rm d}x, \quad 1\leq k\leq k_1$

和

$\|u_2\|_{\alpha_2}^2=\mu_2\, |u_2|_{p+1}^{p+1}-|\beta|\, \int_{\Omega} |u_1|^{\frac{p+1}{2}}|u_2|^{\frac{p+1}{2}}\, {\rm d}x.$

则有

$\begin{eqnarray*}&&\int_{\Omega}\bigg|\sum\limits_{k=1}^{k_1} t_k \, u_1\, \chi_{{\cal O}_k}\bigg|^{\frac{p+1}{2}}|s\, u_2 |^{\frac{p+1}{2}}\, {\rm d}x\\&\leq& \frac{1}{2}\sum\limits_{k=1}^{k_1} t_k^{p+1}\int_{\Omega} |u_1\, \chi_{{\cal O}_k}|^{\frac{p+1}{2}}|u_2|^{\frac{p+1}{2}}\, {\rm d}x+ \frac{1}{2} s^{p+1}\int_{\Omega} |u_1|^{\frac{p+1}{2}}|u_2|^{\frac{p+1}{2}}\, {\rm d}x\\&=&\frac{1}{2|\beta|}\bigg(\mu_1\, \sum\limits_{k=1}^{k_1} t_k^{p+1}|u_1\, \chi_{{\cal O}_k}|_{p+1}^{p+1}+\mu_2\, s^{p+1} |u_2|_{p+1}^{p+1}\bigg)\\&& -\frac{1}{2|\beta|}\bigg( \sum\limits_{k=1}^{k_1} t_k^{p+1}\|v_n\, \chi_{{\cal O}_k} \|_{\alpha_1}^2+s^{p+1} \|u_2\|_{\alpha_2}^2\bigg).\end{eqnarray*}$

由于$u_2\geq 0$,所以对于$t_1, \, \cdots, t_{k_1}, \, s\geq 0$,我们有

(3.6) $\begin{eqnarray}\label{36} I_\beta\bigg(\sum\limits_{k=1}^{k_1} t_k \, u_1\, \chi_{{\cal O}_k}, \, s\, u_2\bigg)&=&\frac{1}{2}\sum\limits_{k=1}^{k_1} t_k^2\|u_1\, \chi_{{\cal O}_k}\|_{\alpha_1}^2+ \frac{1}{2}s^{2}\|u_2\|_{\alpha_2}^2-\frac{\mu_1}{p+1}\sum\limits_{k=1}^{k_1} t_k^{p+1}|u_1\, \chi_{{\cal O}_k}|^{p+1}_{p+1}\\&& -\frac{\mu_2}{p+1} s ^{p+1}|u_2|^{p+1}_{p+1} + \frac{2|\beta|}{p+1} \int_{\Omega} \bigg|\sum\limits_{k=1}^{k_1} t_k \, u_1\, \chi_{{\cal O}_k}\bigg|^{\frac{p+1}{2}}|s\, u_2|^{\frac{p+1}{2}}\, {\rm d}x\\&\leq& \sum\limits_{k=1}^{k_1} \bigg(\frac{1}{2}t_k^2-\frac{1}{p+1}t_k^{p+1}\bigg) \|u_1\, \chi_{{\cal O}_k}\|_{\alpha_1}^2+ \bigg(\frac{1}{2}s^2-\frac{1}{p+1}s^{p+1}\bigg) \|u_2\|_{\alpha_2}^2\\&\leq&\bigg(\frac{1}{2}-\frac{1}{p+1}\bigg)\bigg(\sum\limits_{k=1}^{k_1}\|u_1\, \chi_{{\cal O}_k}\|_{\alpha_1}^2+ \|u_2\|_{\alpha_2}^2\bigg).\end{eqnarray}$

再定义集合

$A_1:=\Big\{u\in \mbox{span}\{u_1\, \chi_{{\cal O}_1}, \, \cdots, u_1\, \chi_{{\cal O}_{k_1}}\}:\, |u|_{p+1}=1\Big\}, $

$A_2:=\bigg\{\frac{u_2}{|u_2|_{p+1}}\bigg\}, $

则由引理$2.1(1)$可知$A:=A_1\times A_2\in \Gamma^{(k_1, \, 1)}$.于是由(3.6)式可知

$\begin{eqnarray*} \sup\limits_{A}\sup _{t, s\geq 0}\, I_\beta(tu_1, \, su_2)&\leq& \bigg(\frac{1}{2}-\frac{1}{p+1}\bigg)\bigg( \sum\limits_{k=1}^{k_1}\|u_1\, \chi_{{\cal O}_k}\|_{\alpha_1}^2+ \|u_2\|_{\alpha_2}^2\bigg)\\&< &\bigg(\frac{1}{2}-\frac{1}{p+1}\bigg)(\|u_1\|_{\alpha_1}^2+\|u_2\|_{\alpha_2}^2)\\& =&I_\beta(u_1, \, u_2)=d_{\beta, \delta_n}^{k_1, 1}\leq d^{k_1, 1}, \end{eqnarray*}$

所以$A\in \Gamma_\beta^{(k_1, 1)}$.于是

$\begin{eqnarray*} d_{\beta, \delta_n}^{k_1, 1}&\leq& \sup\limits_{A\setminus P_{\delta_n}}\sup _{t, s\geq 0}\, I_\beta(tu_1, \, su_2)\\&\leq& \bigg(\frac{1}{2}-\frac{1}{p+1}\bigg)\bigg(\sum\limits_{k=1}^{k_1}\|u_1\, \chi_{{\cal O}_k}\|_{\alpha_1}^2+\|u_2\|_{\alpha_2}^2\bigg) <I_\beta(u_1, \, u_2).\end{eqnarray*}$

矛盾.所以, $\{u_1\neq 0\}$至多有$k_1$个连通分支.

(4)若$(k_1, \, k_2)=(2, \, 1)$.则$\{u_1\neq 0\}$恰有两个连通分支.假设$\{u_2\neq 0\}$至少有两个连通分支${\cal O}_1$和${\cal O}_2$,则对任意的$\beta < 0$, $(u_1, \, u_2\chi_{{\cal O}_1})\in {\cal M}_\beta$.于是和(3.5)式类似讨论得到矛盾,所以$\{u_2\neq 0\}$连通.最后,和上面类似的讨论我们可证$u_1$是方程(3.1)当$i=1$的最小能量变号解,而$u_2$是方程(3.1)当$i=2$的最小能量变号解.