令${\Bbb R}_+^n=\{x=(x_1, x_2, \ldots, x_n) \in {\Bbb R}^n \mid x_n>0\}$是上半欧氏空间.该文考虑${\Bbb R}_+^n$上如下积分方程组

$ \begin{eqnarray}\label{MG-sys} \left\{\begin{array}{ll} u(x)=\int_{{\Bbb R}_+^n}\Big(\frac{1}{|x-y|^{n-\alpha}}-\frac{1}{|\bar{x}-y|^{n-\alpha}}\Big)f_1(u(y))g_1(v(y)){\rm d}y, \\ v(x)=\int_{{\Bbb R}_+^n}\Big(\frac{1}{|x-y|^{n-\alpha}}-\frac{1}{|\bar{x}-y|^{n-\alpha}}\Big)f_2(u(y))g_2(v(y)){\rm d}y, \end{array}\right. \end{eqnarray} $

(1.1)

其中$\bar{x}=(x_1, \cdots, x_{n-1}, -x_n)$是点$x$关于超平面$x_n=0$的对称点.若$n\geq 4$, 则$\alpha\in (0, n)$; 若$n=3$, 则$\alpha\in (0, 1)\cup (1, n)$.并且, 假设$f_i, g_i (i=1, 2):\, [0, \infty)\to [0, \infty)$满足如下条件:

(ⅰ) $f_i(t), g_i(s)$在$(0, \infty)$上非减;

(ⅱ) $F_i(t)=\frac{f_i(t)}{t^{p_i}}, G_i(s)=\frac{g_i(s)}{s^{q_i}}$在$(0, \infty)$上非增, 且$0<\alpha<n, $ $p_i + q_i=\frac{n+\alpha}{n-\alpha}.$并且, $F_i$和$ G_i$都是局部有界的.

假设$\alpha$是偶数, 容易验证方程组(1.1) 正的光滑解满足如下的Navier边值问题

$ \begin{eqnarray}\label{Diff} \left\{\begin{array}{ll} (-\Delta)^\frac\alpha2 u=f_1(u)g_1(v), &\quad\mbox{在}~{\Bbb R}_+^n~\mbox{中, }\\ (-\Delta)^\frac\alpha2 v=f_2(u)g_2(v), &\quad\mbox{在}~{\Bbb R}_+^n~\mbox{中, }\\ (-\Delta)^k u=0, &\quad\mbox{在}~\partial{\Bbb R}_+^n~\mbox{上, }\\ (-\Delta)^k v=0, &\quad\mbox{在}~\partial{\Bbb R}_+^n~\mbox{上, } \end{array}\right. \end{eqnarray} $

(1.2)

其中$k=0, 1, 2, \cdots, \frac{\alpha}{2}-1$.

Liouville型定理, 即非平凡解的不存在性定理, 在诸多领域中发挥着重要的作用, 例如先验估计的证明、全空间上的椭圆问题和半空间上的Neumann问题的非负解的研究.可参见文献[1-8]及其中的文献.

特别地, 当$f_i(u)=u^{p_i}$, $g_i(v)=v^{q_i}(i=1, 2), $ $\frac{n}{n-\alpha} < p_i+q_i\leq \frac{n+\alpha}{n-\alpha}$时, 文献[9]利用Kelvin变换和移动球面法, 对方程组(1.1) 的正解建立了Liouville型定理.对于单个方程, Cao和Dai^[1]利用Kelvin变换和移动平面法证明了积分方程

$ \begin{eqnarray*} u(x)=\int_{{\Bbb R}_+^n}\Big(\frac{1}{|x-y|^{n-2m}}-\frac{1}{|\bar{x}-y|^{n-2m}}\Big)u^{p}(y){\rm d}y \end{eqnarray*} $

的Liouville型定理.这个积分方程与带有Navier边值条件的多调和方程

$ \left\{ \begin{array}{*{35}{l}} {{(-\Delta )}^{m}}u={{u}^{p}}, & \quad \rm{在}~\mathbb{R}_{+}^{\mathit{n}}~\rm{中, } \\ u=(-\Delta )u=\cdots ={{(-\Delta )}^{\frac{\alpha }{2}-1}}u=0, & \quad \rm{在}~\partial \mathbb{R}_{+}^{\mathit{n}}~\rm{上, } \\ \end{array} \right. $

(1.3)

密切相关, 其中$0 < 2m < n$, $1 < p\le\frac{n+2m}{n-2m}$, 这里$m$是一个整数.

Fang和Chen^[10]考虑了如下单个积分方程

$ \begin{eqnarray}\label{Fang-Chen} u(x) = \int_{{\Bbb R}^n_+} G(x, y)u^p(y){\rm d}y, ~~1<p \leqslant \frac{n+2m}{n-2m}, \end{eqnarray} $

(1.4)

其中$G(x, y)$是${\Bbb R}^n_+$上的Green函数.在局部可积的条件下, Fang和Chen利用积分形式的移动平面法证明了积分方程(1.4)正解不存在.并且, Fang和Chen还证明了积分方程(1.4)与如下带有Dirichlet边值条件的多调和方程

$ \begin{eqnarray}\label{Reichel-Weth} \left\{\begin{array}{ll} (-\Delta)^m u=u^p, \quad&\mbox{在}~{\Bbb R}^n_+ ~\mbox{中}, \\[2mm] u=\frac{\partial}{\partial x_n}u=\cdots=\frac{\partial^{m-1}}{\partial^{m-1}x_n}u=0, \quad&\mbox{在}~\partial{\Bbb R}^n_+ ~\mbox{上} \end{array}\right. \end{eqnarray} $

(1.5)

等价, 其中, 若$2m\ge n$, 则$p>1$; 若$n>2m>0$, 则$1<p\le\frac{n+2m}{n-2m}$. Reichel和Weth^[8]利用Kelvin变换, 对Dirichlet问题(1.5)经典的非负有界解证明了Liouville型定理.特别地, 对于$m=1$的情况, Gidas和Spruck^[4]对于相应的Dirichlet问题证明了Liouville型定理; Li和Zhu^[7]分别在临界指数和次临界指数的条件下考虑了(1.5) 正解的不存在性.

注意到Dou和Li^[11]在上半空间中建立了如下方程组

$ \begin{equation}\label{Dou-Li} \left\{\begin{array}{ll} u(x)=\int_{{\Bbb R}^{n-1}} \frac{1}{|x-y|^{n-\alpha}} {f(v(y))}{\rm d}y, &\quad x\in{\Bbb R}_+^{n}, \\[3mm] v(y)=\int_{{\Bbb R}^{n}_+} \frac{1}{|x-y|^{n-\alpha}} {g(u(x))} {\rm d}x, &\quad y\in\partial{\Bbb R}^{n}_+ \end{array}\right. \end{equation} $

(1.6)

的Liouville型定理, 其中$1<\alpha<n$, 且$f, g$满足类似于(ⅰ)和(ⅱ)的某些自然结构条件.特别地, 当$f(v)=v^{\frac{n+\alpha}{n-\alpha}}$及$g(u)=u^{\frac{n+\alpha-2}{n-\alpha}}$时, Dou和Zhu^[12]表明方程组(1.6) 恰好是上半空间相应于HLS不等式的Euler-Lagrange方程.并且, 利用积分形式的移动球面法及Li-Zhu引理^[7], Dou和Zhu对方程组(1.6) 的正解进行了分类.

在全空间${\Bbb R}^{n}$的情形下, 存在类似于方程组(1.1) 的积分方程组

$ \begin{equation}\label{Dou-Qu1} \left\{\begin{array}{ll} u(x)=\int_{{\Bbb R}^n} \frac{1}{|x-y|^{n-\alpha}} u^{p_1}(y)v^{q_1}(y){\rm d}y, \\[4mm] v(x)=\int_{{\Bbb R}^n}\frac{1}{|x-y|^{n-\alpha}} u^{p_2}(y)v^{q_2}(y){\rm d}y, \end{array}\right. \end{equation} $

(1.7)

其中指数满足: $\frac{n}{n-\alpha} < p_i+q_i\leq \frac{n+\alpha}{n-\alpha}\, (i=1, 2).$ Dou, Qu^[13]和Dou^[14]分别利用积分形式的移动平面法和积分形式的移动球面法对方程组(1.7) 给出正解的分类.

在$p_1=q_2=0, \, q_1=q, \, p_2=p$的情形下, 积分方程组(1.7) 简化为

$ \begin{equation}\label{whole-space} \left\{\begin{array}{ll} u(x)=\int_{{\Bbb R}^{n}} \frac{1}{|x-y|^{n-\alpha}}{v^{q}(y)}{\rm d}y, &\quad x\in{\Bbb R}^{n}, \\[4mm] v(y)=\int_{{\Bbb R}^{n}} \frac{1}{|x-y|^{n-\alpha}} {u^{p}(x)} {\rm d}x, &\quad y\in{\Bbb R}^{n}, \end{array}\right. \end{equation} $

(1.8)

其中$1 < p, q < \infty$满足$\frac{1}{p+1}+\frac{1}{q+1}=\frac{n-\alpha}{n}.$方程组(1.8) 的解恰好是${\Bbb R}^{n}$上HLS不等式的极值函数.通过引入积分形式的移动平面法, 在自然可积条件$u\in L^{p+1}({\Bbb R}^{n})$及$v\in L^{q+1}({\Bbb R}^{n})$下, Chen, Li和Ou^[15]对方程组(1.8) 的正解进行了分类.后来, Hang^[16]把这个结果发展到$\frac \alpha {n-\alpha}<p, q<\infty$及$u\in L^{p+1}_{\mbox{loc}}({\Bbb R}^{n})$的情形下.对于次临界指数$\frac{1}{p+1}+\frac{1}{q+1}>\frac{n-\alpha}{n}$的情形, 当$0<p\leq\frac{n}{n-\alpha}$或$0<q\leq\frac{n}{n-\alpha}$时, Chen和Li^[17]证明了方程组(1.8) 正解的不存在性.特别地, 当$u=v$及$p=q=\frac{2n}{n-\alpha}$时, 方程组(1.8) 简化为单个方程. Chen, Li和Ou^[18]利用积分形式的移动平面法给出方程正解的分类.同时, Li^[5]利用积分形式的移动球面法给出方程正解的分类.

然而, 到目前为止, 上半空间${\Bbb R}_+^n$方程组(1.1) 正解的Liouville型定理还没有得到建立.受到文献[9, 11]中结果的启发, 本文将利用积分形式的移动球面法对方程组(1.1) 建立正解的Liouville型定理.本文的主要结论有:

定理1.1  假设$f_i, g_i:\, [0, \infty)\to [0, \infty)$是非负函数且满足条件(ⅰ)和(ⅱ).并且, $(u, v)$是方程组(1.1) 的非负解, 且满足条件

$ ({\rm H}) \qquad u, \, v, \, F_i^{\frac1 {p_i}}(u )u, \, F_i^{\frac 1 {p_i-1}}(u )u, \, G_i^{\frac 1 {q_i}}(v )v, \, G_i^{\frac 1 {q_i-1}}(v )v \in{L^{\theta}({\Bbb R}^{n}_+)}, $

其中$\theta=\frac {2n}{n-\alpha}$.则对任意的$\frac{n}{n-\alpha}<s<\infty $, 有$ u, v\in L^s ({\Bbb R}_+^n)\bigcap L^\infty({\Bbb R}_+^n) $.并且, $u, v \in C^\infty({\Bbb R}_+^n)$.

定理1.2  在定理1.1的条件下, 方程组(1.1) 只有平凡解.

注1  定理1.2的证明主要采用积分形式的移动球面法, 这个方法首次由Li和Zhu^[7]提出.随后, Li, Zhang^[6]和Li^[5]分别改进了Li和Zhu的结果.移动球面法可以快速地表明具有共形不变的椭圆方程解的表达式, 并且还可以证明具有次临界指数的椭圆方程解的不存在性, 无需事先给出方程解的对称性.

注2  近几年来, 非线性方程解的性质逐渐成为研究热点.例如, 赵金虎等^[19]利用移动平面法研究了一类完全非线性椭圆型方程组解的对称性.胡良根^[20]利用Pohozaev恒等式研究了非线性Hénon方程解的Liouville型定理.

本文组织如下:第$2$节证明方程组(1.1) 正解的正则性结论(定理1.1).第$3$节给出本文要用到的一些引理.第$4$节利用积分形式的移动球面法证明定理1.2.

在这一节, 我们证明方程组(1.1) 正解的正则性结果.对于$R>0, $记

$ \ \ \ \ \ \ \ \ \ \ \ B_R(x)=\{y\in{\Bbb R}^n\, |\, |y-x|\leq R, x\in{\Bbb R}^n\}, \\ B_R^+(x)=\{y=(y_1, y_2, \cdots, y_n)\in B_R(x)\, |\, y_n>0, x\in\partial{\Bbb R}^{n}_+\}, \\ \ \ \ \ \ \ \ \ \ \ \ \Sigma_{x, \lambda}=\{y\in{\Bbb R}_+^n:\ |y-x|\geq\lambda, \ x\in\partial{\Bbb R}_+^n\}. $

当$x=0$时, 记$B_R=B_R(0), B_R^+=B_R^+(0), \Sigma_{R}=\Sigma_{0, R}$.在后文中, 用$\|u\|_p$来表示相应区域上的$L^p$范数.

首先, 回顾${\Bbb R}^n_+$上的HLS不等式(可参见文献[21-22]):令$C(n, \alpha, p)$是一个正常数, 且定义

$ Tf(x)=\int_{{\Bbb R}_+^n} f(y)|x-y|^{n-\alpha}{\rm d}y, $

则对任意的$p>\frac{n}{n-\alpha}$, 有

$ \begin{equation}\label{HLS} \|Tf\|_p\le C(n, \alpha, p)\|f\|_{\frac{np}{n+\alpha p}}. \end{equation} $

(2.1)

定义

$u_R(x)= \left\{\begin{array}{ll}u(x), \quad&\mbox{若}~~|u(x)|>R \quad\mbox{或}\quad |x|>R, \\ 0, \quad& \mbox{否则}, \end{array}\right. $

及

$ u_b(x)=u(x)-u_R(x). $

显然, $u_b(x)$有界, 且$\mbox{supp}u_b \subset B_R^+(0)$.类似地定义$v_R$及$v_b$.对$\frac n{n-\alpha}<s<\infty$, 假设$\phi, \varphi \in L^s({\Bbb R}_+^n) $.记

$ T_1(\varphi)=\int_{{\Bbb R}_+^n} \Big(\frac{1}{|x-y|^{n-\alpha}}-\frac{1}{|\bar{x}-y|^{n-\alpha}}\Big) F_1(u_R(y))G_1(v_R(y))u_R^{p_1}(y)v_R^{q_1-1}(y)\varphi(y){\rm d}y.\\ T_2(\phi)=\int_{{\Bbb R}_+^n} \Big(\frac{1}{|x-y|^{n-\alpha}}-\frac{1}{|\bar{x}-y|^{n-\alpha}}\Big) F_2(u_R(y))G_2(v_R(y))u_R^{p_2-1}(y)v_R^{q_2}(y)\phi(y){\rm d}y.\\ H_R(x)=\int_{{\Bbb R}_+^n} \Big(\frac{1}{|x-y|^{n-\alpha}}-\frac{1}{|\bar{x}-y|^{n-\alpha}}\Big) F_1(u_b(y))G_1(v_b(y))u_b^{p_1}(y)v_b^{q_1-1}(y)v(y){\rm d}y.\\ I_R(x)=\int_{{\Bbb R}_+^n} \Big(\frac{1}{|x-y|^{n-\alpha}}-\frac{1}{|\bar{x}-y|^{n-\alpha}}\Big) F_2(u_b(y))G_2(v_b(y))u_b^{p_2-1}(y)v_b^{q_2}(y)u(y){\rm d}y. $

定义$L^{s}({\Bbb R}_+^n)\times L^{s}({\Bbb R}_+^n)$上的范数为

$ \|(\varphi, \phi)\|_{s\times s}=\|\varphi\|_{s}+\|\phi\|_{s}, $

并且, 记$T\, :\, L^{s}({\Bbb R}_+^n)\times L^{s}({\Bbb R}_+^n)\to L^{s}({\Bbb R}_+^n)\times L^{s}({\Bbb R}_+^n)$为映射

$ T(\varphi, \phi)=(T_1(\varphi), T_2(\phi)). $

定理1.1的证明  第一步. 首先对任意的$s> \frac{n}{n-\alpha}$, 证明$u, v\in L^s ({\Bbb R}_+^n)$.事实上, 对任意的$\varphi, \phi\in L^s({\Bbb R}_+^n)$, 由HLS不等式(2.1)及Hölder不等式, 有

$ \begin{eqnarray}\label{reg-1} \|T_1(\varphi)\|_{s} &\leq& C \big\|F_1(u_R)G_1(v_R)u_R^{p_1}v_R^{q_1-1}\varphi\big\| _{\frac{ns}{n+\alpha s}}\nonumber\\ &\leq& C \big\|{F_1^{1/{p_1}}(u_R)u_R}\big\|^{p_1}_{\theta}\big\|{G_1^{1/{q_1-1}}(v_R)v_R}\big\|^{q_1-1}_{\theta} \|\varphi\|_{s}, \end{eqnarray} $

(2.2)

其中$\theta=\frac{n(p_1+q_1-1)}{\alpha}=\frac{2n}{n-\alpha}$及$p_1 + q_1=\frac{n+\alpha}{n-\alpha}.$

由于${F_1^{1/{p_1}}(u)u}, $$\, G_1^{1/{q_1-1}}(v)v\in L^{\theta}({\Bbb R}_+^n), $可以选择充分大的$R$使得如下不等式成立

$ C \big\|{F_1^{1/{p_1}}(u_R)u_R}\big\|^{p_1}_{\theta}\big\|{G_1^{1/{q_1-1}}(v_R)v_R}\big\|^{q_1-1}_{\theta}\leq \frac{1}{2}. $

因此, 由(2.2) 式推出

$ \begin{eqnarray}\label{reg-2} \|T_1 (\varphi)\|_{s }\leq \frac{1}{2}\|\varphi\|_{s} . \end{eqnarray} $

(2.3)

类似地, 有

$ \begin{eqnarray}\label{reg-3} \|T_2 (\phi)\|_{s }\leq \frac{1}{2}\|\phi\|_{s}. \end{eqnarray} $

(2.4)

从(2.3) 式及(2.4) 式, 推出

$ \|T(\varphi, \phi)\|_{s\times s}\le\frac{1}{2}\Big(\|\varphi\|_{s}+\|\phi\|_s\Big). $

因此, $T$是从$L^{s}({\Bbb R}_+^n)\times L^{s}({\Bbb R}_+^n)$到自身的压缩映射.

根据(1.1) 式, 容易证明

$ (u, v) =T (u, v) +(H_R, I_R). $

由于函数$u_b$和函数$v_b$在${\Bbb R}_+^n$中都是一致有界且有紧支集, 且$F_i, \, G_i \, (i=1, 2)$都是局部有界的, 因此从(2.1) 式推出

$ H_R, \, I_R\in L^s({\Bbb R}_+^n). $

根据正则提升定理(参见文献[22, 定理3.3.1]),

$ u, v\in L^s({\Bbb R}_+^n), \quad \frac{n}{n-\alpha}<s<\infty. $

利用类似文献[13]中的推导, 可以证明$u, v\in L^\infty({\Bbb R}_+^n)$.

第二步.对$R>0$, 记

$ u_1(x)=\int_{B_R^+(0)}\Big(\frac{1}{|x-y|^{n-\alpha}}-\frac{1}{|\bar{x}-y|^{n-\alpha}}\Big)F_1(u(y))G_1(v(y))u^{p_1}(y)v^{q_1}(y){\rm d}y, \\ v_1(x)=\int_{B_R^+(0)}\Big(\frac{1}{|x-y|^{n-\alpha}}-\frac{1}{|\bar{x}-y|^{n-\alpha}}\Big)F_2(u(y))G_2(v(y))u^{p_2}(y)v^{q_2}(y){\rm d}y, \\ u_2(x)=\int_{{\Bbb R}_+^n\backslash B_R^+(0)}\Big(\frac{1}{|x-y|^{n-\alpha}}-\frac{1}{|\bar{x}-y|^{n-\alpha}}\Big)F_1(u(y))G_1(v(y))u^{p_1}(y)v^{q_1}(y){\rm d}y, \\ v_2(x)=\int_{{\Bbb R}_+^n\backslash B_R^+(0)}\Big(\frac{1}{|x-y|^{n-\alpha}}-\frac{1}{|\bar{x}-y|^{n-\alpha}}\Big)F_2(u(y))G_2(v(y))u^{p_2}(y)v^{q_2}(y){\rm d}y. $

于是, 我们有

$ u(x)=u_1(x)+u_2(x)\quad\mbox{及}\quad v(x)=v_1(x)+v_2(x). $

注意到$u_2, v_2\in C^\infty({\Bbb R}_+^n)$, $R>0$, 且$u_1$及$v_1$在$B_R^{+}(0)\backslash\{0\}$中是Hölder连续的.因为$R$是任意的, 所以$u, v$在${\Bbb R}^{n}_+$中至少是Hölder连续的.故由靴套定理推出$u, v\in C^\infty({{\Bbb R}^{n}_+})$.

在这一节中, 我们列出几个本文要用到的引理.首先, Kelvin变换定义为

$ \omega_{x, \lambda}(\xi)=\left(\frac\lambda{|\xi-x|}\right)^{n-\alpha}\omega(\xi^{x, \lambda}), \quad\xi\in {\Bbb R}^n_+, $

其中$\xi^{x, \lambda}=x+\frac{\lambda^2(\xi-x)}{|\xi-x|^2}, x\in\partial {\Bbb R}^n_+, \lambda>0.$

我们首先建立如下引理.

引理3.1  令$1<\alpha<n, $且$(u, v)$是方程组(1.1) 的一组正解.则对$x\in \partial{\Bbb R}^{n}$, 有

$ \begin{eqnarray} u_{x, \lambda}(\xi) &=&\int_{{\Bbb R}^n_+}\Big(\frac{1}{|\xi-z|^{n-\alpha}}-\frac{1}{|\bar{\xi}-z|^{n-\alpha}}\Big)\nonumber\\ &&\cdot \bigg[F_1\Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}u_{x, \lambda}(z)\Big) G_1\Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}v_{x, \lambda}(z)\Big) u_{x, \lambda}^{p_1}(z)v_{x, \lambda}^{q_1}(z)\bigg]{\rm d}z, \label{K-1} \end{eqnarray} $

(3.1)

$ \begin{eqnarray} v_{x, \lambda}(\xi) &=&\int_{{\Bbb R}^n_+}\Big(\frac{1}{|\xi-z|^{n-\alpha}}-\frac{1}{|\bar{\xi}-z|^{n-\alpha}}\Big)\nonumber\\ &&\cdot \bigg[F_2\Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}u_{x, \lambda}(z)\Big) G_2\Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}v_{x, \lambda}(z)\Big) u_{x, \lambda}^{p_2}(z)v_{x, \lambda}^{q_2}(z)\bigg]{\rm d}z.\label{K-2} \end{eqnarray} $

(3.2)

并且,

$ \begin{eqnarray} & &u_{x, \lambda}(\xi)-u(\xi)\nonumber\\ & =&\int_{\Sigma_{x, \lambda}}P(x, \lambda;\xi, z) \bigg[F_1\Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}u_{x, \lambda}(z)\Big) G_1\Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}v_{x, \lambda}(z)\Big) u_{x, \lambda}^{p_1}(z)v_{x, \lambda}^{q_1}(z)\nonumber\\ & & -F_1(u(z))G _1(v(z)) u^{p_1}(z)v^{q_1}(z)\bigg]{\rm d}z, \label{K-3} \end{eqnarray} $

(3.3)

$ \begin{eqnarray} & &v_{x, \lambda}(\xi)-v(\xi)\nonumber\\ & =&\int_{\Sigma_{x, \lambda}}P(x, \lambda;\xi, z) \bigg[F_2\Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}u_{x, \lambda}(z)\Big) G_2\Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}v_{x, \lambda}(z)\Big) u_{x, \lambda}^{p_2}(z)v_{x, \lambda}^{q_2}(z)\nonumber\\ & &-F_2(u(z))G _2(v(z)) u^{p_2}(z)v^{q_2}(z)\bigg]{\rm d}z, \label{K-4} \end{eqnarray} $

(3.4)

其中,

$ P(x, \lambda;\xi, z)=\Big(\frac{1}{|\xi-z|^{n-\alpha}}-\frac{1}{|\bar{\xi}-z|^{n-\alpha}}\Big) -\Big(\frac\lambda{|\xi-x|}\Big)^{n-\alpha}\Big(\frac{1}{|\xi^{x, \lambda}-z|^{n-\alpha}}-\frac{1}{|\bar{\xi}^{x, \lambda}-z|^{n-\alpha}}\Big). $

并且, 我们有

$ P(x, \lambda;\xi, z)>0, ~~~~~\mbox{对}~\forall \, \xi, \, z\in\Sigma_{x, \lambda}, \, \lambda>0. $

证  对任意的$x\in\partial{\Bbb R}^n_+, z\in{\Bbb R}^n_+$, 令

$ y=z^{x, \lambda}=x+\frac{\lambda^2(z-x)}{|z-x|^2}. $

则我们有

$ {\rm d}y=\Big(\frac{\lambda}{|z-x|}\Big)^{2n}{\rm d}z. $

为简单起见, 记

$ A^+(\xi^{x, \lambda})=\int_{\Sigma_{x, \lambda}}\frac{f_1(u(y))g_1(v(y))}{|\xi^{x, \lambda}-y|^{n-\alpha}}{\rm d}y, \quad A^-(\xi^{x, \lambda})=\int_{B^+_\lambda(x)}\frac{f_1(u(y))g_1(v(y))}{|\xi^{x, \lambda}-y|^{n-\alpha}}{\rm d}y, $

因此, 方程组(1.1) 中的$u(\xi^{x, \lambda})$重新记为

$ u(\xi^{x, \lambda})=(A^+(\xi^{x, \lambda})+A^-(\xi^{x, \lambda})) -(A^+(\bar{\xi}^{\bar{x}, \lambda})+A^-(\bar{\xi}^{\bar{x}, \lambda})), ~\mbox{对}~~\xi\in{\Bbb R}^n_+, \, x\in\partial {\Bbb R}^n_+. $

令$y=z^{x, \lambda}$, 直接计算得

$\begin{eqnarray*} A^+(\xi^{x, \lambda})&=&\int_{B^+_\lambda(x)}\frac{f_1(u(z^{x, \lambda}))g_1(v(z^{x, \lambda}))}{|\xi^{x, \lambda}-z^{x, \lambda}|^{n-\alpha}} \Big(\frac{\lambda}{|z-x|}\Big)^{2n}{\rm d}z\\ &=&\int_{B^+_\lambda(x)}\frac{F_1(u(z^{x, \lambda})) G_1(v(z^{x, \lambda})) u_{x, \lambda}^{p_1}(z)v_{x, \lambda}^{q_1}(z)}{|\xi^{x, \lambda}-z^{x, \lambda}|^{n-\alpha}} \Big(\frac{\lambda}{|z-x|}\Big)^{2n-\tau(n-\alpha)}{\rm d}z, \end{eqnarray*} $

其中$\tau:=p_1 + q_1=\frac{n+\alpha}{n-\alpha}.$利用如下关系式^[5]

$ \frac{|z-x|}\lambda\frac{|\xi-x|}\lambda|\xi^{x, \lambda}-z^{x, \lambda}|=|\xi-z|, $

我们推出

$ \begin{eqnarray*} A^+_{x, \lambda}(\xi)&=&\Big(\frac\lambda{|\xi-x|}\Big)^{n-\alpha}A^+(\xi^{x, \lambda})\\ &=&\int_{B^+_\lambda(x)}\frac{F_1(\big(\frac{|z-x|}{\lambda}\big)^{n-\alpha}u_{x, \lambda}(z)) G_1(\big(\frac{|z-x|}{\lambda}\big)^{n-\alpha}v_{x, \lambda}(z)) u_{x, \lambda}^{p_1}(z)v_{x, \lambda}^{q_1}(z)}{|\xi-z|^{n-\alpha}}{\rm d}z. \end{eqnarray*} $

类似地, 我们有

$ A^-_{x, \lambda}(\xi)=\int_{\Sigma_{x, \lambda}}\frac{F_1( \big(\frac{|z-x|}{\lambda}\big)^{n-\alpha}u_{x, \lambda}(z)) G_1(\big(\frac{|z-x|}{\lambda}\big)^{n-\alpha}v_{x, \lambda}(z)) u_{x, \lambda}^{p_1}(z)v_{x, \lambda}^{q_1}(z)}{|\xi-z|^{n-\alpha}}{\rm d}z, \\ A^+_{x, \lambda}(\bar{\xi}) =\Big(\frac\lambda{|\bar{\xi}-x|}\Big)^{n-\alpha}A^+(\bar{\xi}^{x, \lambda})\\ \ \ \ \ \ \ \ \ \ \ \ \ =\int_{B^+_\lambda(x)}\frac{F_1(\big(\frac{|z-x|}{\lambda}\big)^{n-\alpha}u_{x, \lambda}(z)) G_1(\big(\frac{|z-x|}{\lambda}\big)^{n-\alpha}v_{x, \lambda}(z)) u_{x, \lambda}^{p_1}(z)v_{x, \lambda}^{q_1}(z)}{|\bar{\xi}-z|^{n-\alpha}}{\rm d}z, \\ A^-_{x, \lambda}(\bar{\xi}) =\int_{\Sigma_{x, \lambda}}\frac{F_1(\big(\frac{|z-x|}{\lambda}\big)^{n-\alpha}u_{x, \lambda}(z)) G_1(\big(\frac{|z-x|}{\lambda}\big)^{n-\alpha}v_{x, \lambda}(z)) u_{x, \lambda}^{p_1}(z)v_{x, \lambda}^{q_1}(z)}{|\bar{\xi}-z|^{n-\alpha}}{\rm d}z, $

其中, 对任意的$x\in\partial{\Bbb R}^n_+, \xi\in{\Bbb R}^n_+, $用到了如下关系式:

$ |\xi-x|=|\bar{\xi}-x|, \quad |\bar{\xi}^{x, \lambda}-x|=|\xi^{x, \lambda}-x|. $

因此, 我们有

$ \begin{eqnarray*} u_{x, \lambda}(\xi) &=&\Big(A^+_{x, \lambda}(\xi)+A^-_{x, \lambda}(\xi)\Big) -\Big(A^+_{x, \lambda}(\bar{\xi})+A^-_{x, \lambda}(\bar{\xi})\Big)\\ &=&\int_{{\Bbb R}^n_+}\Big(\frac{1}{|\xi-z|^{n-\alpha}}-\frac{1}{|\bar{\xi}-z|^{n-\alpha}}\Big)\nonumber\\ && \cdot\bigg[F_1 \Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}u_{x, \lambda}(z)\Big) G_1\Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}v_{x, \lambda}(z)\Big) u_{x, \lambda}^{p_1}(z)v_{x, \lambda}^{q_1}(z)\bigg]{\rm d}z. \end{eqnarray*} $

这表明(3.1) 式成立.类似地可以推出(3.2) 式成立.

下面, 证明(3.3) 式.首先, 我们有

$ \begin{eqnarray}\label{K-5} u_{x, \lambda}(\xi)-u(\xi) &=& \big(A^+_{x, \lambda}(\xi)+A^-_{x, \lambda}(\xi)\big) -\big(A^+_{x, \lambda}(\bar{\xi})+A^-_{x, \lambda}(\bar{\xi})\big)\nonumber\\ & &-\Big[\big(A^+(\xi)+A^-(\xi)\big) -(A^+(\bar{\xi})+A^-(\bar{\xi}))\Big]\nonumber\\ &=& \Big[\big(A^-_{x, \lambda}(\xi)-A^-_{x, \lambda}(\bar{\xi})\big) -(A^+(\xi)-A^+(\bar{\xi}))\Big]\nonumber\\ & &+\Big[\big(A^+_{x, \lambda}(\xi)-A^+_{x, \lambda}(\bar{\xi})\big) -(A^-(\xi)-A^-(\bar{\xi}))\Big]\nonumber\\ &=&\int_{\Sigma_{x, \lambda}}\Big(\frac{1}{|\xi-z|^{n-\alpha}}-\frac{1}{|\bar{\xi}-z|^{n-\alpha}}\Big)\nonumber\\ & &\cdot\bigg[F_1\Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}u_{x, \lambda}(z)\Big) G_1\Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}v_{x, \lambda}(z)\Big) u_{x, \lambda}^{p_1}(z)v_{x, \lambda}^{q_1}(z)\nonumber\\ &&-F_1(u(z))G _1(v(z)) u^{p_1}(z)v^{q_1}(z)\bigg]{\rm d}z\nonumber\\ &&+\big[A^+_{x, \lambda}(\xi)-A^+_{x, \lambda}(\bar{\xi})\big]-\big[(A^-(\xi)-A^-(\bar{\xi}))\big]. \end{eqnarray} $

(3.5)

从$A^-_{x, \lambda}(\xi)$及$A^-_{x, \lambda}(\bar{\xi})$的计算过程, 容易证明

$ \begin{eqnarray*} A^+_{x, \lambda}(\xi)-A^+_{x, \lambda}(\bar{\xi}) &=&\Big(\frac\lambda{|\xi-x|}\Big)^{n-\alpha}\Big(A^+(\xi^{x, \lambda})-A^(\bar{\xi}^{x, \lambda})\Big)\\ &=&\Big(\frac\lambda{|\xi-x|}\Big)^{n-\alpha} \int_{\Sigma_{x, \lambda}}\Big(\frac{1}{|\xi^{x, \lambda}-z|^{n-\alpha}}-\frac{1}{|\bar{\xi}^{x, \lambda}-z|^{n-\alpha}}\Big) \\ &&\cdot F_1(u(z))G _1(v(z)) u^{p_1}(z)v^{q_1}(z){\rm d}z. \end{eqnarray*} $

从$A^+_{x, \lambda}(\xi)$及$A^+_{\bar{x}, \lambda}(\bar{\xi})$的计算过程, 并利用$(\xi^{x, \lambda})^{x, \lambda}=\xi, $ $(\omega_{x, \lambda})_{x, \lambda}=\omega$及$|\xi^{x, \lambda}-x|=\frac{\lambda^2}{|\xi-x|}$, 推出

$ \begin{eqnarray*} &&A^-(\xi)-A^-(\bar{\xi})\\ &=&A^-((\xi^{x, \lambda})^{x, \lambda})-A^-(\bar{\xi}^{x, \lambda})^{x, \lambda})\\ & =&\int_{B^+_\lambda(x)}\Big(\frac{1}{|(\xi^{x, \lambda})^{x, \lambda} -z|^{n-\alpha}}-\frac{1}{|({\bar{\xi}}^{x, \lambda})^{x, \lambda}-z |^{n-\alpha}}\Big)\\ &&\cdot F_1(u(z))G _1(v(z)) u^{p_1}(z)v^{q_1}(z){\rm d}z\\ &=&\Big(\frac\lambda{|\xi^{x, \lambda}-x|}\Big)^{\alpha-n} \int_{\Sigma_{x, \lambda}}\Big(\frac{1}{|\xi^{x, \lambda}-z|^{n-\alpha}}-\frac{1}{|\bar{\xi}^{x, \lambda}-z|^{n-\alpha}}\Big)\\ & &\cdot\bigg[F_1 \Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}u_{x, \lambda}(z)\Big) G_1\Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}v_{x, \lambda}(z)\Big) u_{x, \lambda}^{p_1}(z)v_{x, \lambda}^{q_1}(z)\bigg]{\rm d}z\\ &=&\Big(\frac\lambda{|\xi-x|}\Big)^{n-\alpha} \int_{\Sigma_{x, \lambda}}\Big(\frac{1}{|\xi^{x, \lambda}-z|^{n-\alpha}}-\frac{1}{|\bar{\xi}^{x, \lambda}-z|^{n-\alpha}}\Big)\\ & &\cdot\bigg[F_1 \Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}u_{x, \lambda}(z)\Big) G_1\Big(\Big(\frac{|z-x|}{\lambda}\Big)^{n-\alpha}v_{x, \lambda}(z)\Big) u_{x, \lambda}^{p_1}(z)v_{x, \lambda}^{q_1}(z)\bigg]{\rm d}z. \end{eqnarray*} $

将上面两个不等式带入(3.5) 式, 可以得到(3.3) 式.类似地, (3.4) 式也可得证.与文献[9, 11-12]中的推导类似, 有$ P(x, \lambda;\xi, z)>0, $对$ \forall |\xi-x|, \, |z-x|\geq\lambda>0$.因此, 引理3.1得证.

特别地, 当$F_i, G_i(i=1, 2)$为常数时, (3.1) 式与(3.2) 式是方程组(1.1) 的共形不变形式, 参见文献[23]及其中的文献.

本文还需要如下的两个引理.第一个引理由文献[5]给出, 并且在更强假设下的结论由Li和Zhu^[7]及Li和Zhang^[6]给出.第二个引理由Dou和Zhu^[12]给出, 它把文献[5]中的引理$5.7$推广到了上半空间.

引理3.2^{[5, 引理5.8]}  令$n\ge 1$, $\mu \in {\Bbb R}$, 及$f\in C^0({\Bbb R}^n).$假设对任意的$x\in{\Bbb R}^n$, 存在$\lambda>0$使得

$ \Big(\frac\lambda{|y-x|}\Big)^\mu f\Big(x+\frac{\lambda^2(y-x)}{|y-x|^2}\Big)= f(y), \ \ \quad ~ \forall y\in{\Bbb R}^n\setminus\{x\}. $

因此, 存在$a\ge0, d>0$及$\bar{x}\in{\Bbb R}^n$, 使得

$ f(x)\equiv\pm a\Big(\frac1 {d+|x-\bar{x}|^2}\Big)^\frac{\mu}2. $

引理3.3^{[12, 引理3.7]}  对$n\ge 1$及$\mu \in {\Bbb R}$, 若$f$是定义在${\Bbb R}^n_+$上的实值函数且满足

$ \Big(\frac\lambda{|y-x|}\Big)^\mu f\Big(x+\frac{\lambda^2(y-x)}{|y-x|^2}\Big)\le f(y), \quad\forall\lambda>0, ~ y\in\Sigma_{x, \lambda}, y\in{\Bbb R}^n_+, x\in\partial{\Bbb R}^n_+, $

则我们有

$ f(z)=f(z', t)=f(0, t), ~~~~~~\forall\, z=(z', t)\in{\Bbb R}^n_+. $

本节利用积分形式的移动球面法证明定理1.2.记

$ \begin{eqnarray*} \Sigma_{x, \lambda}^{u}=\{y\in\Sigma_{x, \lambda}:\ u(y)<u_{x, \lambda}(y)\}, ~~\Sigma_{x, \lambda}^{v}=\{y\in\Sigma_{x, \lambda}:\ v(y)<v_{x, \lambda}(y)\}. \end{eqnarray*} $

首先建立方程组(1.1) 正解的单调性引理.

引理4.1  令$u$是方程组(1.1) 的正解.则对任意的$x\in\partial{\Bbb R}_+^n$, 存在$\lambda_0(x)>0$使得

$ \begin{equation}\label{monotony-1} u_{x, \lambda}(\xi)\leq u(\xi), \quad v_{x, \lambda}(\xi)\leq v(\xi), \quad\quad\forall 0<\lambda<\lambda_0(x), \ \forall \xi\in\Sigma_{x, \lambda}. \end{equation} $

(4.1)

证  利用(3.3) 式, 假设(ⅰ), (ⅱ)及平均值定理, 对$\xi\in\Sigma_{x, \lambda}^{u}$, 我们有

$ \begin{eqnarray}\label{monotony-2} 0&\leq &u_{x, \lambda}(\xi)-u(\xi)\nonumber\\ &\leq&\int_{\Sigma_{x, \lambda}}P(x, \lambda;\xi, z) \big[F_1(u_{x, \lambda}(z)) G_1(v_{x, \lambda}(z)) u_{x, \lambda}^{p_1}(z)v_{x, \lambda}^{q_1}(z)\nonumber\\ &&-F_1(u(z))G _1(v(z)) u^{p_1}(z)v^{q_1}(z)\big]{\rm d}z\nonumber\\ &=&\int_{\Sigma_{x, \lambda}}P(x, \lambda;\xi, z) f_1(u_{x, \lambda}(z)) \big[g_1(v_{x, \lambda}(z))-g _1(v(z))\big]{\rm d}z\nonumber\\ && +\int_{\Sigma_{x, \lambda}}P(x, \lambda;\xi, z) g_1(v(z)) \big[f_1(u_{x, \lambda}(z))-f_1(u(z))\big]{\rm d}z\nonumber\\ &\leq&\int_{\Sigma^v_{x, \lambda}}P(x, \lambda;\xi, z) f_1(u_{x, \lambda}(z)) \big[g_1(v_{x, \lambda}(z))-g _1(v(z))\big]{\rm d}z\nonumber\\ && +\int_{\Sigma^u_{x, \lambda}}P(x, \lambda;\xi, z) g_1(v(z)) \big[f_1(u_{x, \lambda}(z))-f_1(u(z))\big]{\rm d}z\nonumber\\ &\leq& \int_{\Sigma_{x, \lambda}^v}\frac{1}{|\xi-z|^{n-\alpha}}F_1(u_{x, \lambda}(z)) u_{x, \lambda}^{p_1}(z) G_1(v_{x, \lambda}(z)) \left[v^{q_1}_{x, \lambda}(z)-v^{q_1}(z)\right]{\rm d}z\nonumber\\ && +\int_{\Sigma_{x, \lambda}^u}\frac{1}{|\xi-z|^{n-\alpha}} G _1(v(z))v^{q_1}(z)F_1(u_{x, \lambda}(z)) \left[u^{p_1}_{x, \lambda}(z)-u^{p_1}(z)\right]{\rm d}z\nonumber\\ &\leq& \int_{\Sigma_{x, \lambda}^v}\frac{q_1}{|\xi-z|^{n-\alpha}}F_1(u_{x, \lambda}(z)) G_1(v_{x, \lambda}(z)) u_{x, \lambda}^{p_1}(z)v_{x, \lambda}^{q_1-1}(z) \left[v_{x, \lambda}(z)-v(z)\right]{\rm d}z\nonumber\\ && +\int_{\Sigma_{x, \lambda}^u}\frac{p_1}{|\xi-z|^{n-\alpha}}F_1(u_{x, \lambda}(z)) G _1(v(z)) u^{p_1-1}(z)v^{q_1}(z) \left[u_{x, \lambda}(z)-u(z)\right]{\rm d}z\nonumber\\ &=:&q_1I_1+p_1I_2. \end{eqnarray} $

(4.2)

令$\theta=\frac {2n}{n-\alpha}$, 则对$I_1$利用HLS不等式(2.1) 及广义的Hölder不等式推出

$ \begin{eqnarray}\label{monotony-3} &&\big\|I_1\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^u)}}\\ & \leq& C(n, \theta) \big\|F_1(u_{x, \lambda}) G_1(v_{x, \lambda}) u_{x, \lambda}^{p_1}v_{x, \lambda}^{q_1-1} \left(v_{x, \lambda}-v\right)\|_{{L^{\frac{n\theta}{n+\alpha\theta}}}{(\Sigma_{x, \lambda}^v)}}\nonumber\\ &\leq& C(n, \theta)\big\|F_1^{1/{p_1}}(u_{x, \lambda})u_{x, \lambda}\big\|^{p_1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^v)}} \big\|G_1^{1/{(q_1-1)}}(v_{x, \lambda})v_{x, \lambda}\big\|^{q_1-1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^v)}} \big\|v_{x, \lambda}-v\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^v)}}, \end{eqnarray} $

(4.3)

这里利用到了关系式

$ \frac{p_1}{\theta}+\frac {q_1-1}{\theta}+\frac {1}{\theta}=\frac{n+\alpha\theta}{n\theta}. $

类似(4.3) 式的推导, 有

$\begin{eqnarray}\label{monotony-4} &&\big\|I_2\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^u)}}\\ & \leq& C(n, \theta) \big\|F_1(u_{x, \lambda}) G_1(v) u_{x, \lambda}^{p_1-1}v^{q_1} \left(u_{x, \lambda}-u\right)\|_{{L^{\frac{n\theta}{n+\alpha\theta}}}{(\Sigma_{x, \lambda}^u)}}\nonumber\\ &\leq& C(n, \theta)\big\|F_1^{1/{(p_1-1)}}(u_{x, \lambda})u_{x, \lambda}\big\|^{p_1-1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^u)}} \big\|G_1^{1/{q_1}}(v)v\big\|^{q_1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^u)}} \big\|u_{x, \lambda}-u\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^u)}}. \end{eqnarray} $

(4.4)

根据(4.2)-(4.4) 式, 推出

$ \begin{eqnarray}\label{monotony-5} && \big\|u_{x, \lambda}-u\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^u)}} \\ &\leq& c\big\|F_1^{1/{(p_1-1)}}(u)u\big\|^{p_1-1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \big\|G_1^{1/{q_1}}(v)v\big\|^{q_1}_{{L^{\theta}}{(\Sigma_{x, \lambda})}} \big\|u_{x, \lambda}-u\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^u)}}\nonumber\\ && +c\big\|F_1^{1/{p_1}}(u)u\big\|^{p_1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \big\|G_1^{1/{(q_1-1)}}(v)v\big\|^{q_1-1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \big\|v_{x, \lambda}-v\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^v)}}, \end{eqnarray} $

(4.5)

其中$\Sigma_{x, \lambda}^C$是$\Sigma_{x, \lambda}$的补集.类似地, 可以证明

$ \begin{eqnarray}\label{monotony-6} && \big\|v_{x, \lambda}-v\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^v)}} \\ &\leq& c\big\|F_2^{1/{(p_2-1)}}(u)u\big\|^{p_2-1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \big\|G_2^{1/{q_2}}(v)v\big\|^{q_2}_{{L^{\theta}}{(\Sigma_{x, \lambda})}} \big\|u_{x, \lambda}-u\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^u)}}\nonumber\\ && +c\big\|F_2^{1/{p_2}}(u)u\big\|^{p_2}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \big\|G_2^{1/{(q_2-1)}}(v)v\big\|^{q_2-1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \big\|v_{x, \lambda}-v\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^v)}}. \end{eqnarray} $

(4.6)

结合(4.5) 和(4.6) 式, 推出

$ \begin{eqnarray}\label{monotony-7} &&\big\|u_{x, \lambda}-u\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^u)}}+\big\|v_{x, \lambda}-v\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^v)}}\nonumber\\ &\leq& c\Big(\big\|F_1^{1/{(p_1-1)}}(u)u\big\|^{p_1-1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \big\|G_1^{1/{q_1}}(v)v\big\|^{q_1}_{{L^{\theta}}{(\Sigma_{x, \lambda})}}\nonumber\\ &&+\big\|F_2^{1/{(p_2-1)}}(u)u\big\|^{p_2-1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \big\|G_2^{1/{q_2}}(v)v\big\|^{q_2}_{{L^{\theta}}{(\Sigma_{x, \lambda})}} \Big)\big\|{u_{x, \lambda}-u}\big\|_{L^\theta {(\Sigma_{x, \lambda}^u)}}\nonumber\\ &&+c\Big(F_1^{1/{p_1}}(u)u\big\|^{p_1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \big\|G_1^{1/{(q_1-1)}}(v)v\big\|^{q_1-1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}}\nonumber\\ &&+\big\|F_2^{1/{p_2}}(u)u\big\|^{p_2}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \big\|G_2^{1/{(q_2-1)}}(v)v\big\|^{q_2-1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \Big)\big\|v_{x, \lambda}-v\big\|_{L^\theta{(\Sigma_{x, \lambda}^v)}}. \end{eqnarray} $

(4.7)

根据假设(H), 选取足够小的$\lambda_0(x)$使得对$0<\lambda<\lambda_0$, 有

$ \begin{eqnarray*} &&c\Big(\big\|F_1^{1/{(p_1-1)}}(u)u\big\|^{p_1-1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \big\|G_1^{1/{q_1}}(v)v\big\|^{q_1}_{{L^{\theta}}{(\Sigma_{x, \lambda})}} \\ && +\big\|F_2^{1/{(p_2-1)}}(u)u\big\|^{p_2-1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \big\|G_2^{1/{q_2}}(v)v\big\|^{q_2}_{{L^{\theta}}{(\Sigma_{x, \lambda})}} \Big)\leq \frac{1}{2}, \nonumber\\ \\ &&c\Big(\|F_1^{1/{p_1}}(u)u\big\|^{p_1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \big\|G_1^{1/{(q_1-1)}}(v)v\big\|^{q_1-1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}}\\ &&+\big\|F_2^{1/{p_2}}(u)u\big\|^{p_2}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \big\|G_2^{1/{(q_2-1)}}(v)v\big\|^{q_2-1}_{{L^{\theta}}{(\Sigma_{x, \lambda}^C)}} \Big)\leq \frac{1}{2}. \end{eqnarray*} $

因此, 我们有

$ \begin{eqnarray*} \big\|u_{x, \lambda}-u\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^u)}}+\big\|v_{x, \lambda}-v\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^v)}} \leq \frac{1}{2} \Big(\big\|u_{x, \lambda}-u\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^u)}}+\big\|v_{x, \lambda}-v\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^v)}} \Big). \end{eqnarray*} $

这表明$\big\|u_{x, \lambda}-u\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^u)}}=$$\big\|v_{x, \lambda}-v\big\|_{{L^\theta}{(\Sigma_{x, \lambda}^v)}}=0$, 所以$\Sigma_{x, \lambda}^u$和$\Sigma_{x, \lambda}^v$的测度都为零, 从而根据$u$和$v$的连续性可知$\Sigma_{x, \lambda}^u$和$\Sigma_{x, \lambda}^v$都是空集.因此, (4.1) 式得证.

对$x\in\partial{\Bbb R}_+^n, \ \xi\in{\Bbb R}_+^n, $定义

$ \begin{eqnarray*} \bar{\lambda}(x)=\sup\{\mu>0:\ u_{x, \lambda}(\xi)\leq u(\xi), \ v_{x, \lambda}(\xi)\leq v(\xi), \ \forall 0<\lambda<\mu, \ \xi\in \Sigma_{x, \lambda}\}. \end{eqnarray*} $

引理4.2  若$f_i(u)=m_i u^{p_i}, \, g_i(v)=l_i v^{q_i} \, (i=1, 2), $则对任意的$x\in\partial{\Bbb R}^{n}_+$, 有$\bar{\lambda}(x)<\infty$, 并且

$ u_{x, \bar{\lambda}}(\xi)=u(\xi), \quad v_{x, \bar{\lambda}}(\xi)=v(\xi), \quad \xi\in\Sigma_{x, \bar{\lambda}}. $

若$f_i(u)=m_i u^{p_i}, \, g_i(v)=l_i v^{q_i} \, (i=1, 2), $即$F_i(u)=m_i, \, G_i(v)=l_i \, (i=1, 2), $这种情形的结果已由文献[9]给出, 此处省略证明.

下面将证明当$f_i(u)\neq m_i u^{p_i}$或$ g_i(v)\neq l_i v^{q_i} \, (i=1, 2)$时, 球面会一直向外移动, 不会停止.

引理4.3  若$F_i(u)\neq m_i $或$ G_i(v)\neq l_i \, (i=1, 2), $则$\bar{\lambda}(x)=\infty$, 其中$x\in\partial{\Bbb R}^{n}_+.$

证  本定理的证明与文献[11]中的引理$3.3$和文献[12]中的引理$3.4$的证明类似.利用反证法证明, 假设$F_i(u)\neq m_i $或$ G_i(v)\neq l_i \, (i=1, 2), $则存在$x_0\in\partial{\Bbb R}^{n}_+$使得$\bar{\lambda}(x_0)<\infty $.于是, 根据$\bar{\lambda}$的定义推出

$ \begin{eqnarray*} u_{x_0, \bar{\lambda}}(\xi)\le u(\xi), \quad v_{x_0, \bar{\lambda}}(\xi)\le v(\xi), \quad \quad \xi\in\Sigma_{x_0, \bar{\lambda}}. \end{eqnarray*} $

令$x=x_0, \lambda=\bar{\lambda}$, 根据(3.3) 和(3.4) 式推出

$ \begin{eqnarray*} u_{x_0, \bar{\lambda}}(\xi)<u(\xi), \quad v_{x_0, \bar{\xi}}(\eta)<v(\xi), \quad\quad \xi\in\Sigma_{x_0, \bar{\lambda}}. \end{eqnarray*} $

对固定的$R$和任意的$\delta>0$, 存在$c_1, c_2$使得

$ \begin{eqnarray*} u(\xi)-u_{x_0, \bar{\lambda}}(\xi)>c_1, \quad v(\xi)-v_{x_0, \bar{\lambda}}(\xi)>c_2, \quad \quad \xi\in\Sigma_{x_0, \bar{\lambda}+\delta}\cap B^+_R(x_0). \end{eqnarray*} $

因此, 可选取足够小的$\varepsilon<\delta$使得对$\lambda\in[\bar{\lambda}, \bar{\lambda}+\varepsilon)$, 有

$ \begin{eqnarray*} u(\xi)\ge u_{x_0, \lambda}(\xi), \quad v(\xi)\ge v_{x_0, \lambda}(\xi)\quad \mbox{对}~\xi\in\Sigma_{x_0, \bar{\lambda}+\delta}\cap B^+_R(x_0). \end{eqnarray*} $

上式表明$\Sigma_{x_0, \lambda}^u$和$\Sigma_{x_0, \lambda}^v$与$\Sigma_{x_0, \bar{\lambda}+\delta}\cap B^+_R(x_0) $没有交集.于是, $\Sigma_{x_0, \lambda}^u$和$\Sigma_{x_0, \lambda}^v$都包含在如下的集合

$ \Omega_{\lambda, R}=: \Big({\Bbb R}^n_+\setminus B^{+}_R(x_0)\Big)\cup \Big(\Sigma_{x_0, \lambda}\backslash\Sigma_{x_0, \bar{\lambda}+\delta}\Big). $

并且, 在关于球面$\{\xi: |\xi-x_0|=\lambda, ~\xi\in{\Bbb R}^n_+\}$的Kelvin变换下, 集合$\Omega_{\lambda, R}$的反射集合为

$ (\Omega_{\lambda, R})^*=B_{\varepsilon_1}^+(x_0)\cup(B_{\lambda}^+(x_0)\backslash B_{\lambda^2/(\bar \lambda +\delta)}^+(x_0)), $

其中, 当$R\to \infty$时, $\varepsilon_1=\lambda/R$任意小.显然, 对足够小的$\varepsilon_1, \delta$及足够大的$R$, 集合$(\Omega_{\lambda, R})^*$的测度任意小.

类似于(4.7) 式, 对$\lambda\in[\bar{\lambda}, \bar{\lambda}+\varepsilon), $可以推出

$ \begin{eqnarray*} &&\big\|u_{x_0, \lambda}-u\big\|_{{L^\theta}{(\Sigma_{x_0, \lambda}^u)}}+\big\|v_{x_0, \lambda}-v\big\|_{{L^\theta}{(\Sigma_{x_0, \lambda}^v)}}\nonumber\\ &\leq& c\big[\big\|F_1^{1/{(p_1-1)}}(u)u\big\|^{p_1-1}_{{L^{\theta}}{((\Omega_{\lambda, R})^*)}} \big\|G_1^{1/{q_1}}(v)v\big\|^{q_1}_{{L^{\theta}}{(\Omega_{\lambda, R})}}\nonumber\\ &&+\big\|F_2^{1/{(p_2-1)}}(u)u\big\|^{p_2-1}_{{L^{\theta}}{((\Omega_{\lambda, R})^*)}} \big\|G_2^{1/{q_2}}(v)v\big\|^{q_2}_{{L^{\theta}}{(\Omega_{\lambda, R})}} \big]\big\|{u_{x, \lambda}-u}\big\|_{L^\theta {(\Sigma_{x, \lambda}^u)}}\nonumber\\ &&+c\big[F_1^{1/{p_1}}(u)u\big\|^{p_1}_{{L^{\theta}}{((\Omega_{\lambda, R})^*)}} \big\|G_1^{1/{(q_1-1)}}(v)v\big\|^{q_1-1}_{{L^{\theta}}{((\Omega_{\lambda, R})^*)}}\nonumber\\ &&+\big\|F_2^{1/{p_2}}(u)u\big\|^{p_2}_{{L^{\theta}}{((\Omega_{\lambda, R})^*)}} \big\|G_2^{1/{(q_2-1)}}(v)v\big\|^{q_2-1}_{{L^{\theta}}{((\Omega_{\lambda, R})^*)}} \big]\big\|v_{x, \lambda}-v\big\|_{L^\theta{(\Sigma_{x, \lambda}^v)}}\nonumber\\ &=:& J\big\|{u_{x, \lambda}-u}\big\|_{L^\theta {(\Sigma_{x, \lambda}^u)}}+K\big\|v_{x, \lambda}-v\big\|_{L^\theta{(\Sigma_{x, \lambda}^v)}}. \end{eqnarray*} $

根据假设(H), 可以选取足够小的$\varepsilon_1, \, \delta$及足够大的$R$, 使得对$\lambda\in[\bar{\lambda}, \bar{\lambda}+\varepsilon), $有

$ J\leq \frac{1}{2}, ~~ ~~~~K\leq \frac{1}{2}. $

因此, 得到

$ \|u_{x_0, \lambda}-u\|_{L^\theta(\Sigma_{x_0, \lambda}^{u})}=\|v_{x_0, \lambda}-v\|_{L^\theta(\Sigma_{x_0, \lambda}^v)}=0. $

于是推出$\Sigma_{x_0, \lambda}^{u}$和$\Sigma_{x_0, \lambda}^v$都是零测集.从而, 有

$ u_{x_0, {\lambda}}(\xi)\le u(\xi), \quad v_{x_0, {\lambda}}(\xi)\le v(\xi), \quad \mbox{对}~\xi\in\Sigma_{x_0, {\lambda}}, $

其中$ \lambda\in[\bar{\lambda}, \bar{\lambda}+\varepsilon) $, 这与$\bar{\lambda}$的定义矛盾.所以, 引理4.3得证.

定理1.2的证明  情形1  若$F_i(u)=m_i, \, G_i(v)=l_i, \, (i=1, 2), $即, $f_i(u)=m_i u^{p_i}, \, g_i(v)=l_i v^{q_i}, \, p_i + q_i=\frac{n+\alpha}{n-\alpha}.$从引理4.2, 我们推出对任意的$x\in\partial{\Bbb R}^{n}_+$, 有$\bar{\lambda}(x)<\infty$, 且

$ u_{x, \bar{\lambda}}(\xi)=u(\xi), ~~~~v_{x, \bar{\lambda}}(\xi)=v(\xi), ~~~~~~~ \forall\, \xi\in {\Bbb R}^{n}_+. $

根据定理1.1和定理3.2, 推出

$ \begin{equation}\label{Liouville-1} u(\xi', 0)=\Big(\frac{c_1}{(|\xi'-\xi_0'|^2+d^2)}\Big)^{\frac{n-\alpha}2}, \end{equation} $

(4.8)

$ \begin{equation}\label{Liouville-2} v(\xi', 0)=\Big(\frac{c_2}{(|\xi'-\xi_0'|^2+d^2)}\Big)^{\frac{n-\alpha}2}, \end{equation} $

(4.9)

其中$c_1, c_2, d>0$, $\xi_0=(\xi_0', d)\in{\Bbb R}_+^n.$

然而, 从方程组(1.1) 的第一个方程, 推出

$ \begin{eqnarray*} u(\xi', 0)&=&\lim\limits_{\xi_n\to0}u(\xi', \xi_n)\\ & =&\lim\limits_{\xi_n\to 0}\int_{{\Bbb R}^n_+}\Big(\frac1{(|\xi'-y'|^2+|\xi_n-y_n|^2)^{\frac{n-\alpha}{2}}} - \frac1{(|\xi'-y'|^2+|\xi_n+y_n|^2)^{\frac{n-\alpha}{2}}}\Big)\\ & & \cdot m_1l_1u^{p_1}(y)v^{q_1}(y){\rm d}y\\ &=&0, \end{eqnarray*} $

这与(4.8) 式矛盾.于是, 推出$u\equiv0 $.类似地, 推出$v\equiv0 $.

情形2  若$F(t)\neq m_1$或$G(s)\neq m_2, $从引理4.3, 推出对任意的$x \in\partial{\Bbb R}^{n}_+$, 有$\bar{\lambda}(x)=\infty $.于是

$ \begin{eqnarray*} u_{x, \bar{\lambda}}(\xi)\leq u(\xi), \quad v_{x, \bar{\lambda}}(\xi)\leq v(\xi), \quad \forall \xi\in\Sigma_{x, \bar{\lambda}}, ~ x\in\partial{\Bbb R}_+^n. \end{eqnarray*} $

利用引理3.3, 推出$u(\xi)=u(\xi_n)$和$v(\xi)=v(\xi_n)$, 其中$\xi=(\xi', \xi_n)\in {\Bbb R}^{n-1}\times (0, \infty).$

类似于文献[24], 利用Silvestre^[25]的极值原理, 假设$u>0, \, v>0$.于是, 由(1.1) 式, 推出$F_i, \, G_i >0(i=1, 2).$

下面对$G(\xi, y)=:\frac{1}{|\xi-y|^{n-\alpha}}-\frac{1}{|\bar{\xi}-y|^{n-\alpha}} $进行估计.通过直接计算, 推出

$ \begin{eqnarray*} G(\xi, y)&=&\frac{1}{|\xi-y|^{n-\alpha}}\bigg[1- \Big(\frac{|\xi-y|}{|\bar{\xi}-y|}\Big)^{n-\alpha}\bigg]\\ & =&\frac{1}{|\xi-y|^{n-\alpha}}\bigg[1- \Big(\frac{|\xi'-y'|^2+|\xi_n-y_n|^2}{|\xi'-y'|^2+|\xi_n+y_n|^2} \Big)^{(n-\alpha)/2}\bigg]\\ & =&\frac{1}{|\xi-y|^{n-\alpha}}\bigg[1- \Big(1-\frac{4 \xi_n y_n}{|\bar{\xi}-y|^2}\Big)^{(n-\alpha)/2}\bigg], \end{eqnarray*} $

其中$\xi=(\xi', \xi_n), y=(y', y_n)\in {\Bbb R}^{n-1}\times (0, \infty).$

为简单起见, 记$s=|\bar{\xi}-y|^2, \, t=4 \xi_n y_n.$令$s\rightarrow \infty, $即$\frac t s \rightarrow 0, $则有

$ \begin{eqnarray}\label{Liouville-3} G(\xi, y)&=&\frac{1}{|\xi-y|^{n-\alpha}} \bigg[1-\Big(1-\frac{t}{s}\Big)^{(n-\alpha)/2}\bigg]\nonumber\\ & =&\frac{1}{|\xi-y|^{n-\alpha}}\cdot\frac{t}{s}\cdot K^{(n-\alpha)/2-1}(s, t)\nonumber\\ &\sim&\frac{1}{|\xi-y|^{n-\alpha}}\cdot\frac{t}{s}, \end{eqnarray} $

(4.10)

这里用到了平均值定理, 且$1-\frac{t}{s}\leq K(s, t)\leq 1.$

记$r=|\xi'-y'|$及$a=\xi_n+y_n.$若$u(\xi)=u(\xi_n)$和$v(\xi)=v(\xi_n)$是方程组(1.1) 的一组解, 则对取定的$\xi\in {\Bbb R}_+^{n}$及足够大的$R$, 由(4.10) 式推出

$ \begin{eqnarray}\label{Liouville-4} \infty>u(\xi_n)&=& \int_0 ^\infty {F_1(u(y_n))G_1(v(y_n))u^{p_1}(y_n)v^{q_1}(y_n)}\int_{{\Bbb R}^{n-1}}G(\xi, y){\rm d}y' {\rm d}y_n\nonumber\\ &\geq&C\int_R ^\infty {u^{p_1}(y_n)v^{q_1}(y_n)y_n}\int_{{\Bbb R}^{n-1}\setminus B_R^+}\frac 1 {|\bar{\xi}-y|^{n-\alpha+2}}{\rm d}y' {\rm d}y_n\nonumber\\ &\geq& C\int_R ^\infty {u^{p_1}(y_n)v^{q_1}(y_n)y_n}\int_R ^\infty\frac {r^{n-2}} {(r^2+a^2)^{(n-\alpha+2)/2}}{\rm d}r {\rm d}y_n\nonumber\\ &\geq& C\int_R ^\infty {u^{p_1}(y_n)v^{q_1}(y_n)\frac{y_n}{|\xi_n+y_n|^{3-\alpha}}}\int_{R/a} ^\infty\frac {t^{n-2}} {(t^2+1)^{(n-\alpha+2)/2}}{\rm d}t {\rm d}y_n\nonumber\\ &\geq& C\int_R ^\infty {u^{p_1}(y_n)v^{q_1}(y_n)y_n^{\alpha-2}} {\rm d}y_n. \end{eqnarray} $

(4.11)

这表明当$i\rightarrow \infty$时, 存在序列$\{y_n^i\}\rightarrow\infty$使得

$ \begin{equation}\label{Liouville-5} u^{p_1}(y_n^i)v^{q_1}(y_n^i)(y_n^i)^{\alpha}\rightarrow 0. \end{equation} $

(4.12)

类似于(4.11) 式, 对任意的$\xi=(0, \xi_n)\in {\Bbb R}_+^{n}, $可以推出

$ \begin{equation}\label{Liouville-6} \infty>u(\xi_n)\geq C_0 \int_0 ^\infty {u^{p_1}(y_n)v^{q_1}(y_n)\frac {\xi_n y_n} {|\xi_n+y_n|^{3-\alpha}}} {\rm d}y_n. \end{equation} $

(4.13)

令$\xi_n=2R$足够大.由(4.13) 式推出

$ \begin{eqnarray}\label{Liouville-7} \infty>u(\xi_n)&\geq& C_0\int_0 ^1 {u^{p_1}(y_n)v^{q_1}(y_n)\frac {\xi_n y_n} {|\xi_n+y_n|^{3-\alpha}}} {\rm d}y_n\nonumber\\ &\geq&\frac{C_0}{(2R)^{3-\alpha}}(2R)\int_0 ^1 {u^{p_1}(y_n)v^{q_1}(y_n) y_n} {\rm d}y_n\nonumber\\ &\geq& C_1 (2R)^{\alpha-2} =C_1 \xi_n^{\alpha-2}. \end{eqnarray} $

(4.14)

类似地, 我们有

$ \begin{equation}\label{Liouville-8} \infty>v(\xi_n)\geq C_2 \xi_n^{\alpha-2}. \end{equation} $

(4.15)

当$\alpha\geq 2$时, (4.14) 及(4.15) 式与(4.12) 式矛盾.于是, 下面只需证明$\alpha\in (0, 2)$的情形.

联合(4.13), (4.14) 及(4.15) 式, 对足够大的$\xi_n=2R$, 有

$ \begin{eqnarray*} u(\xi_n)&\geq& C_0\int_{R/2} ^R (C_1 y_n)^{p_1(\alpha-2)}(C_2 y_n)^{q_1(\alpha-2)}\frac {\xi_n y_n} {|\xi_n+y_n|^{3-\alpha}} {\rm d}y_n\\ &\geq& C(\tau, \alpha)R^{(p_1+q_1)(\alpha-2)+\alpha} =C(\tau, \alpha)\xi_n^{\tau(\alpha-2)+\alpha}, \end{eqnarray*} $

其中$\tau=p_1+q_1=\frac{n+\alpha}{n-\alpha}.$继续迭代$m$次, 对$\xi_n=2R, $推出

$ \begin{equation}\label{Liouville-9} u(\xi_n)\geq C(m, \tau, \alpha)\xi_n^{\tau^m(\alpha-2)+\frac{\tau^m-1}{\tau-1}\alpha}. \end{equation} $

(4.16)

类似地, 有

$ \begin{equation}\label{Liouville-10} v(\xi_n)\geq C(m, \tau, \alpha)\xi_n^{\tau^m(\alpha-2)+\frac{\tau^m-1}{\tau-1}\alpha}. \end{equation} $

(4.17)

从(4.16) 和(4.17) 式, 对足够大的$\xi_n$, 推出

$ \begin{equation}\label{Liouville-11} u^{p_1}(\xi_n)v^{q_1}(\xi_n){\xi_n}^{\alpha}\geq C(m, \tau, \alpha)\xi_n^{P(\tau)}> C(m, \tau, \alpha)>0, \end{equation} $

(4.18)

其中$P(\tau)=:\big[\tau^m(\alpha-2)+\frac{\tau^m-1}{\tau-1}\big]\tau+\alpha>0.$事实上, 记

$ f(\tau)=:(\tau-1)P(\tau)=\tau^{m+2}(\alpha-2)+2\tau^{m+1}-\alpha. $

则有

$ \begin{eqnarray*} f'(\tau)&=&\tau^{m}\big[\tau (m+2)(\alpha-2)+2(m+1)\big]\\ &=&\frac{\tau^{m}}{n-\alpha}\big[m(n\alpha+\alpha^2-4\alpha)-(2n+6\alpha-2n\alpha-2\alpha^2)\big]>0, \end{eqnarray*} $

其中, 当$n\geq 4, \, \alpha\in (0, 2)$时, 选取$m\geq \big[\frac{8-2\alpha-2\alpha^2}{\alpha^2} \big]+1 $, 且当$n= 4, \, \alpha\in (0, 1)\cup (1, 2)$时, $m\geq \big[\frac{6-2\alpha^2}{\alpha^2-\alpha} \big]+1 .$这里, $ \big[a \big] $表示$a$的整数.从而, $f(\tau)>f(1)=0, $即$P(\tau)>0.$

显然, (4.18) 与(4.12) 式矛盾.于是推出方程组(1.1) 没有正解.