为了研究Markov过程的稳定性, 引入了常返性(recurrence)和非常返性(transience)的概念, Meyn和Tweedie给出了离散时间Markov链的常返性和非常返性的定义和判定^[2]. 文献[5]提到过连续时间Markov过程的常返性, 但是目前对于一般状态空间连续时间Markov过程的常返性还没有系统的研究成果. 本文研究的一般状态空间连续时间Markov过程都是跳过程.

下面先简单介绍一下离散时间Markov链的常返性和非常返性的一些重要研究成果.

定义 1.1^[2]  集合$ A $称为常返的, 若

$ E_{x}[\eta_{A}] = \infty, \forall x\in A, $

其中$ \eta_{A}: = \sum\limits^{\infty}_{n = 1}I_{\{\Phi_{n}\in A\}} $称为占有时, 它表示从时刻$ 0 $开始访问集合$ A $的次数.

注 1.1

$ E_{x}[\eta_{A}] = U(x, A) = \sum\limits^{\infty}_{n = 1}P^{n}(x, A). $

称Markov链$ \Phi $是常返的, 若$ {\cal B^{+}}(X): = \{A\in {\cal B}(X):\Psi(A)>0\} $里每个集合都是常返集.

注 1.2  $ \Psi $是最大不可约概率测度. 由定义知常返Markov链是$ \Psi $不可约的.

定义 1.2  $ ^{[2]} ${\quad} 集合$ A $称为一致非常返的, 如果对所有$ x\in A $, 都存在$ M<\infty $, 使得

$ E_{x}[\eta_{A}] = U(x, A)\leq M. $

集合$ A $称为非常返集, 若$ A\in {\cal B}(X) $能被若干一致非常返集可数覆盖.

称Markov链$ \Phi $是非常返的, 如果它是$ \Psi $不可约且$ X $是非常返的.

定义 1.3  $ ^{[2]} ${\quad} Markov链$ \Phi $是常返的, 如果它是$ \Psi $不可约的且

$ U(x, A): = E_{x}[\eta_{A}] = \sum\limits_{n = 1}^{\infty}P^{n}(x, A) = \infty, \; \; x\in X, \; A\in {\cal B^{+}}(X). $

Markov链$ \Phi $是非常返的, 如果它是$ \Psi $不可约且$ X $是非常返的.

设$ x\in X, C\in{\cal B}(X) $, 记

$ \tau_{C}: = \min\{n\geq 1:\Phi_{n}\in C\}, $

$ L(x, C): = P_{x}(\tau_{C}<\infty), $

称$ \tau_{C} $为首次返回集合$ C $的时间. $ L(x, C) $表示从状态$ x $出发, 有限时间内返回到集合$ C $的概率.

定义 1.4  $ ^{[2]} ${\quad} 集合$ C\in {\cal B}(X) $称为Markov链$ \Phi $的细集(petite set), 若存在$ {{\Bbb R}} _{+} $上的分布$ a = \{a(n):n\in N_{+}\} $和$ {\cal B}(X) $上的非平凡测度$ \nu_{a} $, 使得对$ \forall x\in C, B\in{\cal B}(X) $,

$ K_{a}(x, B)\geq \nu_{a}(B), $

这里$ \nu_{a} $是$ {\cal B}(X) $上的一非平凡测度.其中概率转移核

$ K_{a}(x, B): = \sum\limits^{n}P^{n}(x, B)a(n), \forall x\in X, B\in{\cal B}(X). $

称为$ \Phi $的一个抽样链(sampled chain).

定理 1.1^[2]  设$ \Phi $是$ \Psi $不可约Markov链, 则

$ {\rm (i)}\; \Phi $是常返, 如果存在某细集$ C\in {\cal B}(X) $使得对所有的$ x\in C, L(x, C)\equiv 1 $.

$ {\rm (ii)}\; \Phi $是非常返, 当且仅当存在两个集合$ D, C\in {\cal B}^{+}(X) $使得对所有的$ x\in D, L(x, C)< 1 $.

$ {\rm (iii)}\; \Phi $是常返或非常返.

(i) 和(ii)的证明见文献[2]的定理8.3.6, (iii)的证明见文献[2]的定理8.2.5.

定理 1.2^[2]  设$ \Phi $是$ \Psi $不可约Markov链且非周期的, 则

$ {\rm (i)}\; $Markov链$ \Phi $是非常返的$ \iff $其$ m $骨架链$ \Phi^{m} $是非常返的;

$ {\rm (ii)}\; $Markov链$ \Phi $是常返的$ \iff $其$ m $骨架链$ \Phi^{m} $是常返的.

证明参见文献$ [2] $的定理$ 8.2.6 $.

定理 1.3^[2]  设$ D^{c} $是吸收集且$ \forall x\in D, L(x, D^{c})>0 $, 则$ D $是非常返集.

证明参见文献$ [2] $的定理$ 8.3.3 $.

定理 1.4^[2]  设Markov链$ \Phi $是$ \Psi $不可约的, 则$ \Phi $是常返的或非常返的.

证明参见文献$ [2] $的定理$ 8.3.4 $.

设$ \{X_t, t\in {{\Bbb R}} _{+}\} $为一个连续时间Markov过程, 有时直接简记为$ X_t $.状态空间$ X $为Polishi空间(即完备可分的度量空间), $ {\cal B}(X) $是由$ X $中可数个子集生成的$ \sigma $代数, 用$ \{P(t, x, A), t\in {{\Bbb R}} _{+}, A\in {\cal B}(X)\} $来表示Markov过程的转移概率函数, 即

$ P(t, x, A) = P_{x}[X_{t}\in A] = E_{x}[I_{\{X_{t}\in A\}}]. $

有时也用$ P(t) $来表示连续时间Markov过程. 设$ x\in X, A\in{\cal B}(X) $, 记

$ \eta_{A}: = \int_{0}^{\infty}I_{\{X_{t}\in A\}}{\rm d}t, $

$ W(x, A): = E_{x}[\eta_{A}] = \int_{0}^{\infty}P(t, x, A){\rm d}t, $

$ W{_{A}}(x, A): = \int_{0}^{\infty}{_{A}}P(t, x, A){\rm d}t, $

$ \tau_{A}: = \min\{t>0:X_{t}\in A\}, $

$ L(x, A): = P_{x}(\tau_{A}<\infty)>0, $

易得

$ L(x, A) = \int_{0}^{\infty}{_{A}}P(t, x, A){\rm d}t = W{_{A}}(x, A). $

定义 2.1  设$ A\in{\cal B}(X) $, 称$ \{X_t, t\in {{\Bbb R}} _{+}\} $是$ \varphi $不可约的, 若存在$ {\cal B}(X) $上的测度$ \varphi $使得, 当$ \varphi(A)>0 $时, 对所有$ x\in X $, 都有$ L(x, A)>0 $.

引理 2.1^[4]  下列条件是等价的:

$ (1) $连续时间Markov过程$ \{X_t, t\in {{\Bbb R}} _{+}\} $是$ \varphi $不可约的;

$ (2) $当$ \varphi(A)>0 $时, $ x\in X $, 存在(或任意的)$ t>0, $有

$ P(t, x, A)>0; $

$ (3) $当$ \varphi(A)>0 $时, $ x\in X $, 有

$ E_{x}[\eta_{A}] = E_{x} \bigg[\int_{0}^{\infty}I_{\{X_{t}\in A\}}{\rm d}t\bigg] = \int_{0}^{\infty}P(t, x, A){\rm d}t>0; $

$ (4) $当$ \varphi(A)>0 $时, $ x\in X $, 有

$ \int_{0}^{\infty}P(t, x, A)e^{-t}{\rm d}t>0. $

参见文献$ [4] $的引理$ 2.2.5 $.

引理 2.2^[4]  设连续时间Markov过程$ \{X_t, t\in {{\Bbb R}} _{+}\} $是$ \varphi $不可约的, 令

$ \Psi(A) = \int_{E}\varphi({\rm d}x)\int_{0}^{\infty}P(t, x, A)e^{-t}{\rm d}t, $

(1) Markov过程$ \{X_t, t\in {{\Bbb R}} _{+}\} $是$ \Psi $不可约的;

(2) 对任意其它测度$ \upsilon $, Markov过程$ \{X_t, t\in {{\Bbb R}} _{+}\} $是$ \upsilon $不可约的$ \iff $$ \upsilon\ll\Psi; $

(3) 当$ \Psi(A) = 0 $时, $ \Psi\{x\in X:P_{x}(\tau_{A}<\infty)>0\} = 0 $.

注 2.1  不妨假设$ \varphi $是概率测度, 则由上面的定义知$ \Psi $也是概率测度.满足引理$ 1.2 $中条件$ (1)(2) $的概率测度$ \Psi $称为最大概率测度. 以后总是假设$ \Psi $是最大不可约概率测度.

证明参见文献[4]的定理$ 2.2.6 $.

定义 3.1  设$ A\in{\cal B}(X) $, 若存在正常数$ M<\infty $, 使$ \forall x\in A $都有$ W(x, A)\leq M, $则称集合$ A $为一致非常返集.

如果集合$ B\in{\cal B}(X) $可被可数个一致非常返集覆盖, 则称$ B $是非常返集.

定义 3.2 集合$ A $称为常返的, 若$ E_{x}[\eta_{A}] = \infty, \forall x\in A. $

定义 3.3  称连续时间Markov过程$ \{P(t), t\in {{\Bbb R}} _{+}\} $是Harris常返的, 若存在某个$ \sigma $有限测度$ \nu $, 当$ \nu(B)>0 $时, 有$ P_{x}(\eta_{B} = \infty ) = 1, \forall x\in X. $

显然, 常返性几乎等价于Harris常返.

定理 3.1  设集合$ A $为一致非常返集, 且$ \forall x\in A, W(x, A)\leq M, $则

$ \forall x\in X, W(x, A)\leq 1+M, $

证记

$ _{A}P(t, x, B): = P_{x}\{X_{t}\in B, \tau_{A}\geq t\}. $

由于

$ \begin{eqnarray*} P(t, x, B)& = &P_{x}\{X_{t}\in B\}\\ & = &P_{x}\{X_{t}\in B, \tau_{A}\geq t\}+P_{x}\{X_{t}\in B, \tau_{A}< t\}\\ & = &{_{A}}P(t, x, B)+P_{x}\{X_{t}\in B, \tau_{A}< t\}\\ & = &{_{A}}P(t, x, B)+\int_{0}^{t}\int_{A}{_{A}}P(s, x, {\rm d}y) P(t-s, y, B){\rm d}s. \end{eqnarray*} $

所以有

$ P(t, x, A) = {_{A}}P(t, x, A)+\int_{0}^{t}\int_{A}{_{A}}P(s, x, {\rm d}y) P(t-s, y, A){\rm d}s. $

所以

$ \int_{0}^{\infty} P(t, x, A){\rm d}t = \int_{0}^{\infty}{_{A}}P(t, x, A){\rm d}t+\int_{0}^{\infty}\int_{0}^{t}\int_{A}{_{A}}P(s, x, {\rm d}y) P(t-s, y, A){\rm d}s{\rm d}t. $

显然

$ \int_{0}^{\infty}{_{A}}P(t, x, A){\rm d}t = P_{x}\{\tau_{A}<\infty\}\leq 1. $

而

$ \begin{eqnarray*} \int_{0}^{\infty}\int_{0}^{t}\int_{A}{_{A}}P(s, x, {\rm d}y)P(t-s, y, A){\rm d}s{\rm d}t & = &\bigg[\int_{0}^{\infty}\int_{A}{_{A}}P(s, x, {\rm d}y){\rm d}s \bigg]\bigg[\int_{0}^{\infty}P(t, y, A){\rm d}t\bigg]\\ & = &\bigg[\int_{0}^{\infty}{_{A}}P(s, x, A){\rm d}s\bigg]\bigg [\int_{0}^{\infty}P(t, y, A){\rm d}t\bigg]\\ & = &P_{x}\{\tau_{A}<\infty\}\bigg[\int_{0}^{\infty}P(t, y, A){\rm d}t\bigg]\\ &\leq& \int_{0}^{\infty}P(t, y, A){\rm d}t \; \; (y\in A)\\ &\leq &M. \end{eqnarray*} $

故$ \forall x\in X, W(x, A)\leq 1+M $.

定理 3.2  若$ A_1, A_2, \cdot, \cdot, \cdot, A_n $是一致非常返集, 则$ \bigcup\limits_{i = 1}^{n}A_i $也是一致非常返集.

证当$ x_i\in A $时, 存在正数$ M_i<\infty, W(x, A_i)\leq M_i, (i = 1, 2, \cdot, \cdot, \cdot, n) $. 令

$ M = n+\sum\limits_{i = 1}^{n}M_i, $

对$ \forall x\in \bigcup\limits_{i = 1}^{n}A_i , $有

$ \begin{eqnarray*} W(x, \bigcup\limits_{i = 1}^{n}A_i)& = &\int_{0}^{\infty}P(t, x, \bigcup\limits_{i = 1}^{n}A_i){\rm d}t \leq\int_{0}^{\infty}\sum\limits_{i = 1}^{n}P(t, x, A_i){\rm d}t\\ & = &\sum\limits_{i = 1}^{n}\int_{0}^{\infty}P(t, x, A_i){\rm d}t = \sum\limits_{i = 1}^{n}W(x, A_i)\\ &\leq&n+\sum\limits_{i = 1}^{n}M_i = M. \end{eqnarray*} $

故$ \bigcup\limits_{i = 1}^{n}A_i $是一致非常返集.

设$ a(\cdot) $为$ {{\Bbb R}} _{+} $上的概率分布, 记

$ K_{a}(x, A): = \int^{\infty}_{0}P(t, x, A)a({\rm d}t), \; \; \forall x\in X, \; A\in{\cal B}(X). $

引理 3.1  $ ^{[4]} ${\quad} $ {\rm (i)} $若$ a $和$ b $是$ {{\Bbb R}} _{+} $上的分布, 则具有转移法则$ K_{a} $和$ K_{b} $的抽样链满足广义的C-K方程

(3.1) $ \begin{equation} K_{a\star b}(x, A) = \int K_{a}(x, {\rm d}y)K_{b}(y, A), \end{equation} $

这里$ a\star b $表示$ a $和$ b $的卷积.

$ {\rm (ii)} $若$ a $是$ {{\Bbb R}} _{+} $上的分布, 则具有转移法则$ K_{a} $的抽样链满足关系:

(3.2) $ \begin{equation} W(x, A)\geq\int W(x, {\rm d}y)K_{a}(y, A), \end{equation} $

证明参见文献$ [4] $的命题$ 2.3.3 $.

定理 3.3  设集合$ A, B\in{\cal B}(X) $, 若$ A $是一致非常返集, 且存在$ {{\Bbb R}} _{+} $上的概率分布$ a(\cdot) $使

$ \inf\limits_{x\in B}K_{a}(x, A) = \delta>0, $

则$ B $也是一致非常返集.

证

$ \begin{eqnarray*} W(x, A) \geq \int_{X}W(x, {\rm d}y)K_{a}(y, A) \geq\int_{B}W(x, {\rm d}y)K_{a}(y, A) \geq W(x, B)\inf\limits_{y\in B}K_{a}(y, A). \end{eqnarray*} $

由于$ A $是一致非常返集, 所以存在正数$ M_{1}<\infty $使得

$ \forall x\in X, W(x, A)\leq 1+M_{1}, $

记$ M = \frac{1+M_{1}}{\delta} $, 则对

$ \forall x\in X, W(x, B)\leq M, $

故对$ \forall x\in B, W(x, B)\leq M. $即$ B $也是一致非常返集.

定义 3.4  设集合$ A\in{\cal B}(X) $, 若对$ \forall t\in {{\Bbb R}} _{+}, x\in A, $都有$ P(t, x, A^{c)} = 0 $, 则称$ A $是吸收集.

定理 3.4  若存在自然数$ m $和$ 0<\varepsilon<1 $使

$ P_{x}(\eta_{A}\geq m)\leq \varepsilon, x\in A, $

则$ A $是一致非常返集.

证记$ \tau_{A^{(k)}} $表示第$ k $次返回$ A $的时刻. 则有

$ \int_{A}P_{x}(X_{\tau_{A}}\in {\rm d}y) = P_{x}(X_{\tau_{A}}\in A) = P_{x}(\tau_{A}<\infty). $

$ \int_{A}P_{x}(X_{\tau_{A}^{(km)}}\in {\rm d}y) = P_{x}(X_{\tau_{A}^{(km)}}\in A) = P_{x}(\tau_{A^{(km)}}<\infty) = P_{x}(\eta_{A}\geq km). $

归纳地, 有

$ \begin{eqnarray*} P_{x}(\eta_{A}\geq m(k+1)) & = &\int_{A}P_{x}(X_{\tau_{A}^{(km)}}\in {\rm d}y)P_{y}(\eta_{A}\geq m)\\ & = &P_{x}(\eta_{A}\geq km)P_{y}(\eta_{A}\geq m), \; \; \; y\in A\\ &\leq& \varepsilon^{k+1} . \end{eqnarray*} $

对$ \forall x\in A, $有

$ \begin{eqnarray*} W(x, A)& = &\int_{0}^{\infty}P(t, x, A){\rm d}t = \int_{0}^{\infty}P_{x}(\eta_{A}\geq t){\rm d}t\\ & = &\int_{0}^{m}\sum\limits_{k = 1}^{\infty} P_{x}(\eta_{A}\geq r+km){\rm d}r\\ &\leq&\int_{0}^{m}\sum\limits_{k = 1}^{\infty} P_{x}(\eta_{A}\geq km){\rm d}r = \int_{0}^{m}{\rm d}r\bigg[\sum\limits_{k = 1}^{\infty} P_{x}(\eta_{A}\geq km)\bigg]\\ &\leq&m\cdot \sum\limits_{k = 1}^{\infty}\varepsilon^{k+1}\\ & = &\frac{m\varepsilon}{1-\varepsilon}. \end{eqnarray*} $

故$ A $是一致非常返集.

定义 4.1  连续时间Markov过程$ \{P(t), t\in {{\Bbb R}} _{+}\} $是常返的, 如果它是$ \Psi $不可约的且

$ W(x, A) = \int_{0}^{\infty}P(t, x, A){\rm d}t = \infty, A\in {\cal B^{+}}(X), x\in X. $

连续时间Markov过程$ \{P(t), t\in {{\Bbb R}} _{+}\} $是非常返的(或瞬时的), 如果它是$ \Psi $不可约且$ X $是非常返的.

定理 4.1  设连续时间Markov过程$ P(t) $是$ \Psi $不可约的, 则连续时间Markov过程$ P(t) $是常返的或非常返的.

设连续时间Markov过程$ P(t) $不是常返的, 即存在某集合$ A\in {\cal B^{+}}(X), x\in X $使得

$ W(x, A) = \int_{0}^{\infty}P(t, x, A){\rm d}t<\infty. $

为了证明此定理, 我们分下面四步来证明:

(a) 记$ A_{\star}: = \{y\in X:W(y, A) = \infty\}, $则$ \Psi(A_{\star}) = 0 $.

(b) 记$ A_{r}: = \{y\in A:W(y, A)\leq r\}, $则$ \forall y\in X, W(y, A_{r})\leq r+1 $.

(c) 记

$ \overline{A}_{r}(M): = \bigg\{y\in X:\int_{0}^{M}(t, y, A_{r}){\rm d}t>M^{-1}\bigg\}, $

则$ \overline{A}_{r}(M) $是一致非常返集.

(d) $ \bigcup\limits_{M}\overline{A}_{r}(M) = X. $

证  (a) 记$ A_{\star}: = \{y\in X:W(y, A) = \infty\}, $用反证法证明$ \Psi(A_{\star}) = 0 $. 假设$ \Psi(A_{\star})>0 $, 由引理$ 2.1 $不可约知, $ \exists t\geq 0 $, 使得$ P(t, x, A_{\star})>0 $应用C-K方程

$ \begin{eqnarray*} P(t+s, &x, &A) = \int_{X}P(t, x, {\rm d}y)P(s, y, A), \\ W(x, A)&\geq&\int_{0}^{\infty}P(t+s, x, A){\rm d}s\\ & = &\int_{0}^{\infty}\int_{X}P(t, x, {\rm d}y)P(s, y, A){\rm d}s\\ & = &\int_{X}P(t, x, {\rm d}y)\int_{0}^{\infty}P(s, y, A){\rm d}s\\ & = &\int_{X}P(t, x, {\rm d}y)W(y, A)\\ &\geq &\int_{A_{\star}}P(t, x, {\rm d}y)W(y, A)\\ & = &\infty. \end{eqnarray*} $

与假设$ W(x, A)<\infty $相矛盾, 故$ \Psi(A_{\star}) = 0 $.

(b) 记$ A_{r}: = \{y\in A:W(y, A)\leq r\} $由于

$ \Psi(A)>0, A_{r}\nearrow A\cap A_{\star}^{c}. $

由$ \Psi(A_{\star}) = 0 $得$ \Psi(A\cap A_{\star}) = 0 $. 又由于

$ \Psi(A) = \Psi(A\cap A_{\star}^{c})+\Psi(A\cap A_{\star}), $

所以

$ \Psi(A\cap A_{\star}^{c}) = \Psi(A)>0. $

故必存在某个$ r $, 使得$ \Psi(A_{r})>0. $由定理3.1, $ \forall y\in X, W(y, A)\leq r+1. $由$ A_{r}\subseteq A $可推得$ W(x, A_{r})\leq W(x, A) $. 从而有$ \forall y\in X, W(y, A_{r})\leq r+1 $.

(c) 记

$ \overline{A}_{r}(M): = \{y\in X:\int_{0}^{M}P(t, y, A_{r}){\rm d}t>M^{-1}\}, $

对$ \forall x\in X $有

$ 1+r\geq W(x, A_{r}) = \int_{0}^{\infty}P(t, x, A_{r}){\rm d}t\geq \int_{m}^{\infty}P(t, x, A_{r}){\rm d}t. $

两端让$ m $从$ 1 $至$ M $积分得

$ \begin{eqnarray*} \int_{0}^{M}(1+r){\rm d}m&\geq &\int_{0}^{M}\int_{m}^{\infty}P(t, x, A_{r}){\rm d}t{\rm d}m\\ & = &\int_{0}^{M}\int_{0}^{\infty}P(t+m, x, A_{r}){\rm d}t{\rm d}m\\ & = &\int_{0}^{\infty}\int_{0}^{M}P(t+m, x, A_{r}){\rm d}t{\rm d}m\\ & = &\int_{0}^{\infty}\int_{0}^{M}\int_{X}P(t, x, {\rm d}y)P(m, y, A_{r}){\rm d}t{\rm d}m\\ &\geq &\int_{0}^{\infty}\int_{0}^{M}\int_{\overline{A}_{r}(M)}P(t, x, {\rm d}y)P(m, y, A_{r}){\rm d}t{\rm d}m\\ & = &\int_{0}^{\infty}\int_{\overline{A}_{r}(M)}P(t, x, {\rm d}y)[\int_{0}^{M}P(m, y, A_{r}){\rm d}m]{\rm d}t\\ &\geq & M^{-1}\int_{0}^{\infty}\int_{\overline{A}_{r}(M)}P(t, x, {\rm d}y){\rm d}t\\ & = &M^{-1}\int_{0}^{\infty}P(t, x, \overline{A}_{r}(M)){\rm d}t\\ & = &M^{-1}W(x, \overline{A}_{r}(M)), \end{eqnarray*} $

所以

$ W(x, \overline{A}_{r}(M))\leq M^{2}(1+r). $

从而$ \overline{A}_{r}(M) $是一致非常返集.

(d)

$ \forall x\in X, 0<W(x, A_{r}) = \lim\limits_{M\rightarrow \infty} \int_{0}^{M}P(t, y, A_{r}){\rm d}t. $

记

$ \int_{0}^{M}P(t, y, A_{r}){\rm d}t = \delta>\frac{1}{N}>0. $

取$ M' = \max\{M, N\} $, 则有

$ \int_{0}^{M'}P(t, y, A_{r}){\rm d}t\geq\int_{0}^{M}P(t, y, A_{r}){\rm d}t = \delta>\frac{1}{M'}. $

所以存在$ M' $使得$ x\in \overline{A}_{r}(M'). $从而

$ \forall x\in X, \; \; x\in \bigcup\limits_{M}\overline{A}_{r}(M). $

故

$ X\subseteq \bigcup\limits_{M}\overline{A}_{r}(M). $

又显然

$ \overline{A}_{M}(r)\subseteq X, {\quad} \bigcup\limits_{M}\overline{A}_{r}(M)\subseteq X. $

故有

$ \bigcup\limits_{M}\overline{A}_{r}(M) = X. $

即$ X $被一致非常返集$ \overline{A}_{r}(M) $覆盖, 故$ X $是非常返集. 从而Markov过程$ P(t) $是非常返的.定理4.1证毕.

定理 4.2  设连续时间Markov过程$ P(t) $是$ \Psi $不可约的, 则

(i) Markov过程$ P(t) $是非常返的$ \iff $其$ h $骨架链$ \{X_{nh}, n\in Z_{+}\} $是非常返的.

(ii) Markov过程$ P(t) $是常返的$ \iff $其$ h $骨架链$ \{X_{nh}, n\in Z_{+}\} $是常返的.

证  (i)

$ \begin{eqnarray*} W(x, A)& = &\int_{0}^{\infty}P(t, x, A){\rm d}t\\ & = &\int_{0}^{h}\int_{X}P(r, x, {\rm d}y)\sum\limits_{m = 1}^{\infty} P(mh, y, A){\rm d}r\\ & = &\int_{0}^{h}\int_{X}P(r, x, {\rm d}y){\rm d}r\bigg[\sum\limits_{m = 1}^{\infty} P(mh, y, A)\bigg]\\ & = &\int_{0}^{h}P(r, x, X){\rm d}r\bigg[\sum\limits_{m = 1}^{\infty} P(mh, y, A)\bigg]. \end{eqnarray*} $

"$ \Leftarrow $"设$ h $骨架链$ \{X_{nh}, n\in Z_{+}\} $的$ m $步转移概率函数为

$ P_{h}^{m}(x, A): = P(mh, x, A), m\geq 1. $

若集合$ A $对$ h $骨架链是一致非常返集, 则$ \exists 0<M<\infty, $使

$ \sum\limits_{m = 1}^{\infty} P_{h}^{m}(x, A) = \sum\limits_{m = 1}^{\infty} P(mh, y, A)\leq M. $

从而$ W(x, A)\leq hM $.即对Markov过程$ P(t) $来说, $ A $也是一致非常返集.由定理4.1知$ P(t) $是非常返的.

"$ \Rightarrow $"由于

$ W(x, A) = \int_{0}^{\infty}P(t, x, A){\rm d}t\geq\sum\limits_{n = 1}^{\infty} P(nh, y, A) = \sum\limits_{n = 1}^{\infty}P_{h}^{n}(x, A). $

若$ P(t) $是非常返的, 即$ \exists 0<M<\infty, $使$ W(x, A)\leq M $. 从而$ \sum\limits_{n = 1}^{\infty}P_{h}^{n}(x, A)\leq M $. 故$ h $骨架链$ \{X_{nh}, n\in Z_{+}\} $是非常返的.

(ii) "$ \Leftarrow $"若$ h $骨架链$ \{X_{nh}, n\in Z_{+}\} $是常返的, 即$ \sum\limits_{n = 1}^{\infty}P_{h}^{n}(x, A) = \infty. $又由于

$ W(x, A) = \int_{0}^{\infty}P(t, x, A){\rm d}t\geq\sum\limits_{n = 1}^{\infty} P(nh, y, A) = \sum\limits_{n = 1}^{\infty}P_{h}^{n}(x, A). $

则$ W(x, A) = \infty, $故$ P(t) $是常返的.

"$ \Rightarrow $" 若$ P(t) $是常返的, 假设其$ h $骨架链$ \{X_{nh}, n\in Z_{+}\} $不是常返的, 则$ h $骨架链$ \{X_{nh}, n\in Z_{+}\} $是非常返的, 又由(i)可得$ P(t) $是非常返的, 矛盾. 从而其$ h $骨架链$ \{X_{nh}, n\in Z_{+}\} $是常返的.

记

$ \Pi(x, {\rm d}y): = \frac{q(x, {\rm d}y)}{q(x)}, $

则$ \Pi(x, {\rm d}y) $是一个概率核, 它确定的离散时间Markov链称为嵌入过程(或跳跃链).

令

$ \Pi^{0} = I, \; \; \; \; \Pi^{n+1} = \Pi^{n}\cdot \Pi. $

引理 4.1^[1]  设$ (q(x), q(x, A)) $是保守$ q $对, 则

$ \int_{0}^{\infty}P^{\min}(t, x, A){\rm d}t = \sum\limits_{n = 0}^{\infty}\int_{A}\Pi ^{n} (x, {\rm d}y)\frac{1}{q(y)}, \; \; \; \; x\in X, A\in{\cal B}(X), $

其中$ P^{\min}(t, x, A) $是$ (q(x), q(x, A)) $确定的最小$ q $过程.

引理 4.2^[4]  Markov过程$ P(t, x, A) $是$ \varphi $不可约的$ \iff $其跳跃链$ \Pi(x, A) $是$ \varphi $不可约的.

参见文献$ [4] $的命题$ 2.2.12 $.

定理 4.3  设连续时间Markov过程$ P(t) $是跳过程, 且是$ \Psi $不可约的, 则

(i) Markov过程$ P(t) $是非常返的$ \iff $其跳跃链$ \Pi(x, A) $是非常返的.

(ii) Markov过程$ P(t) $是常返的$ \iff $其跳跃链$ \Pi(x, A) $是常返的.

证  (i)由引理4.1知

$ \begin{eqnarray*} \int_{0}^{\infty}P(t, x, A){\rm d}t& = &\sum\limits_{n = 0}^{\infty}\int_{A}\Pi^{n} (x, {\rm d}y)\frac{1}{q(y)}, \; \; \; \; x\in X, A\in{\cal B}(X). \\ P(t)\ \rm{是非常返的}&\Leftrightarrow&\int_{0}^{\infty}P(t, x, A){\rm d}t<\infty, \; \; \; \; \forall x\in X, A\in{\cal B}(X)\\ &\Leftrightarrow& \sum\limits_{n = 0}^{\infty}\int_{A}\Pi^{n}(x, {\rm d}y)\frac{1}{q(y)}<\infty\\ &\Leftrightarrow &\sum\limits_{n = 0}^{\infty}\int_{A}\Pi^{n}(x, {\rm d}y)<\infty\Leftrightarrow \sum\limits_{n = 0}^{\infty}\Pi^{n}(x, A)<\infty\\ &\Leftrightarrow& \rm{跳跃链 \Pi(x, A) 是非常返的}. \end{eqnarray*} $

(ii) 由引理4.2知, Markov过程$ P(t, x, A) $是$ \varphi $不可约的$ \iff $其跳跃链$ \Pi(x, A) $是$ \varphi $不可约的.

"$ \Rightarrow $" 若$ P(t) $是常返的, 假设其跳跃链$ \Pi(x, A) $不是常返的, 由于Markov过程$ P(t) $是$ \Psi $不可约的, 从而其跳跃链$ \Pi(x, A) $也是$ \Psi $不可约的. 故$ \Pi(x, A) $是非常返的. 由(i)知$ P(t) $也是非常返的.与假设矛盾. 从而$ \Pi(x, A) $是常返的.

"$ \Leftarrow $"若$ \Pi(x, A) $是常返的, 假设$ P(t) $不是常返的, 由$ P(t) $的$ \Psi $不可约性知$ P(t) $是非常返的. 由(i)知$ \Pi(x, A) $也是非常返的.与假设矛盾. 故$ P(t) $也是常返的.

定义 5.1  集合$ C\in {\cal B}(X) $称为Markov过程$ \{P(t), t\in {{\Bbb R}} _{+}\} $的细集(petite set), 若存在$ {{\Bbb R}} _{+} $上的分布$ a = \{a(t):t\in {{\Bbb R}} _{+}\} $和$ {\cal B}(X) $上的非平凡测度$ \nu_{a} $, 使得对$ \forall x\in C, B\in{\cal B}(X) $,

$ K_{a}(x, B)\geq \nu_{a}(B), $

这里$ \nu_{a} $是$ {\cal B}(X) $上的一非平凡测度.其中概率转移核

$ K_{a}(x, B): = \int^{\infty}_{0}P(t, x, B)a({\rm d}t), \forall x\in X, B\in{\cal B}(X) $

称为$ \{P(t), t\in {{\Bbb R}} _{+}\} $的一个抽样链(sampled chain).

若$ a_\varepsilon(t) = (-\ln\varepsilon)\varepsilon^{t}, 0<\varepsilon<1, t\in {{\Bbb R}} _{+} $是$ [0, +\infty] $上的概率分布, 则

$ K_{a_\varepsilon}(x, B): = \int^{\infty}_{0}P(t, x, B)(-\ln\varepsilon)\varepsilon^{t}({\rm d}t), \forall x\in X, B\in{\cal B}(X). $

定义 5.2  称一个集合$ B\in {\cal B}(X) $是从另一集合$ A\in {\cal B}(X) $用a一致可达的, 记为$ A\mathop{\rightsquigarrow}\limits^{a} B $, 如果存在一$ \delta>0 $使得

$ { }\inf\limits_{x\in A}K_{a}(x, B)>\delta. $

定理 5.1  设$ \{P(t), t\in {{\Bbb R}} _{+}\} $是$ \Psi $不可约Markov过程, $ A $是细集, 则存在一抽样分布$ a $, 使得对任意的$ B\in {\cal B^{+}}(X) $, 有$ A\mathop{\rightsquigarrow}\limits^{a} B $.

证  可假设$ A $是$ \Psi_{b} $细集, $ \Psi_{b} $是一最大不可约测度, 则对所有的$ y\in A $, 都有

$ K_{b}(y, B)\geq \Psi_{b}(B). $

对任意的$ x\in A, B\in {\cal B^{+}}(X) $, 由(3.1)式有

$ \begin{eqnarray*} K_{a_{\varepsilon}\star b}(x, B)& = &\int K_{a_\varepsilon}(x, {\rm d}y)K_{b}(y, B) \\ &\geq& \int_{A} K_{a_\varepsilon}(x, {\rm d}y)K_{b}(y, B)\\ &\geq& \Psi_{b}(B)K_{a_{\varepsilon}}(x, A){\quad} \forall x\in A, \\ K_{a_{\varepsilon}}(x, A)& = &\int^{\infty}_{0}P(t, x, A)(-\ln\varepsilon)\varepsilon^{t}({\rm d}t) \\ &\geq& P(0, x, A)(-\ln\varepsilon)\varepsilon^{0}\\ & = & -\ln\varepsilon>0(0<\varepsilon<1). \end{eqnarray*} $

取

$ \delta = (-\ln\varepsilon)\Psi_{b}(B), a = a_{\varepsilon}\star b. $

若$ B\in {\cal B^{+}}(X) $, 有

$ \forall x\in A, K_{a}(x, B)\geq \delta>0. $

所以

$ \inf\limits_ {x\in A}K_{a}(x, B)\geq \delta>0. $

即对任意的$ B\in {\cal B^{+}}(X) $, 有$ A\mathop{\rightsquigarrow}\limits^{a} B $.

定理 5.2  若$ A, B\in{\cal B}(X), A $是常返集, 且对某抽样分布$ a, A\mathop{\rightsquigarrow}\limits^{a} B $, 则$ B $也是常返集.

证  由$ A\mathop{\rightsquigarrow}\limits^{a} B $知, 存在一$ \delta>0 $使得$ { }\inf_{y\in A}K_{a}(y, B)>\delta $. 再由$ (3.2) $式有

$ \begin{eqnarray*} W(x, B)&\geq &\int W(x, {\rm d}y)K_{a}(y, B) \geq \int_ {A} W(x, {\rm d}y)K_{a}(y, B) \geq \delta W(x, A). \end{eqnarray*} $

由$ A $是常返集知$ W(x, A) = \infty, $从而$ W(x, B) = \infty $. 故$ B $也是常返集.

定理 5.3  设$ \{P(t), t\in {{\Bbb R}} _{+}\} $是$ \Psi $不可约Markov过程, 若存在常返细集$ A\in{\cal B}(X), $则Markov过程$ \{P(t), t\in {{\Bbb R}} _{+}\} $是常返的.

证  由定理5.1知, 存在一抽样分布$ a $, 使得对任意的$ B\in {\cal B^{+}}(X) $, 有$ A\mathop{\rightsquigarrow}\limits^{a} B $. 再由定理$ 5.2 $知, $ B $也是常返集. 故对$ \forall B\in {\cal B^{+}}(X) $都是常返的, 所以马氏过程$ \{P(t), t\in {{\Bbb R}} _{+}\} $是常返的.

定理 5.4  若对$ \forall A\in{\cal B}(X), $有$ \forall x\in A, L(x, A) = 1 $, 则$ A $也是常返集.

证  由于

$ \begin{eqnarray*} P(t, x, B)& = &P_{x}\{X_{t}\in B\}\\ & = &P_{x}\{X_{t}\in B, \tau_{A}\geq t\}+P_{x}\{X_{t}\in B, \tau_{A}< t\}\\ & = &{_{A}}P(t, x, B)+P_{x}\{X_{t}\in B, \tau_{A}< t\}\\ & = &{_{A}}P(t, x, B)+\int_{0}^{t}\int_{A}P(s, x, {\rm d}y) {_{A}}P(t-s, y, B){\rm d}s. \end{eqnarray*} $

所以有

$ P(t, x, A) = {_{A}}P(t, x, A)+\int_{0}^{t}\int_{A}P(s, x, {\rm d}y) {_{A}}P(t-s, y, A){\rm d}s. $

因而

$ \begin{eqnarray*} W(x, A)& = &\int_{0}^{\infty} P(t, x, A){\rm d}t\\ & = &\int_{0}^{\infty}{_{A}}P(t, x, A){\rm d}t+\int_{0}^{\infty}\int_{0}^{t}\int_{A}{_{A}}P(s, x, {\rm d}y) P(t-s, y, A){\rm d}s{\rm d}t\\ & = &L(x, A)+\int_{A}\bigg[\int_{0}^{\infty}\int_{0}^{t}{_{A}}P(s, x, {\rm d}y) P(t-s, y, A){\rm d}s{\rm d}t\bigg]\\ & = &L(x, A)+\int_{A}\bigg[\int_{0}^{\infty}{_{A}}P(s, x, {\rm d}y){\rm d}s\bigg]\bigg [\int_{0}^{\infty}P(t, y, A){\rm d}t\bigg]\\ & = &L(x, A)+\int_{A}W_{A}(x, {\rm d}y)W(y, A). \end{eqnarray*} $

所以

(5.1) $ \begin{eqnarray} W(x, A) = L(x, A)+\int_{A}W_{A}(x, {\rm d}y)W(y, A). \end{eqnarray} $

若$ L(x, A) = 1, \forall x\in A $, 则$ W_{A}(x, A) = 1, \forall x\in A. $\\ 从而由(5.1)式得

$ W(x, A) = 1+W(y, A), \forall y\in A. $

因此对$ \forall x\in A $有$ W(x, A) = \infty $, 故$ A $是常返集.

定理 5.5  设$ \{P(t), t\in {{\Bbb R}} _{+}\} $是$ \Psi $不可约Markov过程, 若存在某细集$ C\in{\cal B}(X) $使得对$ \forall x\in C, L(x, C) = 1 $, 则Markov过程$ \{P(t), t\in {{\Bbb R}} _{+}\} $是常返的.

证  由定理$ 5.4 $知$ C $是常返集.由于$ C $是细集, 再由定理$ 5.3 $知Markov过程$ \{P(t), t\in {{\Bbb R}} _{+}\} $是常返的.

定理 5.6  设$ D^{c} $是吸收集且$ \forall x\in D, L(x, D^{c})>0 $, 则$ D $是非常返集.

证  设$ D^{c} $是吸收集, 记

$ B(t) = \{y\in D:P(t, y, D)\geq t^{-1}\}. $

由于$ \forall x\in D, L(x, D^{c})>0 $, 下证

$ D = \bigcup\limits_{t\geq0}B(t). $

$ \begin{eqnarray*} P(t+s, x, D^{c})& = &\int_{X}P(t, x, {\rm d}y)P(s, y, D^{c})\\ & = &\int_{D}P(t, x, {\rm d}y)P(s, y, D^{c})+\int_{D^{c}}P(t, x, {\rm d}y)P(s, y, D^{c})\\ & = &\int_{D}P(t, x, {\rm d}y)P(s, y, D^{c})+P(t, x, D^{c})\\ &\geq& P(t, x, D^{c}). \end{eqnarray*} $

从而$ P(t, x, D^{c}) $关于$ t $单调递增.

由于$ \forall x\in D, L(x, D^{c})>0 $, 故存在$ t>0 $, 使得$ P(t, x, D^{c})\geq t^{-1}. $从而

$ D\subseteq\bigcup\limits_{t\geq0}B(t). $

又显然

$ B(t)\subseteq D, \; \; \bigcup\limits_{t\geq0}B(t)\subseteq D, $

故

$ D = \bigcup\limits_{t\geq0}B(t). $

由于$ D^{c} $是吸收集, 对每个$ y\in B(t), \forall m\in Z_{+}, $有

$ \begin{eqnarray*} P_{y}(\eta_{B(t)}\geq m)&\leq &P_{y}(\eta_{D}\geq m) \leq P(m, y, D) = 1-P(m, y, D^{c}) \leq 1-m^{-1}. \end{eqnarray*} $

由定理3.4知$ B(t) $是一致非常返集, 从而$ D $是非常返集.

引理 5.1  $ ^{[4]} ${\quad} 假设连续时间Markov过程$ \{P(t), t\in {{\Bbb R}} _{+}\} $是$ \Psi $不可约的, 则

$ {\rm (i)} $每个非空吸收集都是满集.

$ {\rm (ii)} $每个满集包含一个满的吸收集.

证明见文献$ [4] $的命题$ 2.2.8 $.

定理 5.7  设$ \{P(t), t\in {{\Bbb R}} _{+}\} $是$ \Psi $不可约Markov过程, 则每个$ \Psi $零测集都是非常返的.

证设$ \{P(t), t\in {{\Bbb R}} _{+}\} $是$ \Psi $不可约Markov过程, $ D $是一$ \Psi $零测集. 由引理$ 5.1 $知$ D^{c} $包含吸收集$ F $, 由定理$ 5.6 $知$ F^{c} $能被一致非常返集覆盖. 由$ F\subseteq D^{c} $有$ D\subseteq F^{c} $. 即$ D $是非常返集.