## Regular Kernel Method for State Space Model

Wang Chao,, Li Bo,, Wang Lei

Department of Mathematics and Statistics, China Central Normal University, Wuhan 430071

 基金资助: 国家自然科学基金.  61877023中央高校基本科研专项资金.  CCNU19TD009湖北省科技创新基地（平台）专项.  2020DFH002

 Fund supported: the NSFC.  61877023the Fundamental Research Funds for the Central Universities.  CCNU19TD009the Hubei Provincial Science and Technology Innovation Base (Platform) Special Project.  2020DFH002

Abstract

State space models(SSMs) provide a general framework for studying stochastic processes, which has been applied in revealing the true underlying economic processes of an economy, recognizing cellphone signals, detecting the loaction of an airplane on a radar screen, et al. In this paper, we study the Markov state space models by modeling the space transformation with reproducing kernel Hilbert space. Not only the existence and uniqueness of solutions are given, but also the error is estimate in L2 spaces. We applied our method in Visibility prediction at airport in National Post-Graduate Mathematical Contest in Modeling supported by China Academic Degrees & Graduate Education Development Center.

Keywords： State Space Model ; Function Reconstruction ; Kernel Method ; Autoregression

Wang Chao, Li Bo, Wang Lei. Regular Kernel Method for State Space Model. Acta Mathematica Scientia[J], 2022, 42(3): 881-890 doi:

## 1 介绍

$$$\begin{array}{ll} x_t = h_t({\bf{x}}_{t-1}, \epsilon_t)\quad \mbox{或}\quad x_t\sim q_t(\cdot|{\bf{x}}_{t-1}), \\ y_t = g_t({\bf{x}}_t, \delta_t)\quad \mbox{或} \quad y_t\sim f_t(\cdot|{\bf{x}}_t), \end{array}$$$

$$$\begin{array}{ll} \mbox{状态转移方程}: x_t=h_t(x_{t-1}, \epsilon_t)&\quad \mbox{或}\quad x_t\sim q_t(\cdot|x_{t-1}), \\ \mbox{观测方程}: y_t = g_t(x_t, \delta_t)&\quad \mbox{或} \quad y_t\sim f_t(\cdot|x_t). \end{array}$$$

$$$y_t=x_t+\delta_t,$$$

### 2.2 核状态空间模型

$$$\begin{array}{ll} x_t=f(x_{t-1}) + \epsilon_t, \\ y_t=x_t + \delta_t. \end{array}$$$

${\cal H}_{K, \bar{y}}=\{\sum\limits_{z\in\bar{y}}C_zK_z(\cdot)|C_z\in l^2(\bar{y})\}$为连续函数的再生核希尔伯特空间, 其中内积按照定义2.1的方式规定. 且状态转移函数$f(\cdot)\in{\cal H}_{K, \bar{y}}$与时间$t$无关. 由于$y_t=x_t+\delta_t$, 我们不妨假设$y_t $$x_t 来源于同一空间 {\cal X} . 在给出具体的性质之前, 此处给出抽样算子的定义. 定义2.2 设 f\in {\cal H}_{K, \bar{y}} . 抽样算子 S_{\bar{y}}:{\cal H}_{K, \bar{y}}\rightarrow l^2(\bar{y}) 被定义为 $${\nonumber} S_{\bar{y}}(f)=(f(y))_{y\in \bar{y}}.$$ 一般地, 假设 S_{\bar{y}} 是有界算子. 当 \bar{y} 为有限集时, 抽样算子的有界性是明显的. 进一步, 这里给出算子 S_{\bar{y}} 的共轭算子, 记为 S^T_{\bar{y}} . c\in l^2(\bar{y}) , 则有 其中, 最后一个等式是由于核函数的再生性, 即对任意 {\cal H}_{K, \bar{y}} 中的有界函数 f:{\cal X}\rightarrow {{\Bbb R}} , 假设最大范数至多为 \|J\| , 存在 E_y\in{\cal H}_{K, \bar{y}}$$ \|E_y\|_K\leq \|J\|$, 满足

$$$f(x)=\langle f, E_x\rangle_K, \quad \forall f\in {\cal H}_{K, \bar{y}}.$$$

### 2.3 问题求解和唯一性

$\gamma\geq 0$, 给定时间序列样本$\bar{y}=(y_t)_{t\in\bar{t}}$, 考虑下面最优化问题

$$$\tilde{f}:=\mathop{\rm arg\min}\limits_{f\in{\cal H}_{K, \bar{y}}} \left\{\sum\limits_{t\in\bar{t}}(y_t-f(y_{t-1}))^2+\gamma\|f\|^2_K\right\}.$$$

注意到$f\in{\cal H}_{K, \bar{y}}$, 因此有$f=\sum\limits_{t\in\bar{t}}c_tE_{y_t}=S^T_{\bar{y}}c$. 从而该优化问题的内积形式为

### 2.4 预测误差估计

$f^{\star}$为真实状态转移函数, 定义

根据定理5.1, 将$\|\tilde{f} - f^{\star}_{\bar{y}, \gamma}\|^2_{K}$展开为

$$$\mbox{Prob}\left\{\sum\limits_{t\in\bar{t}}w_t[\eta_t-E(\eta_t)]>\epsilon\right\}\leq 2\mbox{exp}\left\{-\frac{\epsilon}{2\|w\|_{\infty}M}\mbox{log}\left(1 + \frac{M\epsilon}{\sigma^2_w}\right)\right\}.$$$

### 2.5 多维状态变量问题

$$$(\tilde{f}_j)^p_{j=1}:\mathop{\rm arg\min}\limits_{f_1, \cdots, f_p\in{\cal H}_{K, \bar{y}}} \left\{\sum\limits_{t\in\bar{t}}\mathop \sum \limits_{j = 1}^p (y_{t, j} - f_{j}(y_{t-1}))^2+\mathop \sum \limits_{j = 1}^p \gamma_j\|f_j\|^2_K\right\},$$$

1) 将原始观测数据拓展为$p$份独立观测数据$(y_0, y_{1, j})$, $(y_1, y_{2, j})$, $\cdots$, $(y_{T-1}, y_{T, j})$, $j=1, \cdots, p$.

2) 对于每一份观测数据, 运用正则化最小二乘法求解$\tilde{f}_j(\cdot):=\sum\limits_{t\in\bar{t}}c_tK_t(\cdot)$.

3) 输出多状态估计函数式$(\tilde{f}_j)^p_{j=1}$.

## 3 模拟实验

设$E(\eta_t)=0$, 记$\eta_t$的方差为$\sigma^2_t=E(\eta^2_t)$. 假设$|\bar{t}|<\infty$, 则有

$$$P:=\mbox{Prob}\left\{\sum\limits_{t\in\bar{t}}w_t\eta_t>\epsilon\right\}\leq \exp\left\{-\frac{\epsilon}{2\|w\|_{\infty}M}\mbox{log}\left(1 + \frac{M\epsilon}{\sigma^2_w}\right)\right\}.$$$

$c$为任意正常数, 根据独立性有

$g(\lambda)=(1 + \lambda)\log(1+ \lambda) - \lambda$, $\lambda \geq 0$, 则有

## 参考文献 原文顺序 文献年度倒序 文中引用次数倒序 被引期刊影响因子

Schulz R , Werwatz A .

A state space model for berlin house prices: Estimation and economic interpretation

The Journal of Real Estate Finance and Economics, 2004, 28 (1): 37- 57

Ding F .

State filtering and parameter estimation for state space systems with scarce measurements

Signal Processing, 2014, 104, 369- 380

Smith A C , Brown E N .

Estimating a state-space model from point process observations

Neural Computation, 2003, 15 (5): 965- 991

Liu J , Xu J C , Hu Y J .

On the expected discounted penalty function in a markov-dependent risk model with constant dividend barrier

Acta Mathematica Scientia, 2010, 30B (5): 1481- 1491

Kim S E , Behr M K , Ba D , Brown E N .

State-space multitaper time-frequency analysis

Proceedings of the National Academy of Sciences, 2018, 115 (1): E5- E14

Fasen-Hartmann V , Scholz M .

Quasi-maximum likelihood estimation for cointegrated continuous-time linear state space models observed at low frequencies

Electronic Journal of Statistics, 2019, 13 (2): 5151- 5212

Welch G, Bishop G, et al. An introduction to the Kalman filter[R]. 1995. http://www.clear.rice.edu/comp450/papers/450_kalman.pdf

Liu J S , Rong C .

Sequential monte carlo methods for dynamic systems

Journal of the America Statistical Association, 1998, 93 (443): 1032- 1044

Pieralberto G , Adam M J , Anthony L .

The iterated auxiliary particle filter

Journal of the American Statistical Association, 2017, 112 (520): 1636- 1647

Akaike H .

Fitting autoregressive models for prediction

Annals of the institute of Statistical Mathematics, 1969, 21 (1): 243- 247

Akaike H .

Statistical predictor identification

Annals of the Institute of Statistical Mathematics, 1970, 22 (1): 203- 217

Kalman R E .

Contributions to the theory of optimal control

Boletín de la Sociedad Matemática Mexicana, 1960, 5 (2): 102- 119

/

 〈 〉