高斯核正则化学习算法的泛化误差

数学物理学报 ›› 2014, Vol. 34 ›› Issue (5): 1049-1060.

• 论文 • 下一篇

高斯核正则化学习算法的泛化误差

张永全^1,2|李有梅¹|曹飞龙¹|徐宗本²

1.中国计量学院 |数学与信息科学系杭州 |310018;
2.西安交通大学 |信息与系统科学研究所西安 |710049

收稿日期:2012-07-28 修回日期:2013-12-27 出版日期:2014-10-25 发布日期:2014-10-25
基金资助:
国家自然科学基金(11301494)和浙江省自然科学基金(Q12A01026)资助.

Generalization Bounds of Compressed Regression Learning Algorithm

ZHANG Yong-Quan^1,2, LI You-Mei¹, CAO Fei-Long¹, XU Zong-Ben²

1.Department of Information and Mathematics Sciences, China Jiliang University, Hangzhou 310018;
2.Institute for Information and System Sciences, Xi'an Jiaotong University, Xi'an 710049

Received:2012-07-28 Revised:2013-12-27 Online:2014-10-25 Published:2014-10-25
Supported by:
国家自然科学基金(11301494)和浙江省自然科学基金(Q12A01026)资助.

摘要/Abstract

摘要：

对广义凸损失函数和变高斯核情形下正则化学习算法的泛化性能展开研究. 其目标是给出学习算法泛化误差的一个较为满意上界. 泛化误差可以利用正则误差和样本误差来测定. 基于高斯核的特性, 通过构构建一个径向基函数(简记为RBF) 神经网络, 给出了正则误差的上界估计, 通过投影算子和再生高斯核希尔伯特空间的覆盖数给出样本误差的上界估计. 所获结果表明, 通过适当选取参数σ和λ，可以提高学习算法的泛化性能.

关键词: 学习理论, RBF神经网络, 高斯核, 泛化误差

Abstract:

This article studies generalization performance of regularized learning algorithm with a general convex loss function and varying Gaussian kernels. Our goal is to give a satisfactory estimate of generalization error for the learning algorithm. The
generalization error is measured by regularization error and sample error. The regularization error is estimated by constructing a radial basis function (briefly denoted by RBF) neural network in view of the special structures of Gaussian kernels. The sample error is obtained by using projection operator and covering number of reproducing kernel Hilbert spaces with Gaussian kernels. The obtained results demonstrate the learning algorithm has good generalization performance with suitable choice of m and λ.

Key words: Learning theory, RBF neural network, Gaussian kernels, Generalization error

中图分类号:

41A25

张永全, 李有梅, 曹飞龙, 徐宗本. 高斯核正则化学习算法的泛化误差[J]. 数学物理学报, 2014, 34(5): 1049-1060.

ZHANG Yong-Quan, LI You-Mei, CAO Fei-Long, XU Zong-Ben. Generalization Bounds of Compressed Regression Learning Algorithm[J]. Acta mathematica scientia,Series A, 2014, 34(5): 1049-1060.

参考文献

[1] Vapnik V, Chervonenkis A. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications, 1971, 16: 264--280

[2] Wu Q, Ying Y M, Zhou D X. Learning rates of least-square regularized regression. Found Comput Math, 2006, 6: 171--192

[3] Cucker F, Smale S. On the mathematical foundations of learning. Bull Amer Math Soc, 2001, 39: 1--49

[4] Caponnetto A, DeVito E. Optimal rates for the regularized least-squares algorithm. Found Comput Math, 2007, 7: 331--368

[5] Cybenko G. Approximtion by superpositon of sigmodial functions. Mathematics of Control, Signal, and Systems, 1989, 3: 303--314

[6] Esposito A, Marinaro M, Oricchio D, Scarpetta S. Approximation of continuous and discontinuous mappings by a growing neural RBF-basd algorithm. Neural Networks, 2000, 13: 651--665

[7] Smale S, Zhou D X. Estimating the approximation error in learning theory. Anal Appl, 2003, 1: 17--41

[8] Smale S, Zhou D X. Shannon sampling and function reconstruction from point values. Bull Amer Math Soc, 2004, 41: 279--305

[9] Tong H Z, Chen D R, Li Z P. Learning rates for regularized classifiers using multivariate polynomial kernel. J Complexity, 2008, 24: 619--631

[10] Wu Q, Ying Y M, Zhou D X. Learning theory: from regression to classification. Studies in Computational Mathematics, 2006, 12: 257--290

[11] Zhou D X, Jetter K. Approximation with polynomial kernels and SVM classifiers. Adv Comput Math, 2006, 25: 323--344

[12] Aronszajn N. Theory of reproducing kernels. Trans Amer Soc, 1950, 68: 337--404

[13] Xiang D H, Zhou D X. Classification with gaussians and convex loss. J Mach Learning Res, 2009, 10: 1447--1468

[14] Steinwart I. Support vector machines are universally consistent. J Complexity, 2002, 18: 768--791

[15] Ye G B, Zhou D X. Learning and approximation by Gaussians on Riemannian manifolds. Adv Comput Math, 2008, 29: 291--310

[16] Maiorov V. Approximation by neural networks and learning theory. J Complexity, 2006, 22: 102--117

[17] Wu Q, Zhou D X. Support vector machine classifiers: linear programming versus quadratic programming. Neural Computation, 2005, 17: 1160--1187

[18] Xie T F, Cao F L. The rate of approximation of Gaussian radial basis neural networks in continuous function space.
Acta Mathematica Sinica, 2013, 29: 295--302

[19] Cucker F, Zhou D X. Learning Theory: An Approximation Theory Viewpoint. Cambridge: Cambridge University Press, 2007

[20] Bartlett P L. The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of network. IEEE Trans Inform Theory, 1998, 44: 525--536

[21] Zhou D X. The covering number in learning theory. J Complexity, 2002, 18: 739--767

[22] Zhou D X. Capacity of reproducing kernel spaces in learning theory. IEEE Trans Inform Theory, 2003, 49: 1734--1752

[23] Williamson R C, Smola A J, Sch\"{o}kopf B. Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators. IEEE Trans Inform Theory, 2001, 47: 2516--2532

[24] Pontil M. A note different covering numbers in learning theory. J Complexity, 2003, 19: 665--671

高斯核正则化学习算法的泛化误差

Generalization Bounds of Compressed Regression Learning Algorithm

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

Metrics

本文评价

推荐阅读 10