特征提取中一类矩阵迹函数极值问题的黎曼优化算法

Abstract

Abstract:

The present study focuses on robust discriminant regression models for feature extraction, which can be rephrased as minimizing matrix trace function subject to product manifold constraints. By building upon the Zhang-Hager technique, the authors develop a Riemannian nonlinear conjugate gradient method for solving a simplified version of the reconstruction problem. The method exploits the geometric properties of the product manifold, and the global convergence analysis of the proposed algorithm is provided. Empirical results demonstrate that the proposed algorithm is effective and feasible for solving the underlying problem. In terms of iteration efficiency, the proposed algorithm outperforms the existing method, other Riemannian gradient-like algorithms and Riemannian first-order and second-order algorithms available in the MATLAB toolbox Manopt.

Key words: Feature extraction, Matrix trace function, Product manifold, Riemannian conjugate gradient method

CLC Number:

O151.1

Li Jiaofen, Kong Lvyuan, Song Jiashuo, Wen Yaqiong. A Riemannian Optimization Approach for a Class of Matrix Trace Function Extremum Problem in Feature Extraction[J].Acta mathematica scientia,Series A, 2024, 44(4): 1012-1036.

Trendmd

Figures/Tables 8

References 44

[1]	Ye J. Least squares linear discriminant analysis//Proceedings of the 24th International Conference on Machine Learning. 2007: 1087-1093
[2]	Ma Z, Nie F, Yang Y, et al. Discriminating joint feature analysis for multimedia data understanding. IEEE Transactions on Multimedia, 2012, 14(6): 1662-1672
[3]	Ma Z, Yang Y, Sebe N, et al. Multimedia event detection using a classifier-specific intermediate representation. IEEE Transactions on Multimedia, 2013, 15(7): 1628-1637
[4]	Ji S, Tang L, Yu S, Ye J. A shared-subspace learning framework for multi-label classification. ACM Transactions on Knowledge Discovery from Data (TKDD), 2010, 4(2): 1-29
[5]	Seung H S, Lee D D. The manifold ways of perception. Science, 2000, 290(5500): 2268-2269 pmid: 11188725
[6]	Roweis S T, Saul L K. Nonlinear dimensionality reduction by locally linear embedding. Science, 2000, 290(5500): 2323-2326 doi: 10.1126/science.290.5500.2323 pmid: 11125150
[7]	Belkin M, Niyogi P. Laplacian eigenmaps and spectral techniques for embedding and clustering//Advances in Neural Information Processing Systems. Cambridge: MIT press, 2001, 14
[8]	Zhang S, Ma Z, Tan H. On the equivalence of HLLE and LTSA. IEEE Transactions on Cybernetics, 2017, 48(2): 742-753
[9]	Lai Z, Mo D, Wong W K, et al. Robust discriminant regression for feature extraction. IEEE Transactions on Cybernetics, 2017, 48(8): 2472-2484
[10]	Nie F, Huang H, Cai X, Ding C H. Efficient and robust feature selection via joint $ L_{2, 1} $ -norms minimization//Advances in Neural Information Processing Systems, 2010: 1813-1821
[11]	Sato K, Sato H. Structure-preserving $ H^{2} $ optimal model reduction based on the Riemannian trust-region method. IEEE Transactions on Automatic Control, 2017, 63(2): 505-512
[12]	Sato H, Sato K. Riemannian gradient-based online identification method for linear systems with symmetric positive-definite matrix// 2019 IEEE 58th Conference on Decision and Control (CDC), 2019: 3593-3598
[13]	Sato K. Riemannian optimal model reduction of linear port-Hamiltonian systems. Automatica, 2018, 93: 428-434
[14]	Sato K. Riemannian optimal control and model matching of linear port-Hamiltonian systems. IEEE Transactions on Automatic Control, 2017, 62(12): 6575-6581
[15]	Sato K, Sato H, Damm T. Riemannian optimal identification method for linear systems with symmetric positive-definite matrix. IEEE Transactions on Automatic Control, 2020, 65(11): 4493-4508
[16]	Chiang C Y, Lin M M, Jin X Q. Riemannian inexact Newton method for structured inverse eigenvalue and singular value problems. BIT Numerical Mathematics, 2019, 59: 675-694
[17]	Ishteva M, Absil P A, Huffel S V, Lathauwer L D. Best low multilinear rank approximation of higher-order tensors, based on the Riemannian trust-region scheme. SIAM Journal on Matrix Analysis and Applications, 2011, 32(1): 115-135
[18]	Sato H, Iwai T. A Riemannian optimization approach to the matrix singular value decomposition. SIAM Journal on Optimization, 2013, 23(1): 188-212
[19]	Wang Y, Zhao Z, Bai Z J. Riemannian Newton-CG methods for constructing a positive doubly stochastic matrix from spectral data. Inverse Problems, 2020, 36(11): 115006
[20]	Yao T T, Bai Z J, Zhao Z, Ching W K. A riemannian fletcher-reeves conjugate gradient method for doubly stochastic inverse eigenvalue problems. SIAM Journal on Matrix Analysis and Applications, 2016, 37(1): 215-234
[21]	Yao T T, Bai Z J, Jin X Q, Zhao Z. A geometric Gauss-Newton method for least squares inverse eigenvalue problems. BIT Numerical Mathematics, 2020, 60: 825-852
[22]	Zhao Z, Jin X Q, Bai Z J. A geometric nonlinear conjugate gradient method for stochastic inverse eigenvalue problems. SIAM Journal on Numerical Analysis, 2016, 54(4): 2015-2035
[23]	Zhao Z, Bai Z J, Jin X Q. A Riemannian inexact Newton-CG method for constructing a nonnegative matrix with prescribed realizable spectrum. Numerische Mathematik, 2018, 140(4): 827-855
[24]	Zhao Z, Jin X Q, Yao T T. A Riemannian under-determined BFGS method for least squares inverse eigenvalue problems. BIT Numerical Mathematics, 2022, 62(1): 311-337
[25]	Zhao Z, Yao T T, Bai Z J, Jin X Q. A Riemannian inexact Newton dogleg method for constructing a symmetric nonnegative matrix with prescribed spectrum. Numerical Algorithms, 2023, 92: 1951-1981
[26]	Jiang Y L, Xu K L. Riemannian modified Polak-Ribière-Polyak conjugate gradient order reduced model by tensor techniques. SIAM Journal on Matrix Analysis and Applications, 2020, 41(2): 432-463
[27]	Oviedo H. Global convergence of Riemannian line search methods with a Zhang-Hager-type condition. Numerical Algorithms, 2022, 91(3): 1183-1203
[28]	Sato H. Riemannian Optimization and Its Applications. Brelin: Springer, 2021
[29]	Sato H, Iwai T. A new, globally convergent Riemannian conjugate gradient method. Optimization, 2015, 64(4): 1011-1031
[30]	Sato H. A Dai-Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditions. Computational Optimization and Applications, 2016, 64(1): 101-118
[31]	Sakai H, Iiduka H. Hybrid Riemannian conjugate gradient methods with global convergence properties. Computational Optimization and Applications, 2020, 77(3): 811-830
[32]	Zhu X. A Riemannian conjugate gradient method for optimization on the Stiefel manifold. Computational optimization and Applications, 2017, 67(1): 73-110
[33]	Vandereycken B. Low-rank matrix completion by Riemannian optimization. SIAM Journal on Optimization, 2013, 23(2): 1214-1236
[34]	Zhang H, Hager W W. A nonmonotone line search technique and its application to unconstrained optimization. SIAM Journal on Optimization, 2004, 14(4): 1043-1056
[35]	Absil P A, Mahony R, Sepulchre R. Optimization Algorithms on Matrix Manifolds. Princeton: Princeton University Press, 2009
[36]	Ring W, Wirth B. Optimization methods on Riemannian manifolds and their application to shape space. SIAM Journal on Optimization, 2012, 22(2): 596-627
[37]	Absil P A, Mahony R, Trumpf J. An extrinsic look at the Riemannian Hessian//International Conference on Geometric Science of Information. Berlin: Springer, 2013: 361-368
[38]	Nocedal J, Wright S J. Numerical Optimization. Springer, 1999
[39]	Grippo L, Lampariello F, Lucidi S. A nonmonotone line search technique for Newton's method. SIAM Journal on Numerical Analysis, 1986, 23(4): 707-716
[40]	Zhang L, Zhou W, Li D. Global convergence of a modified Fletcher-Reeves conjugate gradient method with Armijo-type line search. Numerische Mathematik, 2006, 104(4): 561-572
[41]	王松桂, 吴密霞, 贾忠贞. 矩阵不等式. 北京: 科学出版社, 2006
	Wang S G, Wu M X, Jia Z Z. Matrix Inequalities. Beijing: Science Press, 2006
[42]	Wen Z, Yin W. A feasible method for optimization with orthogonality constraints. Mathematical Programming, 2013, 142(1/2): 397-434
[43]	Li J F, Li W, Vong S W, et al. A Riemannian optimization approach for solving the generalized eigenvalue problem for nonsquare matrix pencils. Journal of Scientific Computing, 2020, 82: 1-43
[44]	Boumal N, Mishra B, Absil P A, Sepulchre R. Manopt, a Matlab toolbox for optimization on manifolds. Journal of Machine Learning Research, 2014, 15(1): 1455-1459

A Riemannian Optimization Approach for a Class of Matrix Trace Function Extremum Problem in Feature Extraction

RichHTML

PDF (PC)

Abstract

Cite this article

share this article

Figures/Tables 8

References 44

Related Articles 2

Metrics

Comments

Recommended 10

[1]	Yueyuan Liu,Kai Wang,Shujuan Qin,Jiaofen Li. An Effective Algorithm for Generalized Sylvester Equation Minimization Problem Under Columnwise Orthogonal Constraints [J]. Acta mathematica scientia,Series A, 2021, 41(2): 479-495.
[2]	Ya Gao,Jing Mao,Chunlan Song. Existence and Uniqueness of Solutions to the Constant Mean Curvature Equation with Nonzero Neumann Boundary Data in Product Manifold $M^{n}\times{\Bbb R}$ [J]. Acta mathematica scientia,Series A, 2020, 40(6): 1525-1536.