NEPv Approach for Optimization on Stiefel Manifold with the -norm Regularization
En palabras de los autores
Row-sparse projection provides a useful tool in machine learning (ML) when it comes to, for example, feature selection, aiming to choose most relevant features for various ML objectives. One way to seek a high quality row-sparse projection is to combine an ML objective, such as the ones for PCA, LDA, and OCCA, with the matrix -norm regularization which is nonsmooth. Such combinations result in challenging optimization problems on the Stiefel manifold that need to be solved efficiently. In this paper, a unifying NEPv framework is established to efficiently deal with optimization on the Stiefel manifold with the -norm regularization. The effect of the -norm regularization is also investigated. The wide applicability of the framework is demonstrated through the combinations of common learning objectives in today's data science applications with the -norm regularization. Numerical experiments are presented to illustrate the use of the NEPv approach and to gain insights as to what a proper regularizing parameter should have in real-world applications.
Apareció: miércoles, 23 de septiembre. arXiv. Preprint, todavía sin revisión por pares.