In this talk, I will discuss two subclasses of nonconvex optimization problems motivated by applications in machine learning. The first one is bound-constrained problems. This problem class captures applications including nonnegative matrix factorization. Inspired by a two-metric projection method from Bertsekas, we propose a projected Newton-CG algorithm equipped with good worst-case complexity guarantees. The algorithm also has competitive fundamental operation complexity in terms of Hessian-vector product, thus suitable for large-scale computation. The second problem class is complicated by nonconvex expected-valued objective functions. Such a formulation can capture deep learning, policy optimization, autoencoder training, etc. It is known in literature that the sample complexity of stochastic first-order methods depend linearly on the problem dimension, which is undesirable for solving large-scale problems. To address this issue, we propose dimension insensitive stochastic first-order methods. Our algorithm allow for non-Euclidean and nonsmooth distance functions as the proximal terms when taking the stochastic gradient step. State-of-art sample complexity guarantees in terms of the dimension are shown.

Speaker: Dr. Yue XIE
Date: 4 November 2024 (Monday)
Time: 10:00am – 11:00am
Poster: Click here