Mean-field Optimization regularized by Fisher Information
Recently there is a rising interest in the research of mean-field optimization, in particular because of its role in analysing the training of neural networks. In this talk, by adding the Fisher Information (in other word, the Schrodinger kinetic energy) as the regularizer, we relate the mean-field optimization problem with a so-called mean field Schrodinger (MFS) dynamics. We develop a free energy method to show that the marginal distributions of the MFS dynamics converge towards the unique minimizer of the regularized optimization problem. We shall see that the MFS is a gradient flow on the probability measure space. Finally we propose a Monte Carlo method to sample the marginal distributions of the MFS dynamics. This is an ongoing joint work with Julien Claisse, Giovanni Conforti and Songbo Wang.