Sampling-based inference and learning techniques, especially Bayesian inference, provide an essential approach to handling uncertainty in machine learning (ML).   As these techniques are increasingly used in daily life, it becomes essential to safeguard the ML systems with various trustworthy-related constraints, such as fairness, safety, interpretability. We propose a family of constrained sampling algorithms which generalize Langevin Dynamics (LD) and Stein Variational Gradient Descent (SVGD) to incorporate a moment constraint or a level set  specified by a general nonlinear function. By exploiting the gradient flow structure of LD and SVGD, we derive algorithms for handling constraints, including a  primal-dual gradient approach and the constraint controlled gradient descent approach.  We investigate the continuous-time mean-field limit of these algorithms and show that they have $O(1/t)$ convergence under mild conditions.

Speaker Bio:
Dr. Xin Tong is an associate professor at the National University of Singapore, department of mathematics. He received his Ph.D. degree from Princeton University in 2013. Prior to his position at the National University of Singapore, he was a postdoc at the Courant Institute of New York University. His recent research focuses on the analysis and derivation of stochastic algorithms.