Latent variable models have become a key tool for the modern
statistician, letting us express complex assumptions about the hidden
structures that underlie our data. Latent variable models have been
successfully applied in numerous fields including natural language
processing, computer vision, electronic medical records, genetics,
neuroscience, astronomy, political science, sociology, the digital
humanities, and many others.
The central computational problem in latent variable modeling is
posterior inference, the problem of approximating the conditional
distribution of the latent variables given the observations.
Posterior inference is central to both exploratory tasks, where we
investigate hidden structures that underlie our data, and predictive
tasks, where we use the inferred structures to generalize about future
data. Approximate posterior inference algorithms have revolutionized
Bayesian statistics, revealing its potential as a usable and
general-purpose language for data analysis.
Bayesian statistics, however, has not yet reached this potential.
First, statisticians and scientists regularly encounter massive data
sets, but existing approximate inference algorithms do not scale well.
Second, most approximate inference algorithms are not generic; each
must be adapted to the specific model at hand. This often requires
significant model-specific analysis, which precludes us from easily
exploring a variety of models.
In this talk I will discuss our recent research on addressing these
two limitations. First I will describe stochastic variational
inference, an approximate inference algorithm for handling massive
data sets. Stochastic inference is easily applied to a large class of
Bayesian models, including time-series models, factor models, and
Bayesian nonparametric models. I will demonstrate its application to
probabilistic topic models of text conditioned on millions of
articles. Stochastic inference opens the door to scalable Bayesian
computation for modern data analysis.
Then I will discuss black box variational inference. Black box
inference is a generic algorithm for approximating the posterior. We
can easily apply it to many models with little model-specific
derivation and few restrictions on their properties. Black box
inference performs better than similarly generic sampling algorithms,
such as Metropolis-Hastings inside Gibbs, and can be composed with
stochastic inference to handle massive data. I will demonstrate its
use on a suite of nonconjugate models of longitudinal healthcare data.
This is joint work based on these two papers:
M. Hoffman, D. Blei, J. Paisley, and C. Wang. Stochastic variational
inference. Journal of Machine Learning Research, 14:1303-1347.
R. Ranganath and D. Blei. Black box variational inference.
Artificial Intelligence and Statistics, 2014.