Machine Learning

Probabilistic programing languages (PPLs) are formal languages for probabilistic modeling that describe complex distributions via programs with random choices. As a description language PPLs are a convenient and powerful way to construct models. I will show several examples drawn from cognitive science, focussing on language understanding: I will use a PPL to construct a model of language understanding as social reasoning; this model captures aspects of vague and figurative language. PPL implementations make it possible to separate inference algorithms from model representation by providing universal inference engines. Many techniques have been explored for inference, but they all struggle with efficiency for different model classes. Instead, we have been exploring systems that learn efficient inference strategies. I will discuss “deep amortized inference for PPLs”, a system that optimizes deep neural network “guide programs” to capture the implicit posterior distribution of a probabilistic program.

Bio: Noah Goodman is Associate Professor of Psychology, Computer Science (by courtesy), and Linguistics (by courtesy) at Stanford University. He studies the computational basis of natural and artificial intelligence, merging behavioral experiments with formal methods from statistics and programming languages. His research topics include language understanding, social reasoning, concept learning, probabilistic programming languages, and applications. ProfessorGoodman received his Ph.D. in mathematics from the University of Texas at Austin in 2003. In 2005 he entered cognitive science, working as Postdoc and Research Scientist at MIT. In 2010 he moved to Stanford where he runs the Computation and Cognition Lab. Professor Goodmanhas published more than 150 papers in fields including psychology, linguistics, computer science, and mathematics. His work has been recognized by the James S. McDonnell Foundation Scholar Award, the Roger N. Shepard Distinguished Visiting Scholar Award, the Alfred P. Sloan Research Fellowship in Neuroscience, and six computational modeling prizes from the Cognitive Science Society. He is Fellow of the Uber AI Labs, Academic Co-Founder and Advisor of Gamalon Labs, and advisor to several other start-ups.

Our ability to collect data has exploded over recent years. Across science, we now collect thousands of measurements on each person. One of the most common tasks of interest is to use these measurements to predict some response. Prediction methods must balance 3 objectives: predictive performance, computational tractability, and, in many applications, interpretability. In this talk we will discuss a broad class of models which effectively balance these objectives: Sparse additive models induced by combining a structural semi-norm and sparsity penalty. These are more flexible than the standard linear penalized model, but maintain its interpretability and computational tractability. We will show when these penalties can and cannot be combined to induce the desired structure and sparsity. We will give an efficient algorithm for fitting a wide class of these models. We will also discuss a particular type of structural penalty (hierarchical sparsity) which has additional nice behaviour. We will consider an application: Predicting disease phenotype in inflammatory bowel disease from gene expression measurements. Time permitting, we will additionally touch on the theoretical properties of these sparse additive estimators.

The field of genomics over the past decade has been driven by technological advances as the throughput and diversity of genomics assays has increased dramatically. To help biologists make sense of the resulting big, heterogeneous data sets, we have developed an unsupervised learning strategy that uses a dynamic Bayesian network in which "time" corresponds to genomic position. I will describe our methodology as well as our recent efforts to improve the utility and interpretability of the resulting annotation. I will also discuss a tensor factorization method that we have successfully employed to impute missing genomics data, allowing us to accurately predict the outcome of genomics experiments that have not yet been performed.

Existing models for situated question answering make strong independence assumptions that negatively impact their accuracy. These assumptions, while empirically false, are necessary to facilitate inference because the number of joint question/environment interpretations is extremely large, typically superexponential in the number of objects in the environment. We present Parsing to Probabilistic Programs (P3), a novel situated question answering model that embraces approximate inference to eliminate these independence assumptions and enable the use of arbitrary global features of the question/environment interpretation. Our key insight is to treat semantic parses as probabilistic programs that are executed nondeterministically, and whose possible executions represent environmental uncertainty. We evaluate our approach on a new, publicly-released data set of 5000 diagram questions from a science domain, finding that our approach outperforms several competitive baselines.