Abstract: Language modeling, in the form of ELMO and BERT, yields useful pre-trained task-tunable models. This success of unsupervised pre-training distinguishes language, with its discrete signals, from continuous-signal domains such as speech and vision. Until now the most effective pre-trained task-tunable models in continuous-signal domains have been generated from human-annotated data. This talk will discuss maximum mutual information (MMI) predictive coding as a unifying framework for unsupervised training in both continuous and discrete domains. This talk will describe the motivation, the mathematics, and some very promising computer vision results out of Montreal. This talk will also present a speculative discussion of MMI predictive coding as a language modeling approach to natural language semantics.

Bio: Professor McAllester received his B.S., M.S., and Ph.D. degrees from the Massachusetts Institute of Technology in 1978, 1979, and 1987 respectively. He served on the faculty of Cornell University for the academic year of 1987-1988 and served on the faculty of MIT from 1988 to 1995. He was a member of technical staff at AT&T Labs-Research from 1995 to 2002. He has been a fellow of the Association for the Advancement of Artificial Intelligence (AAAI) since 1997. From 2002 to 2017 he was Chief Academic Officer at the Toyota Technological Institute at Chicago (TTIC) where he is currently a Professor. He has received two 20 year "test of time" awards --- for a paper on systematic nonlinear planning at the AAAI conference and for a paper on interval methods for constraint solving at the International Conference of Logic Programming.

Speaker: 
David McAllester
Time/Date: 
Tuesday, November 20, 2018 - 10:30
Location: 
CSE 305
Tags: