Abstract: In this talk, I will discuss my group's recent work on using logically-directed textual entailment knowledge to improve a variety of downstream natural language generation tasks such as video captioning, document summarization, and sentence simplification. First, we employ a many-to-many multi-task learning setup to combine a directed premise-to-entailment generation task (as well as a video-to-video completion task) with the given downstream generation task of multimodal video captioning (where the caption is entailed by the video), achieving significant improvements over the state-of-the-art on multiple datasets and metrics. Next, we employ multiple novel multi-task learning setups to achieve state-of-the-art results on the tasks of automatic document summarization and sentence simplification. Secondly, we optimize for entailment classification scores as sentence-level metric rewards in a reinforcement learning style setup (via annealed policy gradient methods). Our novel multi-reward functions correct the standard phrase-matching metric rewards to only allow for logically-implied partial matches and avoid contradictions, hence substantially improving the conditioned generation results for both video captioning and document summarization. Finally, I will discuss some of the other recent work in our group on image, video, and action based language generation and interaction.
Bio: Dr. Mohit Bansal is an assistant professor in the Computer Science department at University of North Carolina (UNC) Chapel Hill. Prior to this, he was a research assistant professor (3-year endowed position) at TTI-Chicago. He received his PhD from UC Berkeley in 2013 (where he was advised by Dan Klein) and his BTech from IIT Kanpur in 2008. His research interests are in statistical natural language processing and machine learning, with a particular interest in multimodal, grounded, and embodied semantics (i.e., language with vision and speech, for robotics), human-like language generation and Q&A/dialogue, and interpretable and structured deep learning. He is a recipient of the 2017 DARPA Young Faculty Award, 2017 ACL Outstanding Paper Award, 2014 ACL Best Paper Award Honorable Mention, 2016 and 2014 Google Faculty Research Awards, 2016 Bloomberg Data Science Award, 2017 Facebook ParlAI Award, and 2014 IBM Faculty Award. Webpage: http://www.cs.unc.edu/~mbansal/