Title: Zero-Shot Activity Recognition with Verb Attribute Induction and Neural Motifs: Scene Graph Parsing with Global Context

Advisors: Yejin Choi and Ali Farhadi

Abstract: In this talk, I will discuss two projects in the intersection of Computer Vision and Natural Language Processing. (1) Zero-shot multimodal learning where the topics of classification are verbs, not objects. We crowdsource verb attributes and build a model to learn them from text (word embeddings and dictionary definitions). We also use these verb attributes alongside word embeddings for activity recognition in images. (2) Scene graph generation: building a semantic graph of an image where the nodes are objects and the edges are pairwise relationships. We find that the Visual Genome dataset has many repeating structures, or motifs, and build a state-of-the-art model that captures these.

Place: 
CSE 303
When: 
Wednesday, January 24, 2018 - 13:30 to Friday, March 29, 2024 - 05:01