Monday, February 8, 2016 - 11:00
Speaker: Roy Schwartz (Hebrew University)
Location: CSE 305
Word embeddings (e.g., word2vec, GloVe) have become increasingly popular in recent years, obtaining state-of-the-art results on numerous semantic tasks. In this talk I will show that in fact, these embeddings are limited in their ability to capture various types of semantic information. I will then present several pattern-based methods (e.g., "X and Y") that greatly alleviate some of these problems, focusing on two word embedding models that replace bag-of-words contexts with pattern contexts. I will show that these models obtain substantial improvements on word similarity tasks, most notably a 20% improvement on verb similarity prediction.
Joint work with Roi Reichart and Ari Rappoport.
Bio: Roy Schwartz is a Ph.D. candidate at the Computer Science department at the Hebrew University, working with Prof. Ari Rappoport. His research focuses on improving vector representations of words and phrases using patterns. Roy received his B.Sc. and M.Sc. degrees (both magna cum laude) from the Computer Science department at the Hebrew University. During his studies, Roy has worked on several NLP tasks, including morphology, dependency parsing, sentiment analysis and authorship attribution, experimenting with both news-wire and social media data. Prior to his studies, Roy worked as a software engineer at Check Point Ltd.
Saturday, January 30, 2016 - 13:53
Speaker: Hoifung Poon (Microsoft Research)
Location: CSE 305
Abstract: Advances in sequencing technology have made available a plethora of panomics data for cancer research, yet the search for disease genes and drug targets remains a formidable challenge. Biological knowledge such as pathways can play an important role in this quest by constraining the search space of complex genetic interactions. The majority of knowledge resides in text such as journal articles, which has been undergoing its own explosive growth, making it mandatory to develop machine reading methods for automating knowledge extraction. In this talk, I will formulate the machine reading task for pathway extraction, review the state of the art and open challenges, and present our Literome project and some initial results in applying the extracted pathways to cancer drug modeling.
Bio: Hoifung Poon is a researcher at Microsoft Research. His research interests lie in advancing machine learning and natural language processing (NLP) to help automate discovery in genomics and precision medicine. His most recent work focuses on scaling semantic parsing to PubMed for extracting biological pathways, and on developing probabilistic methods to incorporate pathways with high-throughput omics data in cancer systems biology. He has received Best Paper Awards in premier NLP and machine learning venues such as the Conference of the North American Chapter of the Association for Computational Linguistics, the Conference of Empirical Methods in Natural Language Processing, and the Conference of Uncertainty in AI.
Thursday, January 28, 2016 - 10:00
Speaker: Regina Barzilay (MIT)
Location: Gates Commons (CSE 691)
Progress on many well-established problems in Natural Language Processing comes from applying generic machine learning techniques to the task at hand. While successful, this perspective omits the hidden simplicity of many tasks or ways that they could be made simpler to solve. In this talk, I will show how simple methods can be effectively applied to core NLP tasks — dependency parsing and information extraction.
Dependency parsing as a structured prediction task is a hard combinatorial problem, typically solved by adapting general optimization methods for parsing. However, we demonstrate that, on average, parsing appears easier than its broader complexity class would suggest, and show that a simple and flexible randomized algorithm outperforms state-of-the-art optimization techniques.
Traditional formulations of information extraction focus on learning extraction patterns from a given document. In contrast, we refocus the effort on finding other sources that contain the information sought but expressed in a form that a basic extractor can “understand”. The final system is implemented in the reinforcement learning framework that combines query reformulation, basic extraction, and answer validation. Empirical performance shows that learning to chase for easy answers yields significant performance gains over traditional extractors.
This is joint work with Tommi Jaakkola, Tao Lei, Karthik Narasimhan and Yuan Zhang.
Bio: Regina Barzilay is a professor in the Department of Electrical Engineering and Computer Science and a member of the Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology. Her research interests are in natural language processing. She is a recipient of various awards including of the NSF Career Award, the MIT Technology Review TR-35 Award, Microsoft Faculty Fellowship and several Best Paper Awards in top NLP conferences. She received her Ph.D. in Computer Science from Columbia University, and spent a year as a postdoc at Cornell University.
Tuesday, January 26, 2016 - 10:00
Speaker: Yoav Goldberg (Bar Ilan University)
Location: CSE 305
While deep learning methods in NLP are arguably overhyped, recurrent neural networks (RNNs), and in particular LSTM networks, emerge as very capable learners for sequential data. After explaining what they are and why they are cool, I will describe some recent work in which we use LSTMs as a building block: learning a shared representation in a multi-task setting; learning feature representations for structure prediction; and an extension from sequences to tree structures. I will conclude with a brief comment on representation of language structures. Pretty much everything in this talk is unpublished.
Bio: Yoav Goldberg is a senior lecturer in Computer Science at Bar Ilan University, Israel, working on natural language processing. Prior to that he was a research scientist at Google. Before the deep learning revolution he used to work on syntactic parsing and structured prediction. He still does, but now he uses some new shiny tools which he is trying to understand and refine.
Tuesday, January 19, 2016 - 15:30
Speaker: Eric Ringger (Facebook)
Location: EEB 105
Monday, December 7, 2015 - 11:00
Speaker: Cristian Danescu-Niculescu-Mizil (Cornell University)
Location: CSE 305 (please note the change from the original room!)
Abstract: More and more of life is now manifested online, and many of
the digital traces that are left by human activity are in
natural-language format. In this talk I will show how exploiting these
resources under a computational framework can bring a new understanding
of online social dynamics; I will be discussing two of my efforts in
The first project explores the relation between users and their
community, as revealed by patterns of linguistic change. I will show
that users follow a determined life-cycle with respect to their
susceptibility to adopt new community norms, and how this insight can be
harnessed to predict how long a user will stay active in the community.
The second project proposes a computational framework for identifying
and characterizing politeness, a central force shaping our communication
behavior. I will show how this framework can be used to study the
social aspects of politeness, revealing new interactions with social
status and community membership.
This talk includes joint work with Dan Jurafsky, Jure Leskovec,
Christopher Potts, Moritz Sudhof and Robert West.
Tuesday, December 1, 2015 - 12:00
Speaker: Waleed Ammar (Carnegie Mellon University)
Location: CSE 305
Abstract: Most natural language processing tools are only available for a few languages for which labeled data are readily available. In this talk, I'll present two approaches for training structured predictors for languages with minimal resources. The first, CRF autoencoders, is a family of feature-rich probabilistic models which enable us to efficiently learn from unlabeled examples. We use instantiations of CRF autoencoders to model POS tagging, word alignment, language identification, and selectional preferences. Second, we introduce a neural-network language-universal model for dependency parsing, which learns simultaneously from labeled data in a variety of high-resource languages and can be readily used to parse text in more languages.
Tuesday, November 17, 2015 - 15:30
Speaker: Dan Jurafsky (Stanford University)
Location: EEB 105
[This talk is part of the CSE Distinguished Lecture Series.]
Natural language processing has an important role to play in
discovering the social meanings behind the language we use, helping
answer central questions in the behavioral and social sciences. In
this talk I apply computational linguistics to help extract and
understand social meaning from different texts with the goal of
understanding aspects of innovation. I'll discuss the way economic,
social, and psychological variables are reflected in the language
we use to talk about food, introduce the "ketchup theory of innovation"
on the crucial role that interdisciplinarity plays in the history
of innovation and how it can be discovered via language, and show
what the language of online communities can tell us about the nature
of linguistic innovation across the lifespan.
Dan Jurafsky is professor and chair of linguistics and professor
of computer science at Stanford University. His research focus is
natural language processing, with special interests in the automatic
extraction of meaning from speech and text in English and Chinese,
and on applying computational linguistics to the behavioral and social sciences.
Dan is a 2002 MacArthur Fellowship recipient. His latest book, The
Language of Food: A Linguist Reads the Menu, was a finalist for the
2015 James Beard Award. He is currently working on the 3rd edition
of his co-authored textbook, Speech and Language Processing. His recipe
for apricot strudel is widely acclaimed but will not be discussed at this talk.
Thursday, November 12, 2015 - 09:00
Speaker: Jia Xu, Chinese Academy of Sciences
Location: CSE 305
Bagging [Breiman, 96] and its variants is one of the most popular methods in aggregating classifiers and regressors. Its original analysis assumes that the bootstraps are built from an unlimited, independent source of samples. In the real world this analysis fails because there is a limited number of training samples.
We analyze the effect of intersections between bootstraps to train different base predictors, which shows that the real-world bagging behaves very differently than its ideal analog [Breiman, 96]. Most importantly, we provide an alternative subsampling method called design-bagging based on a new construction of combinatorial designs. We prove that this is universally better than bagging. Our analytical results are backed up by experiments on general classification and regression settings, and significantly improved all machine translation systems we used in the NIST-15 C-E competition.
Jia Xu is an associate professor at ICT/CAS, after being an assistant professor in Tsinghua University and a senior researcher at DFKI lecturing at Saarland University in Germany. She worked at IBM Watson and MSR Redmond during her Ph.D. advised by Hermann Ney at RWTH-Aachen University. Her current research interests are in Machine Learning with a focus towards highly competitive machine translation systems, where she led and participated in teams winning first place in WMT-11, TC-Star -05-07 and NIST-08. In NIST-15 she led one more team that won 4th place, which is the 1st among academic institutions.
Wednesday, October 28, 2015 - 11:00
Speaker: Charles Sutton, University of Edinburgh
Location: CSE 403
Billions of lines of source code have been written, many of which are freely available on the Internet. This code contains a wealth of implicit knowledge about how to write software that is easy to read, avoids common bugs, and uses popular libraries effectively.
We want to extract this implicit knowledge by analyzing source code text. To do this, we employ the same tools from machine learning and natural language processing that have been applied successfully to natural language text. After all, source code is also a means of human communication.
We present three new software engineering tools inspired by this insight:
* Naturalize, a system that learns local coding conventions.
It proposes revisions to names and to formatting so as to make code more consistent. A version that uses word embeddings has shown promise
toward naming methods and classes.
* Data mining methods have been widely applied to summarize the patterns about how programmers invoke libraries and APIs. We present a new method for mining market basket data, based on a simple generative probabilistic model, that resolves fundamental statistical pathologies that lurk in popular current data mining techniques.
* HAGGIS, a system that learns local recurring syntactic patterns, which we call idioms. HAGGIS accomplishes this using a nonparametric Bayesian tree substitution grammar, and is delicious with whisky sauce.
Charles Sutton is a Reader (equivalent to Associate Professor: http://bit.ly/1W9UhqT
) at the University of Edinburgh. He is interested in a broad range of applications of probabilistic machine learning, including NLP, analysis of computer systems, software engineering, sustainable energy, and exploratory data analysis. Dr Sutton completed his PhD at the University of Massachusetts Amherst, working with Andrew McCallum. He did postdoctoral research at the University of California Berkeley, working with Michael I Jordan. He is Deputy Director of the EPSRC Centre for Doctoral Training in Data Science at the University of Edinburgh.