Machine Learning Seminars

Sponsored by

Title: Recovery of Simultaneously Structured Models by Convex Optimization
Speaker: Maryam Fazel
When: Tuesday, October 16, 2012 - 12:30
Location: Room 305
The topic of deriving a structured model from a small number of linear observations by solving a convex optimization problem has been well-studied in recent years. Examples include the recovery of sparse or group-sparse vectors (which gave rise to the area of Compressed Sensing), low-rank matrices (arising in collaborative filtering and system identification), and the sum of sparse and low-rank matrices (arising in PCA with sparse outliers, graphical models). In many applications in signal processing and machine learning, the model of interest is known to be structured in several ways at the same time, for example, a matrix that is simultaneously sparse and low-rank. An application in signal processing is the classic sparse phase retrieval problem, where the goal is to recover a sparse signal from phaseless (magnitude-only) measurements. In machine learning, the problem comes up when combining several regularizers that each promote a certain desired structure. In this work, we address the questions: what convex relaxation should we use for the recovery of a ``simultaneously structured'' model? And how many measurements are needed generically? Often penalties (norms) that promote each structure are known (e.g. l1 norm for sparsity, nuclear norm for matrix rank), so it is reasonable to minimize a combination of these norms. We show that, surprisingly, if we use as objective function any convex joint function of the individual norms, we can do no better than an algorithm that exploits only one of the several structures. We then specialize our result to the case of simultaneously sparse and low-rank matrices, and present numerical simulations that support the theoretical bounds on required number of observations.

 

Title: Large-Scale Machine Learning
Speaker: Carlos Guestrin
When: Tuesday, October 2, 2012 - 12:30
Location: Room 305
Today, machine learning (ML) methods play a central role in industry and science. The growth of the Web and improvements in sensor data collection technology have been rapidly increasing the magnitude and complexity of the ML tasks we must solve. This growth is driving the need for scalable, parallel ML algorithms that can handle "Big Data." In this talk, I'll first present some recent advances in large-scale algorithms for tackling such huge problems. Unfortunately, implementing efficient parallel ML algorithms is challenging. Existing high-level parallel abstractions such as MapReduce and Pregel are insufficiently expressive to achieve the desired performance, while low-level tools such as MPI are difficult to use, leaving ML experts repeatedly solving the same design challenges. In this talk, I will also describe the GraphLab framework, which naturally expresses asynchronous, dynamic graph computations that are key for state-of-the-art ML algorithms. When these algorithms are expressed in our higher-level abstraction, GraphLab will effectively address many of the underlying parallelism challenges, including data distribution, optimized communication, and guaranteeing sequential consistency, a property that is surprisingly important for many ML algorithms. On a variety of large-scale tasks, GraphLab provides 20-100x performance improvements over Hadoop. In recent months, GraphLab has received thousands of downloads, and is being actively used by a number of startups, companies, research labs and universities. This talk represents joint work with Yucheng Low, Joey Gonzalez, Aapo Kyrola, Jay Gu, Danny Bickson, and Joseph Bradley.

 

Title: Learning Hierarchical Representations for Recognition
Speaker: Liefeng Bo, Dept. Computer Science & Eng., Univ. Washington
When: Tuesday, May 22, 2012 - 12:30
Location: Gates Commons

Humans are able to recognize objects despite significant
variations in their appearance due to changing viewpoints, scales,
lighting conditions and deformations. This ability fundamentally relies
on robust representations/features of the physical world. In this talk,
I will introduce our recent work on representation/feature learning:
hierarchical kernel descriptor and hierarchical matching pursuit.
Hierarchical kernel descriptor is a kernel based feature learning
approach that uses efficient matching kernels to build feature
hierarchies. Hierarchical matching pursuit is a multi-layer sparse
coding approach that learns multi-level features from raw sensor data by
recursively applying matching pursuit coding followed by spatial pyramid
pooling. Our models outperform the state-of-the-art, often by a large
margin, on more than fifteen popular computer vision benchmarks for many
types of recognition tasks including RGB-(D) object recognition,
fine-grained object recognition, digit recognition, face recognition,
material recognition, scene recognition, and RGB-(D) scene labeling. In
addition, I will discuss the connections and differences between our
models, and bag-of-words models, deformable parts model and deep
networks/feature learning in terms of the underlying architecture.

 

Title: Linguistic Structure Prediction with AD³
Speaker: Noah Smith, Carnegie Mellon University
When: Tuesday, May 8, 2012 - 12:30
Location: Gates Commons

In this talk, I will present AD³ (Alternating Directions Dual
Decomposition), an algorithm for approximate MAP inference in loopy
graphical models with discrete random variables, including structured
prediction problems. AD³ is simple to implement and well-suited to
problems with hard constraints expressed in first-order logic. It often
finds the exact MAP solution, giving a certificate when it does; when it
doesn't, it can be embedded within an exact branch and bound technique.
I'll show experimental results on two natural language processing tasks,
dependency parsing and frame-semantic parsing. This work was done in
collaboration with André Martins, Dipanjan Das, Pedro Aguiar, Mário
Figueiredo, and Eric Xing.

 

Title: Decision making in the primate brain: A model based on POMDPs and reinforcement learning
Speaker: Raj Rao, University of Washington
When: Tuesday, April 24, 2012 - 12:30
Location: Room 305

How does the brain learn to make decisions based on noisy sensory
information and incomplete knowledge of the world? In this talk, I
will sketch a neural model of action selection and decision making
based on the general framework of partially observable Markov decision
processes (POMDPs). The model postulates that actions are selected so
as to maximize expected cumulative reward, where rewards can be
external (e.g., food) or internal (e.g., penalty for delay). Action
selection is based on the posterior distribution over states (the
"belief" state) computed using Bayesian inference. A reinforcement
learning algorithm known as temporal difference (TD) learning
maximizes the expected reward. I will describe how such a model
provides a unified framework for explaining recent experimental
results on decision making in the primate brain. The resulting neural
architecture posits an active role for the neocortex in Bayesian
inference while ascribing a role to the basal ganglia in value
computation and action selection. The model suggests an important role
for interactions between the neocortex and the basal ganglia in
learning the mapping between probabilistic sensory representations and
actions that maximize rewards.
(Joint work with Yanping Huang and Abe Friesen)

 

Title: Spectral Clustering and Higher-Order Cheeger Inequalities
Speaker: James Lee, Dept. Computer Science & Eng., Univ. Washington
When: Tuesday, April 10, 2012 - 12:30
Location: Gates Commons

I will try to tell a story of spectral clustering, from
heuristics, to experiments, to a new algorithm and its formal analysis.
In an interesting twist, the same tools used to analyze spectral
partitioning yield new ways to mine the structure of noisy systems of
linear equations modulo a prime. This, in turn, allows us to mount a
spectral attack on one of the most prominent open problems in complexity
theory. [This is based partly on joint work with Shayan Oveis-Gharan and
Luca Trevisan.]

 

Title: Gaussian Processes for Approximating Complex Models
Speaker: Murali Haran, Pennsylvania State University
When: Tuesday, March 6, 2012 - 12:30
Location: EEB 037

This is a tutorial on the use of Gaussian process to approximate complex models often used in modeling phenomena like climate, disease dynamics, etc. The tutorial will begin with an introduction to Gaussian processes followed by a discussion of how Gaussian processes can be used for both computer model emulation (approximation) and calibration.

 

Title: Causality
Speaker: Thomas Richardon, University of Washington
When: Tuesday, February 28, 2012 - 12:30
Location: EEB 037

As these talks are intended as overviews, time permitting, I plan to give two mini-tutorials: The first will be on the idea of counterfactuals/potential outcomes. Basically answering the question: how does data obtained from a simple randomized experiment differ from that obtained from an observational study, and how does that weaken inferences that can be obtained. The second will require assume a little bit of background on Bayesian networks and will answer the question: if we obtain data from a subset of the variables in a causal Bayesian network, which causal effects are identified and how can they be computed efficiently.

 

Title: Counting and Sampling Solutions of Combinatorial Problems
Speaker: Ashish Sabharwal, IBM T,J, Watson
When: Tuesday, February 21, 2012 - 12:30
Location: EEB 037

This tutorial presents a survey of algorithms that are used for counting and sampling of SAT problems.

 

Title: Knowledge Extraction for Biomedical Text
Speaker: Hoifung Poon, Microsoft Research
When: Tuesday, February 14, 2012 - 12:30
Location:

This tutorial summarizes the literature on knowledge extraction in scientific domains, such as biomedical texts.