Robotics Colloquium

The Robotics Colloquium features talks by invited and local researchers on all aspects of robotics, including control, perception, machine learning, mechanical design and interaction. The colloquium is held Fridays between 1:30-2:30pm. Special seminars outside this schedule are indicated with * below. Check schedule below for location. Refreshments are served.

If you would like to give a talk in upcoming Robotics Colloquia, please contact Maya Cakmak (mcakmakatcs). If you would like to get regular email announcements and reminders about the robotics colloquium speakers, please sign up to the Robotics@UW mailing list.

Spring 2018 Organizers: Dieter Fox, Maya Cakmak, Siddhartha S. Srinivasa, Sam Burden
CSE 203
Michael Beetz
University Bremen, IAI
Everyday Activity Science and Engineering (EASE) abstract
No talk
CSE 305
David Rollinson
Hebi Robotics
Building a Force-Controlled Actuator (Company) abstract
CSE 305
Michael A. Goodrich
Brigham Young University
Toward Human Interaction with Bio-Inspired Robot Swarms abstract
CSE 305
Dmitry Berenson
University of Michigan
What Matters for Deformable Object Manipulation abstract
CSE 305
Jeff Mahler
University of California Berkeley
The Dexterity Network: Deep Learning to Plan Robust Robot Grasps using Datasets of Synthetic Point Clouds, Analytic Grasp Metrics, and 3D Object Models abstract
CSE 305
Karen Liu
Georgia Tech
Towards a Generative Model of Natural Motion abstract
CSE 305
Jung-Su Ha
TBA abstract
CSE 305
Guy Hoffman
Cornell University
Designing Robots for Fluent Collaboration and Companionship abstract
CSE 305
Devin Balkcom
Dartmouth College
TBA abstract
Dan Koditschek
TBA abstract
Winter 2018 Organizers: Dieter Fox, Maya Cakmak, Siddhartha S. Srinivasa, Kat Steele
No talk
CSE 691
(10 AM)
Allison Okamura
Stanford University
Let’s be Flexible: Soft Haptics and Soft Robotics abstract
HUB 250
(1 PM)
David Reinkensmeyer
UC Irvine
Robotic-assisted movement training after stroke: Why does it work and how can it be made to work better? abstract
CSE 305
Stefanos Nikolaidis
CMU/University of Washington
Mathematical Models of Adaptation in Human-Robot Collaboration abstract
No talk
CSE 305
(3 PM)
Peter Trautman
1-Dimensional Joint Probability Distributions: the Duality of Shared Control and Crowd Navigation Solutions abstract
EEB 037
Amir Rubin
SLAM and 3D-reconstruction for Real World Use Cases abstract
CSE 305
Emel Demircan
California State University
Human Movement Understanding abstract
CSE 305
Kris Hauser
Duke University
The Space of Spaces: Understanding the Structure Between Motion Planning Problems abstract
CSE 305
Marco Pavone
Stanford University
Planning and Decision Making for Autonomous Spacecraft and Space Robots abstract
CSE 305
Yu Xiang
Perceiving the 3D World from Images and Videos abstract
Autumn 2017 Organizers: Dieter Fox, Maya Cakmak, Siddhartha S. Srinivasa, Kat Steele, Sam Burden
MEB 238
3:30 PM
Michael Tolley
University of California San Diego
ME colloquium: Soft Robotics abstract
CSE 305
Geoffrey A. Hollinger
Oregon State University
Marine Robotics: Planning, Decision Making, and Learning abstract
10/13/2017 DUB retreat
No talk
CSE 305
Byron Boots
Georgia Institute of Technology
Learning Perception and Control for Agile Off-Road Autonomous Driving abstract
CSE 691
Tucker Hermans
University of Utah
Learning and Planning for Autonomous Multi-fingered Robot Manipulation abstract
CSE 305
Joydeep Biswas
University of Massachusetts Amherst
Deploying Autonomous Service Mobile Robots, And Keeping Them Autonomous abstract
11/10/2017 No talk
CSE 691
Oren Salzman
Carnegie Mellon University
The Provable Virtue of Laziness in Motion Planning abstract
11/24/2017 Thanksgiving
No talk
12/01/2017 No talk
CSE 305
Yigit Menguc
Oregon State University
Material Robotics: Soft active materials, bioinspired mechanisms, and additive manufacturing abstract
Spring 2017 Organizers: Aaron Walsman, Sam Burden, Maya Cakmak, Dieter Fox
CSE 305
Richard Vaughan
Simon Fraser University
Simple, Robust Interaction Between Humans and Teams of Robots abstract
CSE 691
Oussama Khatib
Ocean One: A Robotic Avatar for Oceanic Discovery abstract
CSE 305
Debadeepta Dey
Microsoft Research
Learning via Interaction for Machine Perception and Control abstract
CSE 303 (11 AM)
Eric Eaton
University of Pennsylvania
Efficient Lifelong Machine Learning: an Online Multi-Task Learning Perspective abstract
CSE 305
Katsu Ikeuchi
Microsoft Research
e-Intangible Heritage, from Dancing robots to Cyber Humanities abstract
CSE 691
Henrik Christensen
UC San Diego
Object Based Mapping abstract
CSE 305
Alberto Rodriguez
Reactive Robotic Manipulation abstract
CSE 305
Charlie Kemp
Georgia Tech
Mobile Manipulators for Intelligent Physical Assistance abstract
CSE 305
Karol Hausman
University of Southern California
Rethinking Perception-Action Loops abstract
CSE 305
Silvia Ferrari
Cornell University
Neuromorphic Planning and Control of Insect-scale Robots abstract
No Colloquium (ICRA)    
Winter 2017 Organizers: Sam Burden, Maya Cakmak, Dieter Fox
01/06/2017 No colloquium    
Matt Rueben
Oregon State University
Privacy Sensitive Robotics abstract
Frontiers of Science
Savery Hall 260
01/27/2017 Ross Hatton
Oregon State University
Snakes & Spiders, Robots & Geometry abstract
02/03/2017 Avik De
University of Pennsylvania
Anchored Behaviors from Template Compositions abstract
02/10/2017 No talk    
02/17/2017 Sonia Chernova
Georgia Institute of Technology
Reliable Robot Autonomy through Learning and Interaction abstract
02/24/2017 No talk    
03/03/2017 No talk    
03/10/2017 No talk    
Fall 2016 Organizers: Sam Burden, Maya Cakmak, Dieter Fox, Sawyer Fuller
CSE 305
David Remy
University of Michigan
Gaits and Natural Dynamics in Robotic Legged Locomotion abstract
IROS 2016 and DUB retreat
No talk
10/21/2016 Industry Affiliates Week
Check out talks and posters by robotics students
CSE 305
Emo Todorov
University of Washington
Goal-directed Dynamics abstract
CSE 305
Sean Andrist
Microsoft Research
Gaze Mechanisms for Situated Interaction with Embodied Agents abstract
Veterans day
No talk
CSE 305
Nick Roy
Planning to Fly (and Drive) Aggressively abstract
No talk
CSE 305
Shai Revzen
University of Michigan
Seeking simple models for multilegged locomotion: hybrid oscillators, rapid manufacturing, and slippage abstract
CSE 305
Ashis Banerjee
University of Washington
Toward Real-Time Motion Planning and Control of Optically Actuated Micro-Robots abstract
Spring 2016 Organizers: Justin Huang, Leah Perlmutter, Dieter Fox, and Maya Cakmak
CSE 305
Tomás Lozano-Pérez
Integrated task and motion planning in belief space abstract
CSE 305
Henny Admoni
CMU / Yale
Recognizing Human Intent for Assistive Robotics abstract
CSE 305
Wolfram Burgard
University of Frieburg
Deep Learning for Robot Navigation and Perception abstract
CSE 305
Travis Deyle
Cobalt Robotics
RFID-Enhanced Robots Enable New Applications in Healthcare, Asset Tracking, and Remote Sensing abstract
CSE 305
Brian Scassellati
Robots That Teach abstract
CSE 305
Sarah Elliott, Mohammad Haghighipanah, Vikash Kumar, Yangming Li, Muneaki Miyasaka, Leah Perlmutter, Luis Puig, and Yuyin Sun
University of Washington
ICRA 2016 Practice Talks abstract
CSE 305
Sidd Srinivasa
Physics-based Manipulation abstract
CSE 403
3:30 pm
Ashish Kapoor
Microsoft Research
Planetary Scale Swarm Sensing, Planning and Control for Weather Prediction abstract
Winter 2016 Organizers: Kendall Lowrey, Patrick Lancaster, and Dieter Fox
CSE 305
Daniel Butler
Model-based Reinforcement Learning with Parametrized Physical Models and Optimism-Driven Exploration abstract
CSE 305
James Youngquist
DeepMPC: Learning Deep Latent Features for Model Predictive Control abstract
CSE 305
Justin Huang
Place Recognition with ConvNet Landmarks: Viewpoint-Robust, Condition-Robust, Training-Free abstract
EEB 303
Daniel Gordon
Deep Neural Decision Forests abstract
CSE 305
Harley Montgomery
End-to-End Training of Deep Visuomotor Policies abstract
CSE 305
Aaron Walsman
Mastering the game of Go with deep neural networks and tree search abstract
CSE 305
Zachary Nehrenberg
Real-Time Trajectory Generation for Quadrocopters abstract
CSE 305
Patrick Lancaster
Towards Learning Hierarchical Skills for Multi-Phase Manipulation Tasks abstract
CSE 305
Tanner Schmidt
Pose Estimation of Kinematic Chain Instances via Object Coordinate Regression abstract
CSE 305
Kendall Lowrey
Combining the benefits of function approximation and trajectory optimization abstract
CSE 305
Vladimir Korukov
Information-Theoretic Planning with Trajectory Optimization for Dense 3D Mapping abstract
Fall 2015 Organizers: Tanner Schmidt and Dieter Fox
CSE 305
Dan Bohus
Microsoft Research
Physically Situated Dialog: Opportunities and Challenges abstract
CSE 305
Sawyer Fuller
Aerial autonomy at insect scale: What flying insects can tell us about robotics and vice versa abstract
Kane Hall 110
Russ Tedrake
From Polynomials to Humanoid Robots
Part of the MathAcrossCampus Colloquium Series
CSE 305
Frank Dellaert
Factor Graphs for Flexible Inference in Robotics and Vision abstract
CSE 305
Student Research Lightning Talks    
CSE 305
Louis-Philippe Morency
Modeling Human Communication Dynamics abstract
CSE 305
Tom Whelan
Oculus Research
Real-time dense methods for 3D perception abstract
CSE 305
No talk
CSE 305
Seth Hutchinson
Robust Distributed Control Policies for Multi-Robot Systems abstract
CSE 305
Dmitry Berenson
Toward General-Purpose Manipulation of Deformable Objects abstract
Spring 2015 Organizers: Connor Schenck, Maya Cakmak, Dieter Fox
CSE 303
Neil Lebeck and Natalie Brace
Multirotor Aerial Vehicles: Modeling, Estimation, and Control of Quadrotor abstract
CSE 303
Peter Henry
LSD-SLAM: Large-Scale Direct Monocular SLAM abstract
CSE 303
Dan Butler
Probabilistic Segmentation and Targeted Exploration of Objects in Cluttered Environments abstract
CSE 303
Marc Deisenroth
Imperial College, London
Statistical Machine Learning for Autonomous Systems and Robots abstract
CSE 303
Arunkumar Byravan and Kendall Lowrey
Reinforcement Learning in Robotics: A Survey abstract
05/22/15   ICRA practice talks. abstracts
05/29/15 No Colloquium Colloquium cancelled for ICRA 2015.  
CSE 303
Jim Youngquist
A Strictly Convex Hull for Computing Proximity Distances With Continuous Gradients abstract
Winter 2015 Organizers: Connor Schenck, Maya Cakmak, Dieter Fox
CSE 305
Mike Chung
Accelerating Imitation Learning through Crowdsourcing abstract
  Tanner Schmidt
Dense Articulated Real-Time Tracking abstract
CSE 305
Discussion Amazon Picking Challenge <
01/30/15 No colloquium    
CSE 305
Joseph Xu
Design and Control of an Anthropomorphic Robotic Hand: Learning Advantages From the Human Body & Brain abstract
  Vikash Kumar
Dimensionality Augmentation: A tool towards synthesizing complex and expressive behaviors abstract
CSE 305
Sofia Alexandrova
RoboFlow: A Flow-based Visual Programming Language for Mobile Manipulation Tasks abstract
CSE 305
Igor Mordatch
Synthesis of Interactive Control for Diverse Complex Characters with Neural Networks abstract
CSE 305
Richard Newcombe
DynamicFusion: Reconstruction and Tracking of Non-Rigid Scenes in Real-Time abstract
CSE 691
Aaron Steinfeld
Carnegie Mellon University
Understanding and Creating Appropriate Robot Behavior abstract
CSE 305
Luis Puig
Overview of Omnidirectional Vision abstract
Autumn 2014 Organizers: Vikash Kumar, Maya Cakmak, Dieter Fox
CSE 305
Danny Kaufman
Adobe Creative Technologies Lab, Seattle
Geometric Algorithms for Computing Frictionally Contacting Systems abstract
EE 037
Dubi Katz & Michael Abrash
Oculus VR
VR, the future, and you abstract
CSE 503
Kira Mourao
PostDoc, Institute for Language, Cognition and Computation, University of Edinburgh
What happens if I push this button? Learning planning operators from experience abstract
CSE 305
Sam Burden
PostDoc, University of California, Berkeley
Hybrid Models for Dynamic and Dexterous Robots abstract
*10/29/14 (Wed)
HUB 250
Bilge Mutlu
University of Wisconsin, Madison
Human-Centered Principles and Methods for Designing Robotic Technologies
(Joint with DUB seminar, lunch will be served at 12:00)
CSE 305
Sergey Levine
PostDoc, University of California, Berkeley
Learning to Move: Machine Learning for Robotics and Animation abstract
11/07/14 No talk    
11/14/14 Sachin Patil
PostDoc, University of California, Berkeley
Coping with Uncertainty in Robotic Navigation and Manipulation abstract
Gates Commons
HRI Mini Symposium
HRI 2015 Program committee members
11/28/14 No talk, Thanksgiving Break    
CSE 305
Marianna Madry
Royal Institute of Technology (KTH), Sweden
Representing Objects in Robotics from Visual, Depth and Tactile Sensing abstract
*12/18/14 (Thu)
CSE 305
Scott Niekum
Carnegie Mellon University
Structure Discovery in Robotics with Demonstrations and Active Learning abstract
Winter 2014, Organizer: Maya Cakmak
CSE 403
Byron Boots
Learning Better Models of Dynamical Systems abstract
CSE 403
Julie Shah
Integrating Robots into Team-Oriented Environments abstract
CSE 403
Ryan Calo
UW Law
Robotics & The New Cyberlaw abstract
CSE 403
James McLurkin
Rice University
Distributed Algorithms for Robot Recovery, Multi-Robot Triangulation, and Advanced Low-Cost Robots abstract
02/14/14   Cancelled  
CSE 403
Mihai Jalobeanu
Microsoft Research
Towards ubiquitous robots abstract
CSE 403
Cynthia Matuszek
Talking to Robots: Learning to Ground Human Language in Perception and Execution abstract
03/07/14   Cancelled  
CSE 403
Peter H. Kahn, Jr.
UW Psychology
Social and Moral Relationships with Robots abstract
Gates Commons
Gur Kimchi
Amazon Prime Air abstract
Autumn 2013, Organizer: Maya Cakmak
MGH 241
Ashutosh Saxena
Cornell University
How should a robot perceive the world?
(Joint with Machine Learning)
10/18/13   UW/MSR Machine Learning day  
CSE 403
Kat Steele
University of Washington, Mechanical Engineering
Strategies for understanding and improving movement disorders abstract
CSE 403
Maya Cakmak
University of Washington, CSE
Towards seamless human-robot hand-overs abstract
*11/07/13 (Thu)
CSE 403
Ross A. Knepper
Autonomous Assembly In a Human World abstract
CSE 403
Brian Ziebart
University of Illinois, Chicago
Beyond Conditionals: Structured Prediction for Interacting Processes
(Lunch will be served)
Gates Commons
Jenay Beer
University of South Carolina
Considerations for Designing Assistive Robotics to Promote Aging-in-Place abstract
CSE 403
Dinei Florencio
Microsoft Research
Navigation for telepresence robots and some thoughts on robot learning abstract
CSE 403
Andrzej Pronobis
University of Washington, CSE
Semantic Knowledge in Mobile Robotics: Perception, Reasoning, Communication and Actions abstract
Gates Commons
Steve Cousins
Savioke, Inc. & Willow Garage, Inc.
It's Time for Service Robots
(Joint with CSNE)
Spring 2013, Organizers: Cynthia Matuszek, Dieter Fox
04/5/13 Dieter Fox
Cynthia Matuszek
PechaKucha 20x20 for Robotics abstract
04/12/13 No talk    
04/19/13 Robotics Students & Staff PechaKucha-style Robotics Research Overviews abstract
04/26/13 Pete Wurman
Special Wednesday Colloquium, CSE 203
Coordinating Hundreds of Autonomous Vehicles in Warehouses
04/26/13 Matt Mason Learning to Use Simple Hands abstract
05/03/13 Nadia Shouraboura Canceled  
05/10/13 No talk (ICRA)
05/17/13 Tom Daniel Control and Dynamics of Animal Flight: Reverse Engineering Nature's Robots abstract
05/24/13 Katherine Kuchenbecker The Value of Tactile Sensations in Haptics and Robotics abstract
05/31/13 Pieter Abbeel Machine Learning and Optimization for Robotics abstract
06/07/13 Nick Roy Canceled  
Winter 2013, Organizer: Dieter Fox
01/18/13 Robotics and State
Estimation Lab
Overview of RSE Lab Research
01/25/13 Joshua Smith Robotics Research in the Sensor Systems Group abstract
02/01/13 no talk    
02/08/13 Gaurav Sukhatme Persistent Autonomy at Sea abstract
02/15/13 Jiri Najemnik Sequence Optimization in Engineering, Artificial Intelligence and Biology abstract
02/22/13 no talk    
03/01/13 Richard Newcombe Beyond Point Clouds: Adventures in Real-time Dense SLAM abstract
03/08/13 Tom Erez Model-Based Optimization for Intelligent Robot Control abstract
03/15/13 Byron Boots Spectral Approaches to Learning Dynamical Systems abstract
Spring 2012, Organizer: Dieter Fox
3/30/12 Andrea Thomaz Designing Learning Interactions for Robots abstract
4/6/12 Javier Movellan Towards a New Science of Learning abstract
4/13/12 Emanuel Todorov Automatic Synthesis of Complex Behaviors with Optimal Control abstract
4/20/12 Andrew Barto Autonomous Robot Acquisition of Transferable Skills abstract
4/27/12 Dieter Fox Grounding Natural Language in Robot Control Systems abstract
5/4/12 Allison Okamura Robot-Assisted Needle Steering abstract
5/11/12 Blake Hannaford Click the Scalpel -- Better Patient Outcomes by Advancing Robotics in Surgery abstract
5/18/12 no talk  
5/25/12 Malcolm MacIver Robotic Electrolocation abstract
6/1/12 Drew Bagnell Imitation Learning, Inverse Optimal Control and Purposeful Prediction abstract
03/23/2018 Michael Beetz
University Bremen, IAI
Everyday Activity Science and Engineering (EASE)

Recently we have witnessed the first robotic agents performing everyday manipulation activities such as loading a dishwasher and setting a table. While these agents successfully accomplish specific instances of these tasks, they only perform them within the narrow range of conditions for which they have been carefully designed. They are still far from achieving the human ability to autonomously perform a wide range of everyday tasks reliably in a wide range of contexts. In other words, they are far from mastering everyday activities. Mastering everyday activities is an important step for robots to become the competent (co-)workers, assistants, and companions who are widely considered a necessity for dealing with the enormous challenges our aging society is facing.

We propose Everyday Activity Science and Engineering (EASE), a fundamental research endeavour to investigate the cognitive information processing principles employed by humans to master everyday activities and to transfer the obtained insights to models for autonomous control of robotic agents. The aim of EASE is to boost the robustness, efficiency, and flexibility of various information processing subtasks necessary to master everyday activities by uncovering and exploiting the structures within these tasks.

Everyday activities are by definition mundane, mostly stereotypical, and performed regularly. The core research hypothesis of EASE is that robots can achieve mastery by exploiting the nature of everyday activities. We intend to investigate this hypothesis by focusing on two core principles: The first principle is narrative-enabled episodic memories (NEEMs), which are data structures that enable robotic agents to draw knowledge from a large body of observations, experiences, or descriptions of activities. The NEEMs are used to find representations that can exploit the structure of activities by transferring tasks into problem spaces that are computationally easier to handle than the original spaces. These representations are termed pragmatic everyday activity manifolds (PEAMs), analogous to the concept of manifolds as low-dimensional local representations in mathematics. The exploitation of PEAMs should enable agents to achieve the desired task performance while preserving computational feasibility.

The vision behind EASE is a cognition-enabled robot capable of performing human-scale everyday manipulation tasks in the open world based on high-level instructions and mastering them.

Speaker’s Bio: Michael Beetz is a professor for Computer Science at the Faculty for Mathematics & Informatics of the University Bremen and head of the Institute for Artificial Intelligence (IAI). IAI investigates AI- based control methods for robotic agents, with a focus on human-scale everyday manipulation tasks. With his openEASE, a web-based knowledge service providing robot and human activity data, Michael Beetz aims at improving interoperability in robotics and lowering the barriers for robot programming. Due to this the IAI group provides most of its results as open-source software, primarily in the ROS software library. Michael Beetz received his diploma degree in Computer Science with distinction from the University of Kaiserslautern. His MSc, MPhil, and PhD degrees were awarded by Yale University in 1993, 1994, and 1996 and his Venia Legendi from the University of Bonn in 2000. Michael Beetz is currently the Coordinator of the collaborative research center EASE – Every-day Activity Science and Engineering and was a member of the steering committee of the European network of excellence in AI planning (PLANET) and coordinating the research area “robot planning''. He is associate editor of the AI Journal. His research interests include plan-based control of robotic agents, knowledge processing and representation for robots, integrated robot learning, and cognitive perception.
04/06/2018 David Rollinson
Hebi Robotics
Building a Force-Controlled Actuator (Company)
Abstract: In 2014, I became one of 5 people to found HEBI Robotics, with the dream of eventually making the task of building custom robots as easy as building with Lego. A few years later we are now 9 people, and our first product, a series of modular force-controlled actuators, is rapidly being adopted for research and development. In this talk I will discuss the technical aspects of developing force-controlled actuators and the software tools for controlling them, why we chose series-elastic actuation, and various challenges that we encountered during development. I will also talk about what it’s like to be an engineer who is increasingly involved with the business aspects of a growing startup.

Speaker’s Bio: Dave Rollinson is a co-founder and mechanical/controls engineer at HEBI Robotics. He lives and works in Pittsburgh, PA. He received a PhD in Robotics from Carnegie Mellon University in 2014, as well as a B.S. in Mechanical Engineering in 2006, also from Carnegie Mellon University. His thesis research focused on the control and design of modular snake robots with focus towards real-world applications like urban search and rescue and industrial inspection. From 2006 to 2009, he worked as a robotics engineer for RedZone Robotics, designing, building, and deploying systems to inspect large diameter sewers in the U.S., Canada, and Singapore. In 2006, he did a solo bicycle trip across the continental United States.
04/13/2018 Michael A. Goodrich
Brigham Young University
Toward Human Interaction with Bio-Inspired Robot Swarms
Abstract: Bio-inspired robot swarms are being designed and studied for many problems including search, pollution monitoring and control, and security. These swarms have some important advantages compared to traditional multi-agent AI approaches, including: resilience to robot attrition, robustness to communication failures, ability to explore multiple solutions to a single problem, and ability to appropriately (re)distribute resources when problems arise. These advantages come from how decentralized computation and sensing of the robots lead to robust emergent collective behaviors A fundamental challenge is figuring out how to allow humans to influence and manage swarms without imposing the human as a single point of failure, defeating the advantage of decentralized/emergent behaviors. In this talk, I will discuss our approach to enable a human to manage and influence swarms.

Speaker’s Bio: Mike Goodrich is a professor and the chair of the Computer Science Department at Brigham Young University. He's published a lot peer-reviewed papers in a lot of areas including human-robot interaction, decision theory, artificial intelligence, intelligent vehicles, and multi-agent systems;. He's grateful to have received funding for students and research from ONR, ARL, NASA, NSF, DARPA, Honda, INL, and Nissan Motor Company. He helped create and organize the ACM/IEEE International Conference on Human-Robot Interaction, and the open-source Journal of Human-Robot Interaction. He likes to run to blow off steam and to enable him to eat high calorie peanut M&Ms.
04/20/2018 Dmitry Berenson
University of Michigan
What Matters for Deformable Object Manipulation
Abstract: Deformable objects such as cables and clothes are ubiquitous in factories, hospitals, and homes. While a great deal of work has investigated the manipulation of rigid objects in these settings, manipulation of deformable objects remains under-explored. The problem is indeed challenging, as these objects are not straightforward to model and have infinite-dimensional configuration spaces, making it difficult to apply established approaches for motion planning and control. One of the key challenges in manipulating deformable objects is selecting a model which is efficient to use in a control loop, especially when an accurate model is not available. Our approach to control uses a set of simple models of the object, determining which model to use at the current time step via a novel Multi-Armed Bandit algorithm that reasons over estimates of model utility. I will also present our work on interleaving planning and control for deformable object manipulation in cluttered environments, again without an accurate model of the object. Our method predicts when a controller will be trapped (e.g., by obstacles) and invokes a planner to bring the object near its goal. The key to making the planning tractable is to avoid simulating the motion of the object, instead only forward-propagating the constraint on overstretching. This approach takes advantage of the object’s compliance, which allows it to conform to the environment as long as stretching constraints are satisfied. Our method is able to quickly plan paths in environments with complex obstacle arrangements and then switch to the controller to achieve a desired object configuration.

Speaker’s Bio: Dmitry Berenson received a BS in Electrical Engineering from Cornell University in 2005 and received his Ph.D. degree from the Robotics Institute at Carnegie Mellon University in 2011, where he was supported by an Intel PhD Fellowship. He completed a post-doc at UC Berkeley in 2012 and was an Assistant Professor at WPI 2012-2016. He started as an Assistant Professor in the EECS Department and Robotics Institute at the University of Michigan in 2016. He has received the IEEE RAS Early Career award and the NSF CAREER award.
04/25/2018 Jeff Mahler
University of California Berkeley
The Dexterity Network: Deep Learning to Plan Robust Robot Grasps using Datasets of Synthetic Point Clouds, Analytic Grasp Metrics, and 3D Object Models
Abstract: Reliable robot grasping across a wide variety of objects is challenging due to imprecision in sensing, which leads to uncertainty about properties such as object shape, pose, mass, and friction. Recent results suggest that deep learning from millions of labeled grasps and images can be used to rapidly plan successful grasps across a diverse set of objects without explicit inference of physical properties, but training typically requires tedious hand-labeling or months of execution time. In this talk I present the Dexterity Network (Dex-Net), a framework to automatically synthesize training datasets containing millions of point clouds and robot grasps labeled with robustness to perturbations by analyzing contact models across thousands of 3D object CAD models. I will describe generative models for datasets of both parallel-jaw and suction-cup grasps. Experiments suggest that Convolutional Neural Networks trained from scratch on Dex-Net datasets can be used to plan grasps for novel objects in clutter with high precision on a physical robot.

Speaker’s Bio: Jeff Mahler is a Ph.D. student at the University of California at Berkeley advised by Prof. Ken Goldberg and a member of the the AUTOLAB and Berkeley Artificial Intelligence Research Lab. His current research is on the Dexterity Network (Dex-Net), a project that aims to train robot grasping policies from massive synthetic datasets of labeled point clouds and grasps generated using stochastic contact analysis across thousands of 3D object CAD models. He has also studied deep learning from demonstration and control for surgical robots. He received the National Defense Science and Engineering Fellowship in 2015 and cofounded the 3D scanning startup Lynx Laboratories in 2012 as an undergraduate at the University of Texas at Austin, which was acquired by Occipital in 2015.
04/27/2018 Karen Liu
Georgia Tech
Towards a Generative Model of Natural Motion
Abstract: Animals adapt their movements to interact with the world in a way natural to their anatomical structures. Their motor patterns are efficient, robust, and nearly universal across individuals in the same species. My research aims to understand the dynamics and control of animal natural movements through recreating them with minimal engineering effort. Further, I seek to develop effective techniques to transfer these generative models from physical simulation to the real-world robots. In this talk, I will discuss our recent endeavors to advance in these two research areas. To date, lifelike natural motions are typically generated by highly-engineered techniques that demand high-quality motion examples. On the other hand, the minimalist approach, such as deep reinforcement learning, requires little engineering effort but is not able to generate realistic natural motion. In contrast, we show that natural legged locomotion can emerge from two simple and well-known biomechanics principles: minimal-energy and gait symmetry, without using motion examples or complicating the reward function with morphology-specific information. The second topic in this talk focuses on the problem of sim-to-real transfer. Theoretically the ability to simulate an infinite number of scenarios, actions, and physical designs should provide a compelling environment for developing effective real-world control policies for motion. Practically, the “reality gap” between virtual simulations and the physical world renders control policies developed primarily from simulations ineffective in real life scenarios. One of our innovations to bridge the gap is a customizable contact model that incorporates the analytical solution with empirical data collected for a particular scenario, such that the simulated results better match the observed phenomenon.

Speaker’s Bio: C. Karen Liu is an associate professor in School of Interactive Computing at Georgia Tech. She received her Ph.D. degree in Computer Science from the University of Washington. Liu's research interests are in computer graphics and robotics, including physics-based animation, character animation, optimal control, reinforcement learning, and computational biomechanics. She developed computational approaches to modeling realistic and natural animal movements, learning complex control policies for humanoids and assistive robots, and advancing fundamental numerical simulation and optimal control algorithms. The algorithms and software developed in her lab have fostered interdisciplinary collaboration with researchers in robotics, computer graphics, mechanical engineering, biomechanics, neuroscience, and biology. Liu received a National Science Foundation CAREER Award, an Alfred P. Sloan Fellowship, and was named Young Innovators Under 35 by Technology Review. In 2012, Liu received the ACM SIGGRAPH Significant New Researcher Award for her contribution in the field of computer graphics.
05/11/2018 Guy Hoffman
Cornell University
Designing Robots for Fluent Collaboration and Companionship
Abstract: Designing robots for human interaction is a multifaceted challenge involving the robot's intelligent behavior, physical form, mechanical structure, and interaction aspects. In our lab, we develop and study interactive robotic systems, combining methods from AI, Mechanical and User-Centered Design, and Human-Computer Interaction. First, I will present AI systems to support human-robot fluency, including computational cognitive architectures rooted in timing, joint action, and embodied cognition. These systems led to the development of an interactive robotic improvisation system that uses embodied gestures for simultaneous, yet responsive, joint musicianship. We are now investigating how these methods can be used for a wearable robotic arm. When it comes to the robot's physical form, I draw on the fact that the expressive movement of the robot is at the core of its function, and argue for a movement-centric design approach. The robot’s movement is not added on after the robot is designed, but factored in from the onset and converses with both the visual and the pragmatic requirements of the robot. The use of techniques from 3D character animation, sculpture, industrial, and interaction design, will be exemplified through the design process of five socially expressive robots, including Shimon, Travis, Kip, Vyo, and Blossom. The third pillar of our work is the experimental study of people interacting with robots. Our lab developed a series of low-cost smartphone-based robots, which we use in situations of disclosure, conflict, compliance, and joint experiences. Our studies investigate the role of movement, timing, and nonverbal behavior in the social relationship and companionship between humans and robots, in an effort to design robots that better reflect the values we aspire to.

Speaker’s Bio: Guy Hoffman is Assistant Professor and the Mills Family Faculty Fellow in the Sibley School of Mechanical and Aerospace Engineering at Cornell University. Prior to that he was Assistant Professor at IDC Herzliya and co-director of the IDC Media Innovation Lab. Hoffman holds a Ph.D from MIT in the field of human-robot interaction. He heads the Human-Robot Collaboration and Companionship (HRC2) group, studying the algorithms, interaction schema, and designs enabling close interactions between people and personal robots in the workplace and at home. Among others, Hoffman developed the world's first human-robot joint theater performance, and a real-time improvising human-robot Jazz duet. His research papers won several top academic awards, including Best Paper awards at HRI and robotics conferences in 2004, 2006, 2008, 2010, 2013, and 2015. In both 2010 and 2012, he was selected as one of Israel's most promising researchers under forty. His TEDx talk is one of the most viewed online talks on robotics, watched more than 2.9 million times. Hoffman received his M.Sc. in Computer Science from Tel Aviv University as part of the Adi Lautman interdisciplinary excellence scholarship program.

Details of previous Robotics Colloquia can be found here.