Robotics and State Estimation Lab

 

Current Projects

Physically Grounded Language Understanding

A number of long-term goals in robotics – for example, using robots in household settings – require robots to interact with humans. In this project, we explore how robots can learn to correlate natural language to the physical world being sensed and manipulated, an area of research that falls under grounded language acquisition.

Attribute Based Object Identification

Over the last years, the robotics community has made substantial progress in detection and 3D pose estimation of known and unknown objects. However, the question of how to identify objects based on language descriptions has not been investigated in detail. While the computer vision community recently started to investigate the use of attributes for object recognition, these approaches do not consider the task settings typically observed in robotics, where a combination of appearance attributes and object names might be used in referral language to identify specific objects in a scene.

RGB-D Object Dataset

A large dataset of 300 common household objects recorded using a Kinect style 3D camera.

RGB-D Mapping: Using Depth Cameras for Dense 3D Mapping

We align RGB-D (Red, Green, Blue plus Depth) point clouds acquired with a depth camera to create globally consistent dense 3D maps.

Hierarchical Matching Pursuit for RGB-D Recognition

Hierarchical Matching Pursuit uses sparse coding to learn codebooks at each layer in an unsupervised way and then builds hierarchial feature representations from the learned codebooks. It achieves state-of-the-art results on many types of recognition tasks.

Object Segmentation from Motion

We can't be sure where objects are unless we see them move relative to each other. In this project we investigate using motion as a cue to segment objects. We can make use of passive sensing or active vision, and both long-term and short-term motion, to aid segmentation.

Data-Efficient Robot Reinforcement Learning

How long does it take for a robot to learn a task from scratch if no informative prior knowledge is given? Typically, very long. This project aims at developing and applying novel reinforcement learning methods to low-cost off-the-shelf robots to make them learn tasks in a few trials only. We use a standard robot arm by Lynxmotion and a Kinect-depth camera (total cost is 500 USD) and demonstrate that fully autonomous learning (with random intializations) requires only a few trials.

RGB-D Kernel Descriptors

Kernel descriptors is a general approach that extracts multi-level representations from high-dimensional structured data such as images, depth maps, and 3D point clouds.

3-D Object Discovery Using Motion

In contrast to object recognition or object detection, which match data to existing object models, object discovery creates object models. Obviously, we need information sources to compensate for the lack of models. In this project, we investigate using 3-D motion of surface patches between multiple maps of the same environment as such a cue.

Gaussian Processes for Bayesian State Estimation

The goal of this project is to integrate Gaussian process prediction and observation models into Bayes filters. These GP-BayesFilters are more accurate than standard Bayes filters using parametric models. In addition, GP models naturally supply the process and observation noise necessary for Bayesian filters.

RGB-D Object Recognition and Detection

In this project we address joint object category, instance, and pose recognition in the context of rapid advances of RGB-D cameras that combine both visual and 3D shape information. The focus is on detection and classification of objects in indoor scenes, such as in domestic environments

Robot Game Playing

The Gambit manipulator is a novel robotic arm combined with an RGBD camera, used for interacting dextrously with small-scale physical objects, as in game playing.

Robotic In-Hand 3D Object Modeling

We address the problem of active object investigation using robotic manipulators and Kinect-style RGB-D depth sensors. To do so, we jointly tackle the issues of sensor to robot calibration, manipulator tracking, and 3D object model construction. We additionally consider the problem of motion and grasp planning to maximize coverage of the object.

Particle Filters

With our collaborators, we introduced particle filters as a powerful tool for state estimation in mobile robotics. More recently, we developed several improvements to particle filters, including adaptive particle filters, which dynamically adapt the size of sample sets to the complexity of the underlying belief. We also developed real-time particle filters, which avoid loss of sensor information even under limited computational resources.

Robotic Pile Sorting and Manipulation

We are investigating strategies for robot interaction with piles of objects and materials in cluttered scenes. In particular, interaction with unstructured sets of objects will allow a robot to explore and manipulate novel items in order to perform useful tasks, such as counting, arranging, or sorting even without having a prior model of the objects.

 

Inactive Projects

Learning to Navigate Through Crowded Environments

In this project we use inverse reinforcement learning to train a planner for natural and efficient robotic motion in crowded environments.

Museum Tour-guide Robots

The reliability of probabilistic methods for mobile robot navigation has been demonstrated during the deployment of the mobile robots Rhino and Minerva as tour-guides in two populated museums. The task of these robots was to guide people through the exhibitions of the ``Deutsches Museum Bonn'', Germany, and the ``National Museum of American History'' in Washington, D.C.

Plant Care

The plant care project helps us to investigate how mobile robots can interact with environments that are equipped with networks of sensors. The task of the robot is to water the plants and calibrate the sensors in the environment.

People Tracking

Knowing and predicting the locations of people moving through an environment is a key component of many proactive service applications, including mobile robots. Depending on the task and the available sensors, we apply joint probabilistic data association filters, Rao-Blackwellised particle filters, and Voronoi-based particle filters to estimate locations of people. Such estimates build the foundations for learning typical motion patterns of people, as used in the activity recognition project.

Robot Localization

Robot localization is an important application driving our research in belief representations and particle filtering for state estimation. Localization is one of the most fundamental problems in mobile robotics. With our collaborators, we introduced grid-based approaches, tree-based representations, and particle filters for robot localization. We were the first to solve the global localization problem, which requires a robot to estimate its position within an environment from scratch, i.e., without knowledge of its start position.

Active Sensing and Estimation in RoboCup

The task sounds simple: Program Sony AIBO robots to play soccer. We use RoboCup to investigate techniques for multi-robot collaboration, active sensing, and efficient state estimation. Our multi-model technique for ball tracking allows our robots to accurately track the ball and its interactions with the environment; even under the highly non-linear dynamics typically occuring during a soccer game. Our active sensing strategy is based on reinforcement learning.

Mapping and Exploration

We are interested in the development of robust and efficient map buiding techniques. We developed different solutions to this problem, ranging from expectation maximization (EM) to Rao-Blackwellised particle filters. We also introduced novel coordination strategies for large teams of mobile robots. Within the CentiBots project, we developed a decision-theoretic approach that enables teams of robots to build a consistent map of an environment even when the robots start from different, completely unknown locations.

Activity Recognition

This project aims at learning and estimating high-level activities from raw sensor data. To do so, we strongly rely on the etimates generated by our people tracking approaches. We recently demonstrated that it is possible to learn typical outdoor navigation patterns of a person using raw GPS data. For example, our approach uses EM to learn where a person typically gets on or off the bus. Such techniques allow hand-held computer devices to assist people with cognitive disorders during their everyday life.

Semantic Mapping

The goal of this project is to generate models that describe environments in terms of objects and places. Such representations contain far more useful information than traditional maps, and enable robots to interact with humans in a more natural way.

Centibots: the Hundred-Robots Project

The Centibots system is a framework for very large teams of robots that are able to perceive, explore, plan and collaborate in unknown environments. The Centibots were developed in collaboration with SRI International, funded under DARPA's SDR program. The Centibots team currently consists of approximately 100 robots. These robots can be deployed in unexplored areas, and can efficiently distribute tasks among themselves; the system also makes use of a mixed initiative mode of interaction in which a user can influence missions as necessary.