Robotics and State Estimation Lab
Current Projects
Physically Grounded Language Understanding
Attribute Based Object Identification

Over the last years, the robotics community has made substantial progress in detection and 3D pose estimation of known and unknown objects. However, the question of how to identify objects based on language descriptions has not been investigated in detail. While the computer vision community recently started to investigate the use of attributes for object recognition, these approaches do not consider the task settings typically observed in robotics, where a combination of appearance attributes and object names might be used in referral language to identify specific objects in a scene.
RGB-D Object Dataset
A large dataset of 300 common household objects recorded using a Kinect style 3D camera.
RGB-D Mapping: Using Depth Cameras for Dense 3D Mapping
Hierarchical Matching Pursuit for RGB-D Recognition

Object Segmentation from Motion

We can't be sure where objects are unless we see them move relative to each other. In this project we investigate using motion as a cue to segment objects. We can make use of passive sensing or active vision, and both long-term and short-term motion, to aid segmentation.
Data-Efficient Robot Reinforcement Learning

How long does it take for a robot to learn a task from scratch if no informative prior knowledge is given? Typically, very long. This project aims at developing and applying novel reinforcement learning methods to low-cost off-the-shelf robots to make them learn tasks in a few trials only. We use a standard robot arm by Lynxmotion and a Kinect-depth camera (total cost is 500 USD) and demonstrate that fully autonomous learning (with random intializations) requires only a few trials.
RGB-D Kernel Descriptors

3-D Object Discovery Using Motion

Gaussian Processes for Bayesian State Estimation
RGB-D Object Recognition and Detection

Robot Game Playing

Robotic In-Hand 3D Object Modeling
Particle Filters
Robotic Pile Sorting and Manipulation
We are investigating strategies for robot interaction with piles of objects and materials in cluttered scenes. In particular, interaction with unstructured sets of objects will allow a robot to explore and manipulate novel items in order to perform useful tasks, such as counting, arranging, or sorting even without having a prior model of the objects.
Inactive Projects
Learning to Navigate Through Crowded Environments
Museum Tour-guide Robots
Plant Care
The plant care project helps us to investigate how mobile robots can interact with environments that are equipped with networks of sensors. The task of the robot is to water the plants and calibrate the sensors in the environment.
People Tracking
Robot Localization
Robot localization is an important application driving our research in belief representations and particle filtering for state estimation. Localization is one of the most fundamental problems in mobile robotics. With our collaborators, we introduced grid-based approaches, tree-based representations, and particle filters for robot localization. We were the first to solve the global localization problem, which requires a robot to estimate its position within an environment from scratch, i.e., without knowledge of its start position.
Active Sensing and Estimation in RoboCup
The task sounds simple: Program Sony AIBO robots to play soccer. We use RoboCup to investigate techniques for multi-robot collaboration, active sensing, and efficient state estimation. Our multi-model technique for ball tracking allows our robots to accurately track the ball and its interactions with the environment; even under the highly non-linear dynamics typically occuring during a soccer game. Our active sensing strategy is based on reinforcement learning.
Mapping and Exploration
Activity Recognition
Semantic Mapping







cs.