RSE-lab Activity Recognition

Various additional animations related to activity recognition and people tracking can be found on our MCL in action web site.

We develop techniques that can extract high-level information about a person's activties from raw sensor data. Such information can be used in various applications, ranging from helping elderly people or people suffering from brain injuries during their everyday lives (see also the assisted cognition project).

Project Contributors

Dieter Fox, Jeff Bilmes, Brian Ferris, Henry Kautz, Anthony LaMarca, Julie Letchner, Lin Liao, Don Patterson, Matthai Philipose, Alvin Raj, and Amarnag Subramanya

Main publications

• Recognizing Activities and Spatial Context Using Wearable Sensors.
  A. Subramanya, A. Raj, J. Bilmes, and D. Fox. UAI-06.
• Gaussian Processes for Signal Strength-Based Location Estimation.
  B. Ferris, D. Haehnel, and D. Fox. RSS-06.
• Hierarchical Conditional Random Fields for GPS-based Activity Recognition.
  L. Liao, D. Fox, and H. Kautz. ISRR-05.
• Fine-Grained Activity Recognition by Aggregating Abstract Object Usage.
  D. Patterson, D. Fox, H. Kautz, and M. Philipose. ISWC-05. ISWC Best Paper Award.
• Location-Based Activity Recognition.
  L. Liao, D. Fox, and H. Kautz. NIPS-05.
• Large-Scale Localization from Wireless Signal Strength.
  J. Letchner, D. Fox, and A. LaMarca. AAAI-05.
• Location-Based Activity Recognition using Relational Markov Networks.
  L. Liao, D. Fox, and H. Kautz. IJCAI-05.
• Learning and Inferring Transportation Routines.
  L. Liao, D. Fox, and H. Kautz. AAAI-04. AAAI Outstanding Paper Award.
• Opportunity Knocks: a System to Provide Cognitive Assistance with Transportation Services.
  D. J. Patterson, L. Liao, K. Gajos, M. Collier, N. Livic, K. Olson, S. Wang, D. Fox, and H. Kautz. UBICOMP-04.
• Bayesian filtering for location estimation.
  D. Fox, J. Hightower, L. Liao, D. Schulz, and G. Borriello. IEEE Pervasive Computing, 2003.
• Inferring High-Level Behavior from Low-Level Sensors.
  D.J. Patterson, L. Liao, D. Fox, and H. Kautz. UBICOMP-03.
• Voronoi Tracking: Location Estimation Using Sparse and Noisy Sensor Data.
  L. Liao, D. Fox, J. Hightower, H. Kautz, and D. Schulz. IROS-03.

Example Scenario

In our GPS tracking work, for instance, a Rao-Blackwellised particle filter estimates a person's location and mode of transportation (bus, foot, car). A hierarchical dynamic Bayesian network is trained to additionally learn and infer the person's goals and trip segments. If you click on the left picture, you will see an animation that illustrates the filter as it observes a person getting on and off the bus. The color of the particles indicates the current mode of transportation estimate (foot=blue, bus=green, car=red). The middle animation shows the prediction of a person's current goal (black line). Size of blue circles indicates probability of this location being the goal. Right animation shows detection of an error. The person fails to get off the bus at the marked location.

Getting on and off a bus

Goal prediction

Error detection