Being John Malkovich

Given a photo of person A, we seek a photo of person B with similar pose and expression. Solving this problem enables a form of puppetry, in which one person appears to control the face of another. When deployed on a webcam-equipped computer, our approach enables a user to control another person’s face in real-time. This image-retrieval-inspired approach employs a fully automated pipeline of face analysis techniques and is extremely general — we can puppet people directly from their photo collections or videos in which they appear. We show several examples using images and videos of celebrities from the Internet.
(Faculty: Seitz)