We are developing new methods for control optimization aimed at real-time robotic control. This is a collaboration betwen Zoran Popovic's group, which developed high-fidelity controllers for physics-based animation, and Emo Todorov's group, which developed efficient algorithms for optimal control as well as a new physics engine tailored to control optimization. Our approach combines offline learning of semi-global value functions and control policies with online trajectory optimization or model-predictive control (MPC). We recently used MPC to make a 3D robot bounce two ping-pong balls on the same paddle and to generate complex locomotion behaviors, including swimming, hopping, and getting up in real-time. We are now extending this work to humanoid robots by increasing the efficiency of our optimization algorithms as well as porting our physics engine to GPUs.