Speaker: Russ Tedrake, Toyota Professor of EECS, Aero/Astro, ME at Massachusetts Institute of Technology and VP of Research at Toyota Research Institute

Title: Dexterous Manipulation with Diffusion Policies

Date and Time: Friday, May 3 at 11:30 AM CT

Location: Tech ESAM M416 and Zoom

Zoom Link: https://tinyurl.com/CRBSeminar

• NU-authenticated attendees will be automatically admitted. Others, please email amy.nedoss@northwestern.edu to be admitted from the waiting room.

Abstract:

At the Toyota Research Institute (TRI), we've been working on behavior cloning for dexterous manipulation. Building on the Diffusion Policy framework that we've recently developed in collaboration with Shuran Song, we now have a very solid pipeline for taking ~50-100 bimanual haptic teleop demonstrations and turning that into a surprisingly effective visuomotor (+tactile) policy. Because there is no explicit state representation required, these skills work equally well manipulating deformable, liquid, or other difficult to model tasks as they do for more traditional rigid-object manipulation. We're actively scaling this up into the multi-task setting and now see a plausible path towards "Large Behavior Models". This behavior cloning pipeline is working incredibly well, and must be understood deeply in the broader context of output-feedback control. Time permitting, I'll also tell you a bit about some new results in optimization-based planning and control, and where they might fit in the age of foundation models.

Bio:

Russ Tedrake is the Toyota Professor at the Massachusetts Institute of Technology (MIT) in the Department of Electrical Engineering and Computer Science, Mechanical Engineering, and Aero/Astro, and he is a member of MIT’s Computer Science and Artificial Intelligence Lab (CSAIL). He is also the Vice President of Robotics Research at Toyota Research Institute (TRI). He received a B.S.E. in Computer Engineering from the University of Michigan in 1999, and a Ph.D. in Electrical Engineering and Computer Science from MIT in 2004. Dr. Tedrake is the Director of the MIT CSAIL Center for Robotics and was the leader of MIT’s entry in the DARPA Robotics Challenge. He is a recipient of the NSF CAREER Award, the MIT Jerome Saltzer Award for undergraduate teaching, the DARPA Young Faculty Award in Mathematics, the 2012 Ruth and Joel Spira Teaching Award, and was named a Microsoft Research New Faculty Fellow. His research has been recognized with numerous conference best paper awards, including ICRA, Robotics: Science and Systems, Humanoids, Hybrid Systems: Computation and Control, as well as the inaugural best paper award from the IEEE RAS Technical Committee on Whole-Body Control.

Related Info