Robotics Has Been Stuck for 70 Years — Deepak Pathak, Skild AI
Summary
Deepak Pathak argues that robotics progress has stalled for approximately 70 years because the field has been treated as a hardware problem rather than a general intelligence problem. He introduces the concept of 'omni-bodied intelligence'—a single brain model applicable to any robot and any task, regardless of hardware. This approach leverages a 'data flywheel' that combines highly scalable data (simulation, human video) with high-quality, low-volume data (teleoperation) and, critically, real-world deployment data. Demonstrations include complex tasks like AirPods insertion, omelet cooking on $4,000 arms, and robust GPU assembly for NVIDIA's factory, showcasing the system's ability to handle real-world disturbances and zero-shot transfers.
Key takeaways
-
Robotics Stagnation and the General Brain
5:57
Robotics has historically been limited by approaching it as a hardware problem. The field is constrained by the lack of a general brain, leading to the 'Moravec's paradox' (what is easy for humans is hard for machines, and vice versa).
-
The Data Bottleneck
9:02
Collecting robot data via teleoperation is extremely slow and expensive. To reach the data scale of models like GPT-3, the entire US population would take over a century, highlighting the need for scalable data sources.
-
Omni-bodied Intelligence and the Data Flywheel
12:02
The proposed solution is an 'omni-bodied brain': one model for any robot and any task. This system utilizes a data flywheel, pre-training on scalable data (simulation, human video), post-training on teleoperation, and continuous improvement via deployment data.
-
Real-World Deployment and Robustness
The system demonstrates extreme robustness, performing tasks like GPU assembly in a randomized, noisy factory environment, and adapting to disturbances (e.g., recovering movement after disabling legs) without explicit mapping or planning.
Technical details
-
Omni-bodied Intelligence
722s
A single, generalized model designed to operate across diverse hardware (humanoids, quadripeds, robotic arms) and tasks, overcoming the limitations of single-purpose systems.
-
Data Flywheel
1321s
A continuous improvement loop where deployment data feeds back into the pre-training process, enabling scale and robustness. This is crucial because hardware deployment is not limited to a single version.
-
End-to-End System Architecture
1698s
The system operates end-to-end, reading directly from cameras and applying power directly to motors, bypassing traditional robotics pipelines that require separate planning and mapping modules.
-
Task Specifics
1617s
Demonstrated tasks include inserting AirPods (requiring high dexterity with a simple gripper), cooking omelets on $4,000 arms using only camera input, and climbing stairs (a task requiring complex vision and understanding, unlike simple mobility).
Mentioned resources
- Skild AI
- Carnegie Mellon (CMU)
- NVIDIA
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.