# Robotics Has Been Stuck for 70 Years — Deepak Pathak, Skild AI

## Executive summary

Deepak Pathak argues that robotics progress has stalled for approximately 70 years because the field has been treated as a hardware problem rather than a general intelligence problem. He introduces the concept of 'omni-bodied intelligence'—a single brain model applicable to any robot and any task, regardless of hardware. This approach leverages a 'data flywheel' that combines highly scalable data (simulation, human video) with high-quality, low-volume data (teleoperation) and, critically, real-world deployment data. Demonstrations include complex tasks like AirPods insertion, omelet cooking on $4,000 arms, and robust GPU assembly for NVIDIA's factory, showcasing the system's ability to handle real-world disturbances and zero-shot transfers.

## Key takeaways

- Robotics Stagnation and the General Brain: Robotics has historically been limited by approaching it as a hardware problem. The field is constrained by the lack of a general brain, leading to the 'Moravec's paradox' (what is easy for humans is hard for machines, and vice versa).
- The Data Bottleneck: Collecting robot data via teleoperation is extremely slow and expensive. To reach the data scale of models like GPT-3, the entire US population would take over a century, highlighting the need for scalable data sources.
- Omni-bodied Intelligence and the Data Flywheel: The proposed solution is an 'omni-bodied brain': one model for any robot and any task. This system utilizes a data flywheel, pre-training on scalable data (simulation, human video), post-training on teleoperation, and continuous improvement via deployment data.
- Real-World Deployment and Robustness: The system demonstrates extreme robustness, performing tasks like GPU assembly in a randomized, noisy factory environment, and adapting to disturbances (e.g., recovering movement after disabling legs) without explicit mapping or planning.

## Technical details

- Omni-bodied Intelligence: A single, generalized model designed to operate across diverse hardware (humanoids, quadripeds, robotic arms) and tasks, overcoming the limitations of single-purpose systems.
- Data Flywheel: A continuous improvement loop where deployment data feeds back into the pre-training process, enabling scale and robustness. This is crucial because hardware deployment is not limited to a single version.
- End-to-End System Architecture: The system operates end-to-end, reading directly from cameras and applying power directly to motors, bypassing traditional robotics pipelines that require separate planning and mapping modules.
- Task Specifics: Demonstrated tasks include inserting AirPods (requiring high dexterity with a simple gripper), cooking omelets on $4,000 arms using only camera input, and climbing stairs (a task requiring complex vision and understanding, unlike simple mobility).

## Practical implications

- Accelerating factory automation and manufacturing processes (e.g., GPU assembly) in non-traditional, noisy environments.
- Enabling generalized service applications, such as package delivery and complex household tasks (e.g., laundry folding, omelet cooking).
- Improving robot resilience and safety by allowing the system to function even when parts of the body are disabled or damaged.

## Topics

Artificial Intelligence (AI), Robotics, Machine Learning, Deep Learning, Computer Vision, Automation, Build Engineering, Skild AI, Carnegie Mellon (CMU), NVIDIA

Source: https://www.youtube.com/watch?v=jFHteJjRl8A