Tell the Robot What You Want — Sandhya Subramani, AWS
Summary
The presentation demonstrates 'Scout,' a rover running on a Raspberry Pi, that has been upgraded from a traditionally programmed robot into an agentic system using AWS's open-source framework, `strands`. This architecture allows the robot to understand and execute complex commands in natural language (e.g., counting people or responding to arbitrary prompts) by having an AI agent orchestrate calls to existing hardware policies/tools. The system utilizes a multi-agent setup (thinker, communication, voice) and operates on a hybrid cloud/edge model for both training (VLA models) and fast runtime execution.
Key takeaways
-
Agentic Robotics Architecture
2:07
By implementing an agent layer above existing movement policies, the robot can perform tasks it was never explicitly trained for. The agent decides *what* to do, while the policy determines *how* to execute the action.
-
Multi-Agent System Design
5:46
Scout runs three simultaneous agents: a thinker (constantly assessing the environment), a communication agent (via Telegram/web app), and an optional voice agent, allowing for complex, concurrent decision-making.
-
Hybrid Cloud/Edge Deployment
11:57
The system is designed to operate on a hybrid model: training (VLA models) occurs in the cloud using `agentcore`, while execution and policy calls happen rapidly at the edge (on the robot's local hardware).
Technical details
-
System Architecture
The system operates on four layers: 1) The Agent Layer (topmost), which receives instructions; 2) The Policy Provider, which contains trained policies (potentially VLA models); 3) The Backend/Simulation Environment, where the policy runs; and 4) Physical Hardware (the robot).
-
Implementation Details
346s
The framework requires minimal setup—as little as five lines of code using `strands` to import the agent and call a specific hardware tool/policy.
-
Data Collection & Training
1027s
The robot can be used not only for tasks but also as a data collection rig, allowing manual movement and interaction to generate training episodes that improve its reasoning capabilities.
Mentioned resources
- strands agents
- AWS
- Raspberry Pi
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.