Local Agentic Theory For Mobile Games — Shafik Quoraishee & Joanne Song, The New York Times
The presentation outlines 'Local Agentic Theory' for mobile games, arguing that the future of AI gaming lies in decentralized, on-device agents rather than centralized cloud models. These local agents use advanced techniques (like LLM reasoning over game state) to provide highly personalized experiences. Furthermore, this approach allows accessibility features to move beyond static toggles, enabling real-time, dynamic tuning of gameplay difficulty and interaction controls based on the user's live context (e.g., tracking eye gaze or shaky taps).
Key takeaways
-
Local Computation is Essential for Mobile AI
4:09
Running AI locally minimizes latency, avoids expensive cloud inference calls, and ensures privacy by keeping computation within the device's security zone. This allows advanced AI features to function reliably even without constant internet connectivity (e.g., in a subway tunnel).
-
Agentic Systems vs. Reinforcement Learning (RL)
7:36
While RL models train by changing weights through cycles, an agentic system uses language models and in-context learning to reason over the game's state space. This allows for a more dynamic loop that adapts to new situations using local device functionality.
-
The Device Budget Constraint
13:39
Local agents must operate within strict resource budgets: time (fitting planning into a single 16ms frame for 60Hz refresh rates), space (managing model weights and state history compression), and energy (optimizing processing to prevent rapid battery drain).
-
Accessibility as a Graded Dial, Not Toggles
Traditional fixed-state games are limited by static menus. The proposed model uses international standards like WCAG 2.2 to build dynamic player states. Agents can actively monitor user input (e.g., tracking eye gaze or analyzing shaky taps) and dynamically adjust controls—such as resizing targets or injecting exit routes from focus traps—to meet the player's real-time needs.