SOTA Generative Media Panel — Dumitru Erhan, Shane Gu & Nicole Brichtova, Google DeepMind
The session provided an overview of SOTA generative media models, highlighting new APIs like NanoBanana 2 Lite and Gemini Omni Flash. Key architectural discussions centered on the limitations of language as a sole intermediate representation for complex sensory data (taste, smell, skin tone). The consensus points toward a future requiring unified 'World Models' that integrate visual, temporal, and symbolic reasoning, moving beyond single-modality generation. Evaluation remains highly dependent on human judgment, making robust testing and field feedback critical.
Key takeaways
-
New APIs Launched for Developers
0:15
Google launched NanoBanana 2 Lite (the fastest/cheapest image model in the family) and the Gemini Omni Flash APIs. The Omni Flash API enables video generation and editing, priced similarly to V3 fast, making it accessible for developers [0:15-0:40].
-
Generative Media Capabilities
2:00
Models can now take diverse inputs (e.g., a storyboard of images, an audio track) to generate video. Furthermore, natural language processing allows for advanced video editing tasks like adding or removing elements from existing footage [1:20-3:00].
-
The Limitation of Language as Representation
1:50
Speakers argued that language is an insufficient intermediate representation for highly sensitive sensory data (e.g., taste, smell, skin tone). This suggests a need for more foundational representations, potentially including code or direct binary/latent space conditioning [1:50-2:30].
-
Evaluation Challenges and Reward Hacking
0:35
While human preference often favors AI output (e.g., sharper, more saturated images), this metric is unreliable for optimization. External testers have found 'reward hacking' artifacts, such as the model consistently adding wedding rings to hands [0:35-0:45].