# Agentic video understanding in Gemini

## Executive summary

Agentic Video Understanding in Gemini addresses the high token cost and inefficiency of processing entire long videos by utilizing an agentic loop architecture. Instead of feeding the model the full video data (which can exceed 100,000 tokens), the system allows the model to intelligently use specialized tools—such as `get transcript` or `get frames`—to localize its analysis. This approach significantly reduces token consumption while improving accuracy by focusing attention only on relevant segments of the video.

## Key takeaways

- Inefficiency of Naive Video Processing: Processing an entire long video can generate over 100,000 tokens, making it computationally expensive and inefficient if the required information is localized to a small segment (e.g., frame 900).
- Agentic Approach for Efficiency: The new pipeline does not require the entire video; instead, the model references the video and uses tools to extract only necessary data pieces (like transcripts or specific frames) before proceeding.
- Agentic Loop Mechanism: Analysis follows a traditional agentic loop: Thinking $\rightarrow$ Acting (using tools) $\rightarrow$ Observing (receiving observations like extracted frames), and looping until the final answer is derived. This process reduces token cost and enhances performance by zooming in on critical video functions.

## Technical details

- Agentic Video Understanding: This capability allows Gemini to intelligently break down a query against a long video using tools, rather than processing the entire input. It is available in Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite.
- Tool Calling Functions: The model can utilize specific functions to process video data: `get transcript` (for textual context), `get frames` (to specify desired frames and frames per second for localized visual analysis), and accessing audio information.

## Practical implications

- Analyzing long video logs or recordings (e.g., CI failure videos) without incurring massive token costs.
- Improving the precision of automated analysis by forcing the model to focus only on time-stamped events relevant to a specific query.
- Enabling more complex, multi-step reasoning over large media artifacts.

## Topics

Gemini, Video Processing, Agentic AI, Token Efficiency, LLMs, Google Developers Resources

Source: https://www.youtube.com/watch?v=ytjgy30Cono