Google Agile Deliberation: What the 18-Session Study Shows

Google Research published a framework on Nov. 4, 2026 for a problem every content-moderation team knows: you can't train an image classifier for "healthy food" or "misleading reward bait" until you've decided what those mean, and most people only find out what they mean by looking at examples. The framework, called Agile Deliberation, has the model show you the borderline images and rewrites the classifier's prompt from your answers.

Two steps, one loop

In the first step, concept scoping, you split a broad concept into sub-concepts and exclusions. For "Healthy Food," the post's examples are Salads, Grain Bowls and Lean Proteins on the inside, and Processed Foods on the outside.

In the second step, concept iteration, a vision-language model pulls up images that sit on the line. The post's example is a fruit smoothie: healthy because of raw fruit, or excluded as processed because of the sugar? Your yes or no feeds an automatic prompt-optimization loop, so the classifier's boundary moves as your definition does.

The authors tested it on two concepts, Healthy Food and Pay-to-Play, which they describe as clickbait in entertainment or gaming that promises misleading financial or digital rewards.

What 18 sessions can and can't say

The evaluation was 18 live sessions of about 1.5 hours each, run against three alternatives: a zero-shot classifier, an automated "modeling collaborator," and manual deliberation with spreadsheets and hand-edited prompts. Participants used the systems in randomized order.

The paper (first posted Dec. 11, 2025, revised Apr. 4, 2026, and listed for CVPR 2026) reports 7.5% higher F1 than automated decomposition baselines and more than 3% higher than manual deliberation. The blog post says the exact gains appear only in its charts, and it adds that every participant preferred the new method.

Two limits. The ground truth here is the participant's own definition, so there is no public benchmark to compare against; the authors chose that on purpose. And I found no participant count in the post or the abstract, only the session count. Self-built evaluations are common in vision-language work; WeChat's open vision-language model and its self-built benchmark tests scored its biggest gains on tests the team wrote itself.

What's in the repository

The code is a companion repo, built around a Demo.ipynb notebook. Setup runs setup.sh, which downloads LAION-400M data and builds a FAISS index: two shards, about 260 MB, by default, while the README says 100 shards would be roughly 13 GB. The README mentions LLM calls but names no provider, warns a single API key may be rate-limited, and says several endpoints can be registered with ModelClient. It states no license and carries the note "This is not an officially supported Google product."

Where it would fit

Google lists trust-and-safety filters, digital collection curation and scientific image categorization. The research was done in partnership with Google Ads, though the post announces no product.

If you maintain a labeling policy that keeps changing, run the notebook on one category you already argue about and count how many borderline images change your mind. If it's more than a few, your written definition is the thing to fix first.

Source: Google Research, Agile Deliberation: How AI helps humans define the gray areas

google research agile deliberation image classification vision language models cvpr 2026

AI

About the author

AI Briefing Daily Editorial. The editorial desk of AI Briefing Daily, an independent publication covering artificial intelligence. Every story is researched against primary sources, checked by hand and reviewed by a person before publication.