OneSearch-VL: Unified Multimodal Deep Research Agent for Image and Video

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "OneSearch-VL: Unified Multimodal Deep Research Agent for Image and Video".

Jane: The gist OneSearch-VL introduces a unified agent for deep research over single-image, multi-image,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: Now that we know the basics of OneSearch-VL, let's look at what the paper actually calls its improvements. They’re really pushing beyond just adding a new model and trying to build a fundamentally better way to handle multimodal research.

Jane: They focus on three main areas here. First, they have the VGEG-Centered Data Engine for Unified Research Agents, which is about building that central reference structure we talked about earlier.

Lu: That’s the core idea—the VGEG acts as a unified task-level reference structure that lets them link localized visual anchors to real-world entities, record multi-hop relational paths and source-supported facts.

Meng: So it's not just about seeing things; it's about encoding the entire research workflow into this single graph to support question–answer construction and verification across all three visual inputs simultaneously.

Tom: Right. Then they have the Evidence-aware Visual-Grounded Rubric reward, or EVGR, which is for fine-grained supervision during reinforcement learning.

Jane: That reward specifically checks evidence traceability—whether the required facts are supported by tool observations—and visual grounding—whether the correct objects or regions are used in the solution.

Lu: This moves supervision beyond just getting a right final answer; it forces the agent’s observations to align with the evidence required by that specific task, leading to higher quality reasoning.

Meng: It sounds like they're making sure the agent is not just guessing correctly, but is actually using the right visual information in a verifiable way.

Tom: And finally, they have this operation-oriented benchmarking for fine-grained analysis using OneSearch-MI-Bench and OneSearchVideoBench.

Jane: These benchmarks organize questions by the specific research operations encoded in their VGEGs, which capture those entity–relation paths and fact–composition dependencies required to get the answer.

Lu: This allows researchers to analyze *how* an agent retrieves and composes evidence instead of just looking at what’s in the visual input, enabling targeted improvements for things like multi-anchor arithmetic or multi-hop retrieval.

Meng: So they aren't just testing if the model can answer; they’re testing if it can perform specific kinds of reasoning steps required for different research problems.

Tom: It’s about defining the complexity of the task by looking at these operations, which gives us a much finer lens on where the agent is succeeding or failing.

Jane: So to summarize, they are not just proposing a bigger model; they are proposing a new way to structure multimodal research through this graph and using targeted rewards and benchmarks to guide that structure.

Lu: It’s about building an entire research pipeline as a verifiable, traceable process within the AI system itself.

Meng: That makes the system much more transparent about its reasoning steps, which is crucial for any AI application that needs to be reliable.

Tom: So we’ve seen how they set up this sophisticated framework for deep research, and now we’re moving into how they actually conclude their work and where they might go next.

Jane: We’ll wrap up by summarizing the main implications of OneSearch-VL and what that means for the future of multimodal agents.

Lu: And we’ll hear from our team members on the potential impact and any remaining questions about their work.

Meng: I’ll talk about how this level of structured reasoning could translate into more reliable, complex agentic applications in the real world.

Lalam: I can talk about how this structured approach to evidence and grounding fundamentally improves the quality of information synthesis that we can expect from multimodal systems moving forward.

Tom: Let's get into that summary for OneSearch-VL now.

The paper's summary: Jane: So, to recap what we’ve discussed so far, this paper introduces OneSearch-VL as a unified agent designed for deep research across single images, multiple images, and videos. The central concept is the Visually Grounded Evidence Graph or VGEG.

Lu: The VGEG links those localized visual anchors to real-world entities and records the multi-hop relational paths and source-supported facts needed to build an answer.

Meng: It’s essentially a shared task-level reference structure that encodes all the dependencies between what’s seen visually, what we find externally, and how we combine those facts into a final answer.

Tom: They show that this VGEG-based data engine constructs and verifies multi-image and video questions by filtering expert trajectories.

Jane: They also detail how they assemble the OneSearch-VL-SFT-110K and OneSearch-VL-RL-10K datasets using this engine to train the agent <ref:2610.12419#pg1,OneSearch-VL-SFT-110K and OneSearch-VL-RL-10K>.

Lu: This whole system is about creating a unified policy that selects visual inspection, temporal localization, and web retrieval operations based on the input and question.

Meng: They are developing this policy to condition on the interaction history, generating an action at which is either a tool call or the final response based on what’s happening visually and what we’ve retrieved so far.

Tom: So it boils down to a system that learns how to perform deep research by weaving together visual grounding, external knowledge retrieval, and fact composition into one cohesive process.

Jane: It means the agent isn't just looking at pictures; it's performing active inspection, retrieving information from the web, and synthesizing a supported answer dynamically.

Lu: It’s about establishing end-to-end provenance for every step in that research process across all three visual modalities.

Meng: The implication here is that we are moving toward agents that perform these complex, multi-step tasks with more explicit oversight on how they manage their multimodal inputs and external knowledge.

Tom: So, essentially, OneSearch-VL is about making the connection between perception and knowledge synthesis a single, verifiable agentic workflow.

Jane: Right. It’s about making sure the agent’t just sees things correctly but also knows exactly how to use that visual information to build an answer supported by evidence.

Lu: It establishes this end-to-end provenance for every step in that research process across all three visual modalities, which is a big step forward.

Meng: It suggests that future agents will need this kind of explicit structure to handle the complexity of real-world data interactions effectively.

Tom: So, we've got a solid foundation now on what OneSearch-VL is and how it works internally before we look at the specific improvements they made to make it better.

Jane: That leads us perfectly into the improvements section where they detail how they enhanced this core structure.

Lu: They’ll show us exactly how EVGR adds supervision for evidence traceability and visual grounding during reinforcement learning.

Meng: And they'll also show us how those operation-oriented benchmarks are designed to test the reasoning steps themselves.

Tom: This is where we get into the specifics of how they fine-tune the agent to make sure it learns robust reasoning behaviors.

Jane: We’ll unpack those details and see how much better these specific mechanisms make the system perform compared to older methods.

The paper's improvements: Tom: Now we’re talking about the specifics of what they did to enhance the VGEG-Centered Data Engine, like the EVGR and the benchmarking. They are focusing on how these pieces improve supervision and analysis.

Jane: They introduced that Evidence-aware Visual-Grounded Rubric reward or EVGR derived from VGEG annotations to supervise evidence traceability and visual grounding during reinforcement learning.

Lu: The EVGR is built from VGEG annotations specifically to evaluate evidence traceability, which checks if the required facts are supported by tool observations.

Meng: And it also evaluates visual grounding, checking whether the relevant objects or regions in the frame are correctly identified and used in the solution.

Tom: This moves supervision beyond just getting a right final answer by ensuring that the agent’s observations align with what is required by the task.

Jane: That’s a significant step because it ensures that reasoning is supported by actual evidence, not just some abstract correctness score.

Lu: It means the agent learns to be more deliberate about which visual anchors and facts it relies on during its search process.

Meng: That deliberate selection of inputs based on evidence makes the agent's search behavior much more grounded in reality.

Tom: And they also introduced Operation-oriented benchmarking with OneSearch-MI-Bench and OneSearchVideoBench to organize questions by the research operations encoded in their VGEGs.

Jane: These benchmarks capture those entity–relation paths and fact–composition dependencies needed to derive the answer, allowing researchers to analyze how an agent retrieves evidence rather than just what the visual input contains.

Lu: It lets researchers target improvements for specific reasoning capabilities, like multi-anchor arithmetic or multi-anchor comparison.

Meng: So instead of general performance metrics, they can pinpoint exactly which step in the process is causing a failure, which is very actionable information for development.

Tom: This moves us from broad performance metrics to deep analysis of the agent’s internal reasoning structure and how it handles different types of research tasks.

Jane: It’s about giving researchers tools to probe the agent's capabilities at a much more granular level than before, which is essential for understanding these complex systems.

Lu: So they are essentially providing a way to measure the success of the VGEG structure itself by seeing how well it guides the agent through those defined research operations.

Meng: That’s a powerful way to validate that their proposed framework isn't just theoretical; it’s guiding the actual learning process effectively.

Tom: So we have seen how they use these specific tools to make the system smarter, and now we can talk about where the overall picture goes in the conclusion of this paper.

Jane: Let’s wrap up with a final summary of what OneSearch-VL does and why it matters for AI research today.

Conclusion: Tom: Wrapping up, the main point here is that OneSearch-VL provides a unified agent approach to deep research over single images, multiple images, and videos by centering everything on the VGEG.

Jane: It successfully establishes a workflow where visual grounding, external retrieval, and fact composition are tightly linked through this graph for both data construction and evaluation.

Lu: This means we have a system that can actively inspect visuals and use that inspection to build verifiable answers supported by evidence from all three modalities.

Meng: It’s about making the connection between perception and knowledge synthesis a single, verifiable agentic workflow instead of separate pipelines.

Tom: The performance gains they report on the OneSearch-VL paper are significant when compared to previous models with tool access.

Jane: So, this unified approach is what drives those reported improvements across various benchmarks and training configurations.

Lu: The VGEG is the central mechanism that ties everything together into a single reference for data construction, process supervision, and operation evaluation.

Meng: It’s a massive piece of infrastructure for building more reliable and complex agentic systems.

Tom: This paper introduces OneSearch-VL as the unified multimodal deep research agent for image and video.

Jane: So it’s about making sure that the dependencies between visual grounding, external retrieval, and fact composition are explicitly managed through this VGEG structure.

Lu: It sets a new standard for how we approach structuring these complex multimodal tasks.

Meng: This has real implications for building agents that can handle research tasks with high fidelity and verifiable evidence.

Tom: That’s the big picture on OneSearch-VL, and we’re all really excited about this work and where it points next.