Fundus-R1: Training a Fundus-Reading MLLM on Public Data

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Fundus-R1: Training a Fundus-Reading MLLM on Public Data".

Jane: Here's the gist of the paper. The gist The proposed Fundus-R1 model introduces a reasoning-enhanced multimodal large language model trained exclusively on public datasets to perform fundus image reading, overcoming the limitation of relying on inaccessible private clinical reports.

Tom: Problem Statement Fundus image understanding is inherently knowledge-intensive, making it a challenging vision-language task because it requires the model not only to identify (subtle) visual findings in a given fundus image but also to leverage highly specific domain knowledge to decode these findings into diagnostic output.

Jane: Current fundus-reading MLLMs mostly rely on richer supervision constructed from a mixture of public and private resources, such as in-house samples paired with high-quality clinical reports or private retinal images.

Tom: Such valuable samples are not publicly accessible, which not only hinders reproducibility but also practically limits research to few players.

Title and authors: Tom: So let's talk about who wrote this and what the title says. The paper is titled "FundusR1: Training a Fundus-Reading MLLM on Public Data." It’s authored by Yuchuan Deng, Qijie Wei, Kaiheng Qian, Jiazhen Liu, Zijie Xin, Bangxiang Lan, Jingyu Liu, Jianfeng Dong and Xirong Li.

Jane: They are authors from Renmin University of China and The Hong Kong University of Science and Technology. It sounds like a team with a solid background in both the AI side and the medical application.

Lu: Lu looks at this title, and what's interesting is that they focus on training an MLLM specifically for fundus image reading using public data, which is pretty ambitious given how knowledge-intensive that task is.

Meng: Ambitious is one word. The challenge here, as I see it from a deployment standpoint, is if this model can actually generalize well once you move it out of the controlled research environment and onto real patient scans without needing those private reports for every single case.

Lalam: From my perspective, the name itself suggests a focus on reasoning enhancement rather than just simple image recognition, which points toward solving that knowledge gap we talked about earlier.

The paper's summary: Tom: So the summary boils down to them proposing a novel approach to train this FundusR1 MLLM using only public data where over ninety-four percent of the data has only image-level labels. They aren't just feeding it pictures and answers, which is what most supervised learning does.

Jane: It sounds like they're saying that instead of relying on those missing private reports, they can induce reasoning supervision from public data by linking visual findings to specific ophthalmic knowledge in a structured way.

Lu: Exactly. They have two main technical contributions here: first, a RAG-based method for creating image-specific, knowledge-aware reasoning traces that connect what the model sees to actual medical facts.

Meng: A RAG pipeline sounds complex to set up during initial training, and I'm wondering how much overhead that adds to the training pipeline compared to just using standard supervision methods.

Lalam: The paper suggests this RAG approach disentangles the visual findings from the label- and modality-conditioned domain knowledge, which is key because it helps when the base MLLM doesn't already have a lot of ophthalmic knowledge.

The paper's improvements: Tom: The authors propose two main improvements to get this going. First, they introduce this RAG-based pipeline for composing those knowledge-aware reasoning traces, which explicitly links the visual findings to ophthalmic knowledge.

Jane: And the second improvement is enhancing Reinforcement Learning with Verifiable Rewards, or RLVR, by adding a process reward that encourages self-consistency of the generated reasoning trace in each training rollout.

Lu: That process reward is important because it rewards the model for generating a consistent reasoning trace, which helps guide it better during post-training refinement. It’s not just about getting the final answer right; it’s about *how* it gets there.

Meng: I see that as trying to enforce a certain logical flow in its internal thinking process, which is helpful if we want the AI's internal steps to be traceable later on, but I'm still skeptical about how robust those traces are when tested on real-world noisy data.

Lalam: They use this process reward to evaluate whether the generated trace aligns with either the visual findings or the domain knowledge, which means they're checking for consistency at every step of the generation process.

Conclusion: Tom: So to wrap up, FundusR1 uses a two-stage training procedure, starting with supervised finetuning and then refining it with this process-reward enhanced RLVR. This whole setup lets them induce reasoning supervision from public data even when the base MLLM is lacking specific ophthalmic knowledge.

Jane: What this means for us is that we might be able to create a more reliable fundus-reading model using data that's widely available, rather than being stuck needing those private clinical reports.

Lu: It moves the problem of supervision away from needing perfectly annotated reasoning traces and towards dynamically composing them using RAG based on external knowledge.

Meng: From an engineering standpoint, the two-stage training is smart because it acknowledges that early rollouts are often messy, so they use RLVR to clean up that inconsistency during the later stages.

Lalam: FundusR1 shows that you can bridge the gap between just having an image label and actually teaching a model how to reason about what’s in that image based on external domain knowledge.

Tom: So we've discussed how they use RAG for traces, RLVR for rewards, and two stages of training to build FundusR1, which is trained exclusively on public data. That’s where we are with this paper today.