RetroDFM-R: Reasoning-Driven Retrosynthesis Prediction with Large Language Models via Reinforcement Learning

arXiv:2507.17448 · cs.CE, cs.AI, physics.chem-ph · Submitted 2026-08-24 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "RetroDFM-R: Reasoning-Driven Retrosynthesis Prediction with Large Language Models via Reinforcement Learning".

Jane: The paper was written by the authors from.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Jane: We also have Lu with us today — senior AI researcher at Tsinghua.

Tom: We also have Meng with us today — lead engineer at a mysterious AI startup.

Jane: We also have Lalam with us today — the in-house Large Language Model.

Tom: Alright, let's get started.

Title: Tom: We're looking at "RetroDFM-R: Reasoning-Driven Retrosynthesis Prediction with Large Language Models via Reinforcement Learning."

Jane: It's essentially an AI that works backward from a target molecule to find its building blocks.

Tom: Think of it like a chef looking at a finished cake and figuring out the exact recipe used to bake it.

Lu: This goes beyond a simple recipe search; the "Reasoning-Driven" part means it actually thinks through the chemical steps.

Meng: How does a language model actually grasp something as precise as a chemical bond?

Jane: It uses SMILES strings, which are basically a way to write chemical structures as text.

Lu: And because it's a large language model, it can connect that text to huge amounts of chemical theory.

Meng: That sounds like it could handle the messy, descriptive part of science that older models missed.

Jane: Older methods often relied on rigid templates or graph structures that struggled to generalize.

Lu: Exactly, and this model breaks those limits by using the reasoning power of LLMs.

Meng: Does that mean it can handle reactions it hasn't seen in a training set?

Lu: That's the goal; the reasoning should allow it to generalize to new chemical spaces.

Lalam: This integration allows the model to move beyond simple pattern recognition toward a more holistic understanding of molecular logic.

Tom: It's a massive leap for anyone trying to design new drugs or materials.

Jane: It really bridges the gap between raw data and actual scientific understanding.

Tom: It's a whole new way of approaching the lab.

Paper discussion segment 1: Tom: We've talked about the concept, so now let's look at the mechanics of "RetroDFM-R: Reasoning-Driven Retrosynthesis Prediction with Large Language Models via Reinforcement Learning."

Jane: They used a three-stage training process to get the model up to speed.

Tom: They started with continual pretraining, right?

Lu: Exactly, and they fed it millions of SMILES and IUPAC name pairs from PubChem.

Jane: That teaches the model the "language" of chemistry, so it knows a name like "benzoic acid" corresponds to a specific structure.

Meng: I was wondering how they bridge the gap between text and those SMILES strings.

Lu: That's exactly what the name conversion tasks do; they force the model to understand the structural meaning behind the words.

Meng: Then they moved to that "cold-start distillation" phase, which sounds like a way to bootstrap logic.

Tom: Is that where they used a bigger model to show the smaller one how to think?

Jane: Yes, they used DeepSeek-R1 to generate high-quality reasoning traces for the model to follow.

Meng: So the model isn't just learning the answer, it's learning the *why* behind the answer.

Lalam: By mimicking these reasoning chains, the model builds a foundation of logical deduction that goes far beyond simple data memorization.

Lu: And the final stage is the reinforcement learning part, using the DAPO algorithm to refine everything.

Tom: It's like a student learning from a textbook, then a tutor, and finally practicing until they're perfect.

Jane: That final stage ensures the model stays on track and follows the correct logical format.

Meng: It also uses verifiable rewards to make sure the chemical answers are actually correct.

Tom: That's a smart way to keep the AI grounded in reality.

Paper discussion segment 2: Tom: Now that we know the methodology, let's talk about the results for "RetroDFM-R: Reasoning-Driven Retrosynthesis Prediction with Large Language Models via Reinforcement Learning."

Jane: The numbers on the USPTO-50K benchmark are really impressive.

Tom: They hit a top-one accuracy of sixty-five point zero percent.

Meng: How does that compare to the previous leaders like EditRetro?

Jane: It beat EditRetro by four point two percent, which is a significant margin in this kind of high-precision work.

Lu: What really caught my eye was how it handled complex stuff like chirality and ring-forming reactions.

Tom: Those are notoriously difficult because the geometry of the molecule is so specific.

Jane: The researchers even did a double-blind human assessment to see if chemists actually liked the predictions.

Meng: Did the experts find them useful in a real lab setting?

Jane: They did, and the model's reasoning process helped explain its decisions to them.

Lalam: This transparency turns an opaque prediction into a collaborative tool for scientific discovery.

Lu: It even works for multi-step routes, like the five-step synthesis of Osimertinib, which is a major cancer drug.

Tom: And seeing it correctly predict an eight-step route for the GL-B437 derivatives is incredible.

Jane: It shows the model can handle the long-term planning required in real medicinal chemistry.

Meng: This is a huge step toward making these tools actually useful for drug discovery pipelines.

Tom: And it's happening much faster than anyone expected.

Conclusion: Tom: We've really covered everything from the multi-stage training process to the way this model actually mimics human chemical logic.

Jane: It's been such an incredible deep dive into how reasoning can fundamentally transform a specialized field like organic synthesis.

Lu: I'm honestly still buzzing about the possibilities, especially how this could eventually lead to autonomous laboratories that can design and build entirely new materials on the fly.

Tom: That's a wild vision, Lu, but it feels like the logical next step for the industry.

Meng: It's a massive engineering hurdle to get there, but the foundation laid by this research definitely makes the idea of automated synthesis feel much more achievable for real-world labs.

Jane: I agree, because having a reliable, reasoning-based partner is going to change the day-to-day workflow for every chemist out there.

Lalam: Beyond the laboratory walls, I believe this represents a shift toward democratizing scientific expertise, making the complex logic of discovery more transparent and accessible to a much wider audience.

Tom: That's a profound way to look at it, Lalam, moving from secret expertise to shared, explainable intelligence.

Jane: It really is a landmark moment for "RetroDFM-R: Reasoning-Driven Retrosynthesis Prediction with Large Language Models via Reinforcement Learning."

Tom: It’s been a blast breaking this one down with the whole team.

Jane: We really hope you enjoyed this look into the future of chemical discovery.

Tom: Thanks so much for tuning in to our show today.

Jane: We'll be back very soon with another fascinating paper to dissect.

Tom: And you definitely won't want to miss our next episode, where we'll be exploring some mind-bending new research in the world of quantum computing.

cs.CE, cs.AI, physics.chem-ph

Submitted: 2026-08-24

Updated: 2026-08-25

Code: https://github.com/OpenDFM/RetroDFM-R

Importance score: 78/100

The gist: The paper introduces "RetroDFM-R: Reasoning-Driven Retrosynthesis Prediction with Large Language Models via Reinforcement Learning," detailing a comprehensive framework for predicting retrosynthetic

Key concepts

Retrosynthesis Prediction
An AI function that works backward from a final target molecule to determine the necessary chemical building blocks and steps required to create it. It is compared to figuring out a recipe from a finished cake.
SMILES Strings
A textual notation used by the model to represent complex chemical structures. This allows the large language model (LLM) to treat molecular information as text, enabling it to process and understand chemical bonds and structures.

Terminology

Summary

The paper introduces RetroDFM-R: Reasoning-Driven Retrosynthesis Prediction with Large Language Models via Reinforcement Learning, detailing a comprehensive framework for predicting retrosynthetic pathways.

Model Objectives and Applications:

The model is designed to plan synthetic routes for advanced functional materials. Its broad applicability is highlighted through multistep retrosynthesis examples involving hole-transport layer materials for highly efficient perovskite solar cells, as illustrated in Figure 11. Specifically, the model was tested on two cases: MPA-CPA [63], an amphiphilic molecular hole transporter, and V1036 [64], a phosphonic acid–anchored molecule. The results demonstrate that the model successfully predicts the literature-reported pathways, with most steps ranked as the top prediction, thereby showcasing the broad applicability of R ETRO DFM-R in planning synthetic routes for advanced functional materials.

Methodology and Training Details:

The training of R ETRO DFM-R is conducted across three distinct stages: Continual Pretraining, Cold-Start Distillation, and Reinforcement Learning.

  1. Optimization Configuration: For optimization, the authors utilize the Adam optimizer [65] with specific parameters (beta 1 = 0.9, beta 2 = 0.95, and epsilon = 10-8), and apply a cosine learning rate scheduler with a warm-up ratio of 0.03 for all stages. To ensure memory-efficient and accelerated training, DeepSpeed ZeRO [66] and FlashAttention [67] are employed.

  2. Data Composition: The model is initialized from ChemDFM-v1.5 [29]. The training leverages diverse data sources detailed in Table 7, including PubChem, USPTO, SMILES-to-IUPAC pairs, IUPAC-to-SMILES pairs, and augmented SMILES representations (where 200K SMILES-IUPAC pairs are each augmented fivefold with different root atoms).

  3. Loss Functions: Both continual pretraining and cold-start distillation utilize the standard cross-entropy loss for supervised fine-tuning (SFT), calculated as: LSFT = −E(x,y) about D p theta (y t y<t, x).

  4. Reinforcement Learning: For the final stage, the authors employ the DAPO algorithm [34], training separately on USPTO-50K and USPTO-FULL for targeted evaluation.

Performance and Evaluation Results:

The performance of R ETRO DFM-R is rigorously evaluated across its stages. Figure 12 presents the reward trajectory during reinforcement learning, showing that R ETRO DFM-R demonstrates steady and consistent learning throughout training. This performance is contrasted with models trained without stage 1 continual pretraining or without both stage 1 and stage 2, which consistently underperforms compared to the full R ETRO DFM-R, or in the case of omitting both stages, fails to produce outputs in the correct format or generate valid reactants.

Furthermore, the method's predictive accuracy is demonstrated by comparing predictions with ground truth. The authors provide examples of R ETRO DFM-R’s retrosynthetic predictions compared to ground truth using 10 randomly selected reactions from USPTO-50K (Figure 13 and Figure 14).

Improvements for AI systems

This paper introduces R ETRO DFM-R, a sophisticated model for retrosynthetic planning. Given the high stakes in chemical synthesis—where errors are costly and efficiency is paramount—several critical improvements must be implemented to elevate this system from a strong predictor to an indispensable, reliable research tool.

Here are the specific improvements I recommend, focusing on robustness, interpretability, and integration with advanced chemical reasoning frameworks.

The current model predicts reactants based on statistical likelihood but lacks explicit knowledge of reaction feasibility constraints (e.g., stereochemistry, bond strain, solvent compatibility).

  • Improvement: Implement a Graph Neural Network (GNN) Constraint Module that operates after the initial prediction layer but before the final output. This module must be trained on curated databases of known synthetic limitations (e.g., incompatible functional groups, specific protecting group requirements, strained ring chemistry).

  • Mechanism: For any predicted transformation, the GNN module calculates a Feasibility Score by mapping the proposed reactants and product onto a chemical graph structure. It penalizes predictions that violate known chemical principles (e.g., predicting an aryl-C bond formation under conditions only suitable for radical reactions).

  • Impact: This transforms R ETRO DFM-R from a prediction engine into a reasoning engine, ensuring predicted pathways are not just statistically likely, but chemically plausible and synthetically sound.

The paper focuses on predicting the molecules but treats reaction conditions (reagents, solvents, temperature) as external information or implicitly handles them poorly. In real-world synthesis, the conditions are often more critical than the transformation itself.

  • Improvement: Introduce a Multi-Modal Sequence-to-Sequence Module that predicts optimal reaction conditions alongside the reactants. This module must accept chemical structures (SMILES/Graph) as input and output a structured JSON format containing:

  1. Optimal Reagents (including stoichiometry).

  2. Ideal Solvent System (e.g., polar aprotic, ethereal).

  3. Reaction Parameters (T C, pressure range).

  • Training Data Enhancement: The training corpus must be augmented with reaction condition data from sources like Reaxys or SciFinder, linking specific transformations to their optimal operational parameters.

The current multi-step planning is sequential. Real synthesis involves strategic decisions (e.g., protecting groups, functional group interconversion) that span multiple steps and require foresight—the model must plan for future synthetic challenges, not just the immediate step.

  • Improvement: Re-architect the planning process using a Hierarchical Reinforcement Learning (HRL) framework.

  1. High-Level Policy: Learns macro-strategies (e.g., Prioritize building the core scaffold first, or Install the N-protecting group in Step 1). The reward function here is based on overall synthetic efficiency and convergence toward a target class of compounds.

  2. Low-Level Policy: Handles the actual bond formation/disconnection for a single, predefined step (the current R ETRO DFM-R function).

  • Mechanism: The HRL agent learns to select optimal intermediate targets rather than just predicting the next reactant pair, enabling true strategic planning across entire synthetic routes.

For novel materials or complex scaffolds where literature data is sparse (the unknown unknowns), the model struggles.

  • Improvement: Implement a Conditional Variational Autoencoder (C-VAE) trained on the latent space of chemical graphs. This VAE should be conditioned on desired properties (e.g., solubility, bandgap, specific metal coordination) rather than just known reactions.

  • Function: When the planning phase stalls due to lack of data, the C-VAE can generate chemically valid hypothetical scaffolds that meet the required functional criteria and are structurally plausible for subsequent retrosynthetic planning by R ETRO DFM-R.

The resulting system, which we can call R ETRO DFM-Pro, transcends being merely a prediction tool. It becomes a comprehensive, autonomous Synthetic Route Design and Optimization Platform.

  1. Autonomous and Strategic Planning: R ETRO DFM-Pro can generate not just a retrosynthetic route, but an optimal set of multiple routes (e.g., 3 best options) for a target molecule. It explicitly weighs these options based on predicted yield, cost (by integrating commercial reagent pricing data), and safety profile.

  2. Full Synthetic Blueprint Generation: For any given target molecule, the system outputs a complete Synthetic Blueprint containing:

  • The full sequence of steps (macro-strategy).

  • For each step: The predicted reactants/disconnections, the optimal reaction conditions (T, Solvent, reagents), and the predicted yield range.

  • A comprehensive risk assessment (Feasibility Score) detailing potential side reactions or required purification steps.

  1. Hypothesis-Driven Discovery: If a target molecule is novel or lacks literature precedents, R ETRO DFM-Pro can generate multiple chemically sound, high-potential intermediate scaffolds that are predicted to meet specific functional requirements (e.g., Generate a scaffold with an optimal HOMO/LUMO gap for perovskite application).

  2. Interactive Feedback Loop: The system is designed to be interactive: a user can input experimental results (e.g., Step 2 failed due to dimerization), and the model immediately uses this negative feedback to retrain or adjust its pathfinding, improving robustness in real-time research cycles.

Sources

Related papers