PertMind: Eliciting Emergent Biological Reasoning in LLM via Reinforcement Learning on Cellular Perturbation Data

Cellular perturbation atlases can be reorganized as reinforcement-learning environments where measured gene responses provide computable rewards for biological reasoning.

How it works

1. PertMind treats a query as a triplet (c, d, g), where c is a cell line, d is the small-molecule perturbation identity, and g is the target gene. The task label space Y is defined as Up, Down, or No based on computable differential expression statistics from precomputed atlas data.

2. The policy samples a reasoning trajectory (z) conditioned on the query and assembled biological context (K(x)). This trajectory includes a structured final prediction (yˆg) and must adhere to specific structural requirements.

3. The training pipeline involves three stages: first, construction of a perturbation-derived supervision corpus from Tahoe-100M statistics; second, knowledge-augmented, on-policy trajectory sampling followed by one epoch of supervised fine-tuning (SFT) to establish an initialization policy (πθSFT); and third, pathway-supervised Group Relative Policy Optimization (GRPO) for principal optimization.

Reinforcement Learning Signals

The reinforcement learning objective is a composite reward function R(oi) designed to guide the model toward biologically plausible reasoning by optimizing three complementary signals:

1. A gene-level reward (Rgene): This scores the experimentally observed Up, Down, or No endpoint (Rgene = 1[ˆyg,i = yg]).

2. A pathway-level reward (Rpw): This scores structured pathway-direction predictions derived from transcriptional response summaries. It is computed using a condition-level pathway-response proxy label (yP,d˜), which summarizes the coordinated transcriptional response of non-target members under a single experimental condition.

3. A format reward (Rfmt): This binary indicator that validates required fields are present, labels use legal vocabulary, and exactly one terminal answer is parseable.

The total reward is defined as R(oi) = Rgene + λpwmpwRpw + λfmt Rfmt, where auxiliary weights satisfy λpw > 0, λfmt > 0, and λpw +λfmt < 1. The GRPO stage uses a standardized advantage (Aˆi) to compute the group reward (R(oi)), which is then scaled by a confidence weight (wconf) applied uniformly across the parameter update contributed by that triplet.

Knowledge-Augmented Initialization

To provide a stable starting point for reinforcement learning, PertMind first constructs a small corpus of trusted, model-generated trajectories and uses them for one epoch of SFT. This process involves:

1. A retrieval module assembling biological context K(x) from knowledge graphs (PubChem, DrugBank, UniProt, etc.) and outcome-stratified supporting cases (S(x)).

2. Sampling candidate trajectories (z˜m) conditioned on K(x) using the Qwen3-4B Base parameters.

3. Applying strict trusted-trajectory checks to retain only trajectories that satisfy five criteria: matching the final label, following required structure, avoiding access to hidden expression values, and containing a complete reasoning chain.

Cross-Scale Representation Learning

PertMind generates a reusable biological language model policy that transfers across various tasks by creating hierarchical representations:

1. It first produces a functional profile (pg) for each gene g using text-embedding-ada-002, which is then mapped to a task-specific projection (ug).

2. These gene embeddings are used to compose expression-weighted cell embeddings (z(t)i), where expression data determines the weighting of the gene profiles.

3. Cell embeddings are aggregated into donor embeddings (h(t)d) using an attention-based multi-instance learning module, which is then passed to a donor-level classifier. This hierarchy supports molecular annotation, cellular perturbation modeling, and tumor-state reference mapping across different scales.

Emergent Capabilities and Transfer

PertMind demonstrates emergent biological reasoning by transferring its learned strategies across tasks absent from its post-training objective:

1. It improved response inference in unseen cellular contexts while retaining general language capabilities.

2. It transferred to reverse perturbation identification, double-perturbation reasoning, phenotypic-screen prioritization, and biological process interpretation without task-specific post-training.

3. In reverse perturbation condition inference, it approached the performance of a task-specialized model (CellNavi) on primary T-cell perturbations without direct optimization for that task.

4. It improved ranking in double-perturbation tasks by better handling compositional reasoning, suggesting that the reinforcement learning selects policies integrating all four factors (drug identity, gene identity, pathway propagation, and cellular context).

As a fastidious researcher, I have analyzed the PertMind paper and identified several specific, high-impact improvements that can be made to existing Large Language Models (LLMs) by integrating this perturbation-derived reinforcement learning (RL) framework.

Here are the specific improvements and the resulting capabilities of an improved AI system:

---

)

1. Improvements to LLM Reasoning via PertMind:

2. Enhanced Biological Strategy Transfer Across Tasks:

3. Creation of Scalable, Experimentally Grounded Reasoning Environments:

4. Development of Multiscale, Hierarchical Biological Representations:

) 1. Improvements to LLM Reasoning via PertMind:

The core improvement is transforming LLMs from mere knowledge repositories into active reasoning agents by training them on the correctness of experimental outcomes rather than just text generation quality.

• A new RL objective (PertMind) is introduced that optimizes a policy against a composite reward structure:

R = Rgene + λpw mpw Rpw + λfmt Rfmt.

• This allows the model to learn to select reasoning trajectories that not only produce the correct final gene label (Rgene) but also exhibit biologically consistent intermediate pathway predictions (RpW) and adhere to required output formats (RFmt).

• The optimization uses Group Relative Policy Optimization (GRPO), which standardizes advantages across groups, ensuring updates are based on relative quality rather than absolute reward scale.

• A trusted trajectory initialization phase using Supervised Fine-Tuning (SFT) on model-generated, trusted reasoning examples provides a stable starting point for RL, preventing the policy from diverging during optimization.

• The system is trained only on forward perturbation-response prediction, but the resulting policy retains general language capabilities because it is constrained by a KL penalty toward this trusted SFT reference model.

The improved AI system can perform:

• Predicting the direction (Up/Down/No) of a target gene's transcriptional response under novel cell line and small-molecule conditions with high accuracy, exceeding generic pretraining performance.

• Generating auditable, evidence-grounded mechanistic narratives linking perturbations to pathway propagation.

--- 2. Enhanced Biological Strategy Transfer Across Tasks:

The RL framework is designed not just for the initial task (perturbation response) but for broad transfer of learned strategies to related, yet distinct, biological reasoning tasks.

• The trained policy demonstrates emergent capabilities in tasks absent from its direct post-training objective:

• Reverse perturbation identification (inferring the condition that caused an observed cell state transition).

• Double perturbation reasoning and phenotypic screen prioritization.

• Biological process interpretation (naming gene sets correctly).

The improved AI system can perform:

• Inverting experimental data: Given two cell states, it can rank candidate perturbations that could have caused the observed transition.

• Prioritizing candidate genes for phenotypic screens based on learned mechanistic hypotheses derived from perturbation data.

• Assigning accurate biological process names to gene sets with high lexical and semantic agreement with curated databases.

--- 3. Creation of Scalable, Experimentally Grounded Reasoning Environments:

The system leverages public, large-scale experimental atlases (like Tahoe-100M) as training environments instead of relying on expensive human curation.

• The input data (cell line, drug ID, gene target) is treated as a structured query triplet. The model learns to navigate this space using precomputed differential expression statistics as computable rewards.

• The reward interface is fully automated: the system automatically scores every eligible triplet from the atlas without human annotation.

The improved AI system can perform:

• Rapid inference across vast experimental datasets (e.g., testing thousands of drug-gene combinations) by using the atlas as a scalable reinforcement learning environment.

• Generating biological profiles that serve as transferable representations across different scales (molecular, cellular, and donor levels).

--- 4. Development of Multiscale, Hierarchical Biological Representations:

The system generates novel, structured representations that bridge the gap between symbolic biological knowledge and continuous data models.

• A hierarchical representation is constructed:

• Gene profiles are generated from text embeddings (e.g., Ada-002) and projected into a task-specific interface space (512 or 2048 dimensions).

• These gene embeddings are then aggregated expression-weighted to form cell embeddings.

• Cell embeddings are further aggregated via an attention module into donor embeddings, which are used for pan-cancer reference mapping.

The improved AI system can perform:

• Creating reusable molecular/cellular/donor representations that allow downstream models (e.g., tumor state classifiers) to leverage perturbation knowledge without needing complete re-training on the raw data.

• Serving as a plug-and-play biological context source for other specialized executive models, reducing reliance on repeated external database access for specific biological queries.