Do Perturbation Predictors Learn Gene Interactions? Diagnosing Extrapolation in Agentic Screening

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Do Perturbation Predictors Learn Gene Interactions? Diagnosing Extrapolation in Agentic Screening".

Jane: The gist The difference between model prediction and an experimentally grounded additive reference, called additive disagreement (AD),

Tom: First, who's behind it and why it matters.

Title and authors: Tom: We talked about the title and authors, and the authors are Ziyuan Gao from the Department of Computer Science and Technology at the University of Cambridge, who set out to diagnose extrapolation in agentic screening.

Jane: The paper is really focusing on how we can tell when a model is making a big guess about an interaction effect it hasn't seen before.

Lu: They introduce this concept called additive disagreement or AD, which is essentially the difference between what the model predicts for a pair and what you get just by adding up their known single-gene effects.

Meng: So, if we put that in plain English, AD measures how much extra interaction effect the model introduces for a combination it hasn't observed. It directly targets that extrapolation component.

Tom: That’s right, and it’s not just about input novelty like kNN distance; AD directly targets the component that requires compositional generalization, which is what we think is important here.

Jane: And the paper sets up a comparison between this AD signal and other reliability signals like kNN distance and Mahalanobis distance to see which ones are actually better under compositional shift.

Lu: They compare it across eight different prediction methods spanning six modeling paradigms, including Ridge regression, MLP, Transformer, scGPT based predictors, and GEARS.

Tom: The big finding there is that these specialized neural predictors don't consistently beat the simple additive baseline on unseen gene combinations across all these models.

Jane: But the real impact comes when you look at what AD tells us about prediction errors; predictions with larger deviations from that additive reference tend to have larger errors.

Meng: That’s where it gets useful for deployment, because if a model has a high AD value, we know it’s asserting a strong interaction correction for something new and unsupported.

Tom: So, the paper is essentially saying that the source of error isn't just failing to model additive effects; it's introducing interaction corrections without any transferable evidence.

Jane: And that leads us into how they propose we can use this AD signal to improve our screening process.

The paper's summary: Tom: The paper summarizes the method by decomposing each model’s output into a component supported by observed single-gene effects and a residual interaction term that the model extrapolates without direct training signal.

Jane: This residual, which they call additive disagreement, AD, quantifies how strongly a prediction departs from an experimentally grounded additive reference using this formula: AD(A, B) equals one over G times the L1 norm of the difference between the model's prediction f theta of A and B and just adding up delta A plus delta B.

Lu: So, G is just the number of evaluated genes, and that quantity AD directly targets that component that requires compositional generalization with a clear biological interpretation as the mean magnitude of the model-implied departure from additivity.

Meng: This is different from generic uncertainty measures because kNN distance and Mahalanobis distance measure input novelty; they see if the genes themselves are new, but AD sees if their interaction is new.

Tom: And they evaluate this decomposition across eight prediction methods, comparing AD with eight alternative reliability signals spanning embedding-based, output-based, and ensemble-based approaches to see what works best.

Jane: The summary shows that additive disagreement reliably identifies high-error predictions when compared to other signals like prediction magnitude and gene-wise variance.

Lu: They found that additive disagreement is the most consistent indicator of prediction failure across all evaluated compositional settings, even when moving between different experimental contexts like Norman and Joung–Zhang.

Tom: That consistency is what makes it so compelling for use in an agentic screening setting where you need a reliable signal to decide whether to trust a prediction or not.

Jane: So, the paper summarizes that AD is the most consistent ranking signal across settings, which suggests it’s robust against shifts in how the data was organized.

The paper's improvements: Tom: The paper points out a couple of ways they can improve or use this method differently to make it more practical for real-world screening.

Jane: They suggest using additive disagreement specifically for selective abstention, which means replacing high-risk model predictions with the additive reference when AD is too large.

Lu: They found that by using AD for selective abstention, the system can reduce the mean absolute error by twelve percent, nineteen percent, and even thirty-eight percent at different thresholds on the Norman dataset.

Meng: That reduction in MAE means we’re not just flagging bad predictions; we’re actually making a measurable improvement in how accurate our screening decisions become when we use this method.

Tom: And they also found that for GEARS and GEARS+scGPT, routing through this AD signal substantially reduces their errors but generally still leaves them slightly worse than always using the additive reference.

Jane: So, even when we use the AD signal to guide us away from a prediction, it doesn't always beat just sticking with the simple additive reference completely.

Lu: They also found that in GEARS predictions, there are two recurring behaviors: either they produce large interaction corrections that move predictions far from the additive reference, or the corrections are smaller but still introduce unnecessary deviations.

Meng: The gene-level analysis showed that GEARS assigns nine point seven times larger corrections to the top twenty differentially expressed genes than it does to the rest of the transcriptome, suggesting these graph-based corrections often add noise instead of providing transferable interaction signal.

Tom: That points back to the conclusion that the main source of error isn't just a failure to model additive effects; it's introducing interaction corrections without transferable evidence.

Conclusion: Jane: So, to wrap up, this paper on "Do Perturbation Predictors Learn Gene Interactions? Diagnosing Extrapolation in Agentic Screening" shows that additive disagreement is the most consistent indicator of prediction failure because it directly targets the interaction-level extrapolation.

Tom: It means we can use AD to diagnose when a model asserts an interaction correction for a combination it has never observed, which helps us pinpoint exactly where the errors are coming from.

Lu: The paper shows that specialized neural predictors don't always beat simple additive baselines on unseen combinations, but the AD signal is still much better at identifying high-error predictions compared to generic uncertainty measures like kNN distance or Mahalanobis distance.

Meng: From a deployment standpoint, using this AD signal to guide decisions can reduce the mean absolute error by up to thirty-eight percent at a fifty percent abstention threshold on the Norman dataset, which is significant for optimizing our agentic screening workflow.

Tom: It’s a practical tool because it allows us to selectively replace high-risk model predictions with the additive reference, even if it doesn't always beat the absolute best performance of any single model.

Jane: This work on AD provides a solid foundation for building more reliable agentic screening pipelines by focusing on unsupported interaction extrapolation rather than just input novelty.

Lu: They point out that a limitation is that AD measures correction magnitude rather than biological correctness, and they suggest gene-level and pathway-level disagreement measures as a natural extension to this framework.

Meng: And the paper also notes that extending this framework to higher-order combinations will require richer references than just simple pairwise additivity.

Tom: So, while it doesn't solve everything, diagnosing extrapolation via additive disagreement gives us a concrete way to filter out the noise in these complex agentic screening models.