Co-folding with a Soup of Representations

summary

The gist

Co-folding models have rapidly advanced, but no single model consistently performs best across all biomolecular complexes, raising the question of whether independently trained co-folding models

In short

SoupFold improves structure prediction by learning simple mappings between co-folding models' internal representations. It treats one model as a base and others as teachers, transferring their learned patterns to enhance the base model's representation at inference time. This method achieves state-of-the-art results and recovers structures when individual models fail.

Key concepts

Co-folding Models: These are different AI models, like AlphaFold3 or ESMFold2, that all share a similar structure (Pairformer trunk). They predict protein structures but are trained on different recipes, meaning they often capture different structural patterns.
Representation Spaces: This refers to the internal mathematical spaces where each co-folding model stores the information about a complex. SoupFold learns how to translate or map these different spaces so that information from one model can be understood by another.
Teacher-to-Base Map: This is a learned function used during inference that translates the representation generated by a teacher model into the mathematical space of the base model. This allows the base model to incorporate useful structural insights from other models.
Complementary Information: This means that different models capture different aspects of a structure. Even if one model fails on a specific complex, its unique representation might hold information that is crucial for successful prediction when combined with another model's representation.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Retrieval-Augmented Generation (RAG) — explained in 16 episodes
Federated Learning — explained in 15 episodes
Interpretability — explained in 15 episodes
Sample Complexity — explained in 15 episodes
Generalization — explained in 13 episodes

This episode discusses

Co-folding with a Soup of Representations · Paper Radio

The paper

Co-folding with a Soup of Representations · Read on arXiv

Hyosoon Jang, Taewon Kim, Sungsoo Ahn

KAIST

Co-folding models such as AlphaFold3, Protenix, ESMFold2, and OpenDDE have advanced rapidly, yet no single model consistently performs best across all biomolecular complexes. In this paper, we show that their pair representations encode complementary information that can be transferred across models to improve structure prediction. We introduce SoupFold, which combines pair representations from multiple co-folding models in a common representation space and generates structures from the combined representation. Importantly, SoupFold does not retrain the co-folding models and learns only simple mappings to transfer representations across models. We evaluate SoupFold on antibody-antigen, protein-protein, protein-ligand, molecular glue, GPCR, and oligomeric complex prediction using AlphaFold3, Protenix, ESMFold2, and OpenDDE. By combining representations across models, SoupFold improves over individual co-folding models across the considered benchmarks.

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: Today's paper: "Co-folding with a Soup of Representations".

Marcus: Co-folding models have rapidly advanced, but no single model consistently performs best across all biomolecular complexes,

Ines: First, who's behind it and why it matters.

Paper summary: Ines: So, summarizing what we've discussed about "Co-folding with a Soup of Representations," the paper argues that independently trained co-folding models encode complementary information that can be effectively transferred to improve structure prediction at inference time without retraining them.

Marcus: They specifically introduced this mechanism called SoupFold, which learns lightweight mappings between the representation spaces of different models, allowing it to aggregate these representations into a base model's representation before the final structure generation step.

Yuki: The significance lies in the fact that this aggregation successfully achieves state-of-the-art performance on both protein-protein and protein-ligand prediction tasks when tested against four existing co-folding models.

Ines: It really speaks to how different AI models, despite their similar foundational architectures, can still specialize in capturing different structural patterns that are mutually beneficial when combined.

Marcus: The implications for the field are that we might move toward a system where we don't rely on one monolithic model, but rather a curated soup of representations from several specialized models for the most accurate predictions.

Yuki: This suggests that understanding complex biological structures requires looking beyond the output of any single prediction tool to appreciate the broader landscape of structural possibilities encoded across many different AI approaches.

Conclusion: Ines: So, to wrap up this discussion on "Co-folding with a Soup of Representations," the core idea is that combining representations from different co-folding models helps predict protein structures better than any single model alone.

Marcus: Exactly, and I think what's really interesting is how they managed to do this without having to retrain all those massive base models, which saves a ton of computational power.

Yuki: From a population genetics standpoint, it’s fascinating that these distinct predictive capabilities—what we might call different "alleles" or structural patterns—can be aggregated into a more robust prediction for the entire species' protein landscape.

Ines: I'm curious about the authors themselves; what do they think is the big picture implication of finding this representation transfer mechanism?

Marcus: The authors suggest that it opens up a new way to utilize existing AI infrastructure, rather than having to build entirely new predictors from scratch for every complex.

Yuki: That implies a future where we can rapidly assess structural data across vast biological datasets by just querying the aggregated knowledge of these diverse models.

Ines: So, if we simplify it, the paper is basically showing us how to make a smarter prediction engine by letting different AI brains talk to each other during the final calculation.

Marcus: Right, and that means for genomics data scientists like myself, it's a powerful tool because we can use these combined predictions as a cross-validation check for our statistical models.

Yuki: It suggests that the underlying biological truth isn't locked in one single representation but is distributed across multiple, specialized views of the same complex.

Ines: That distributed view is what really intrigues me—does this mean we’re finally starting to see how structural information is encoded in a multi-layered way?

Marcus: It certainly does, and I think the next big thing will be seeing if we can adapt this representation mapping idea to other types of biological data where models aren't as well-established.

Yuki: That would really show how this principle applies beyond just protein folding, connecting it to how we understand evolutionary constraints on molecular shapes.

More episodes

← Home