bioMoR: Biology-Guided Mixture-of-Recursions for Effective Genomic Learning
Koushik Howlader, Tirtho Roy, Md Tauhidul Islam, Wei Le
cs.AI, cs.LG
Submitted: 2026-08-10
Updated: 2026-08-11
License: http://creativecommons.org/licenses/by/4.0/
The gist: Transformer models for high-dimensional omics analysis process thousands of genes or pathways, although only a subset requires deep computation.
Terminology
Abstract
Transformer models for high-dimensional omics analysis process thousands of genes or pathways, although only a subset requires deep computation. Mixture-of-Recursions (MoR) improves efficiency through adaptive token-choice or expert-choice routing. We propose bioMoR, which, to the best of our knowledge, is the first framework to apply MoR to gene-level and pathway-level learning. Our contributions include identifying three locations for integrating structured biological knowledge within an MoR backbone: graph-based information sharing refines token embeddings, a structural bias guides self-attention toward biologically related tokens, and a graph-aware router uses neighborhood information to determine each token's recursion depth. These techniques are centered on our insight that additional knowledge of token interaction can effectively help models construct embeddings and select which tokens should be learned more deeply. Across eight benchmarks spanning diverse omics data types and evaluated under a unified five-fold cross-validation protocol, bioMoR improves average macro-F1 by 8.2 percentage points and balanced accuracy by 7.1 percentage points over the strongest biology-agnostic MoR baseline while using 75 percent fewer parameters and up to 58 percent fewer FLOPs than a non-recursive Transformer. The selected marker genes or pathways provide biological interpretability, while their token-specific recursion depths reveal how computation is allocated.
Sources
- Understanding intermediate layers using linear classifier probes
- Adaptive Computation Time for Recurrent Neural Networks
- Graph Transformer-Based Pathway Embedding for Cancer Prognosis
- Integrating gene regulatory priors into Transformer attention with scTransformer for interpretable scRNA-seq analysis
- Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
- Linformer: Self-Attention with Linear Complexity
- scBiGNN: Bilevel Graph Representation Learning for Cell Type Classification from Single-cell RNA Sequencing Data
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection