LOBERT: Generative AI Foundation Model for Limit Order Book Messages
cs.AI
Submitted: 2025-11-16
Updated: 2026-09-08
Comments: Submission for NeurIPS 2025 GenAI in Finance Workshop
License: http://creativecommons.org/licenses/by/4.0/
The gist: Modeling the dynamics of financial Limit Order Books (LOB) at the message level is challenging due to irregular event timing, rapid regime shifts, and the reactions of high-frequency traders to
Terminology
Abstract
Modeling the dynamics of financial Limit Order Books (LOB) at the message level is challenging due to irregular event timing, rapid regime shifts, and the reactions of high-frequency traders to visible order flow. Previous LOB models require cumbersome data representations and lack adaptability outside their original tasks, leading us to introduce LOBERT, a general-purpose encoder-only foundation model for LOB data suitable for downstream fine-tuning. LOBERT adapts the original BERT architecture for LOB data by using a novel tokenization scheme that treats complete multi-dimensional messages as single tokens while retaining continuous representations of price, volume, and time. With these methods, LOBERT achieves leading performance in tasks such as predicting mid-price movements and next messages, while reducing the required context length compared to previous methods.
Sources
- TLOB: A Novel Transformer Model with Dual Attention for Price Trend Prediction with Limit Order Book Data
- TRADES: Generating Realistic Market Simulations with Diffusion Models
- Simplified State Space Layers for Sequence Modeling
- MarketGPT: Developing a Pre-trained transformer (GPT) for Modeling Financial Time Series
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection