EMRB: A Multi-Level Benchmark for Evaluating LLM Reasoning over Raw Electromagnetic Signals

arXiv:2608.24086 · cs.AI, cs.CE, cs.SE · Submitted 2026-08-25 · Read on arXiv

cs.AI, cs.CE, cs.SE

Submitted: 2026-08-25

Updated: 2026-08-25

Code: https://github.com/mingxuZhang2/EMRB

Terminology

• Benchmark — 52× in this paper · explained in 2 episodes
• Verification — 20× in this paper · explained in 2 episodes
• Verifier — 14× in this paper · explained in 2 episodes
• Accuracy — 9× in this paper · explained in 1 episode
• Consistency — 6× in this paper · explained in 3 episodes
• Large Language Model — 5× in this paper
• Calibration — 3× in this paper · explained in 7 episodes
• LLM judge — 3× in this paper · explained in 2 episodes
• Provenance — 3× in this paper · explained in 4 episodes
• Quantization — 3× in this paper · explained in 12 episodes
• Ablation — 2× in this paper · explained in 1 episode
• Core — 2× in this paper · explained in 2 episodes
• Fingerprinting — 2× in this paper · explained in 2 episodes
• Grounding — 2× in this paper · explained in 3 episodes
• International Conference on Learning Representations — 2× in this paper
• Spectral Gap — 2× in this paper · explained in 5 episodes
• Abstraction — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Conference on Knowledge Discovery and Data Mining — 1× in this paper
• Fidelity — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Large Language Model (LLM) — 1× in this paper · explained in 3 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Physical Constraints — 1× in this paper · explained in 7 episodes
• Prompt Engineering — 1× in this paper · explained in 4 episodes
• Self-consistency — 1× in this paper · explained in 2 episodes
• Sparse Attention — 1× in this paper · explained in 5 episodes
• Trace — 1× in this paper · explained in 2 episodes
• Wireless Anomaly Signal Dataset — 1× in this paper

Sources

• DeepMIMO: A Generic Deep Learning Dataset for Millimeter Wave and Massive MIMO Applications
• DeepSense 6G: A Large-Scale Real-World Multi-Modal Sensing and Communication Dataset
• Program Synthesis with Large Language Models
• Evaluating Large Language Models Trained on Code
• RadioLLM: Introducing Large Language Model into Cognitive Radio via Hybrid Prompt and Token Reprogrammings
• DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
• The Llama 3 Herd of Models
• PReD: An LLM-based Foundation Multimodal Model for Electromagnetic Perception, Recognition, and Decision
• WiSig: A Large-Scale WiFi Signal Dataset for Receiver and Channel Agnostic RF Fingerprinting
• InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks
• MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation
• Kimi K2.5: Visual Agentic Intelligence
• MiniMax Sparse Attention
• DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation
• ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
• MERLIN: Building Low-SNR Robust Multimodal LLMs for Electromagnetic Signals
• MiMo-V2-Flash Technical Report
• GLM-5: from Vibe Coding to Agentic Engineering
• RF-GPT: Teaching AI to See the Wireless World

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
• VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection