What Do CAE Simulation Agents Really Need Beyond a Generic Harness?

arXiv:2609.03718 · cs.CE, cs.CL, physics.comp-ph · Submitted 2026-09-03 · Read on arXiv

cs.CE, cs.CL, physics.comp-ph

Submitted: 2026-09-03

Updated: 2026-09-03

Code: https://github.com/openai/codex

Terminology

• Benchmark — 65× in this paper · explained in 2 episodes
• Ablation — 19× in this paper · explained in 1 episode
• Scaffolding — 13× in this paper · explained in 3 episodes
• RAG — 8× in this paper · explained in 3 episodes
• Consistency — 5× in this paper · explained in 3 episodes
• LLM judge — 5× in this paper · explained in 2 episodes
• Multi-Agent System — 5× in this paper · explained in 5 episodes
• Agent Harness — 4× in this paper · explained in 1 episode
• Trace — 4× in this paper · explained in 2 episodes
• Fine-Tuning — 3× in this paper · explained in 5 episodes
• Multi-Agent Framework — 3× in this paper · explained in 3 episodes
• Failure Modes — 2× in this paper · explained in 3 episodes
• Large Language Model (LLM) — 2× in this paper · explained in 3 episodes
• Agentic Framework — 1× in this paper · explained in 6 episodes
• Core — 1× in this paper · explained in 2 episodes
• Failure Taxonomy — 1× in this paper · explained in 2 episodes
• Fidelity — 1× in this paper · explained in 2 episodes
• Human-in-the-Loop — 1× in this paper · explained in 3 episodes
• Iterative Refinement — 1× in this paper · explained in 3 episodes
• Multi-agent Collaboration — 1× in this paper · explained in 2 episodes
• Multi-hop Reasoning — 1× in this paper · explained in 3 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Retrieval-Augmented Generation (RAG) — 1× in this paper · explained in 16 episodes
• Verification — 1× in this paper · explained in 2 episodes
• alpha — 1× in this paper · explained in 1 episode

Sources

• SimBench: A Framework for Evaluating and Diagnosing LLM-Based Digital-Twin Generation for Multi-Physics Simulation
• OptMetaOpenFOAM: Large Language Model Driven Chain of Thought for Sensitivity Analysis and Parameter Optimization based on CFD
• MetaOpenFOAM: an LLM-based multi-agent framework for CFD
• MetaOpenFOAM 2.0: Large Language Model Driven Chain of Thought for Automating CFD Simulation and Post-Processing
• ALL-FEM: Agentic Large Language models Fine-tuned for Finite Element Methods
• Fine-tuning a Large Language Model for Automating Computational Fluid Dynamics Simulations
• FEABench: Evaluating Language Models on Multiphysics Reasoning Ability
• RExBench: Can coding agents autonomously implement AI research extensions?
• MechAgents: Large language model multi-agent collaborations can solve mechanics problems, generate new data, and integrate knowledge
• FeaGPT: an End-to-End agentic-AI for Finite Element Analysis
• CFDLLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamics
• AgenticSciML: Collaborative Multi-Agent Systems for Emergent Discovery in Scientific Machine Learning
• Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets
• Confucius Code Agent: Scalable Agent Scaffolding for Real-World Codebases
• Integrating Large Language Models for Automated Structural Analysis
• Automating Structural Engineering Workflows with Large Language Model Agents
• VASP Agent: An Agentic Framework for Autonomous First-principles Calculations
• SimuAgent: An LLM-Based Simulink Modeling Assistant Enhanced with Reinforcement Learning
• A Preliminary Assessment of Coding Agents for CFD Workflows
• Dive into Claude Code: The Design Space of Today's and Future AI Agent Systems

Related papers

• Constrained Sensing and Reliable State Estimation with Shallow Recurrent Decoders on a TRIGA Mark II Reactor
• Evidence-Unit Fairness and the Limits of Query-Adaptive Sparse-Dense Fusion in Financial Document Retrieval
• Chemical Chain-of-Thought Functions as a Hallucination-Prone Molecular Scratchpad
• Lightweight Adaptation of EEG Foundation Models for Stroke Motor Imagery Decoding: Domain Shift and Subject-Level Robustness
• RetroDFM-R: Reasoning-Driven Retrosynthesis Prediction with Large Language Models via Reinforcement Learning
• Wildfire Suppression: Complexity, Models, and Instances