OpenAI4S: Code as Action, Science as Sessions
cs.AI, cs.CL, cs.SE
Submitted: 2026-09-14
Updated: 2026-09-14
Code: https://github.com/PKU-YuanGroup/OpenAI4S
License: http://creativecommons.org/licenses/by/4.0/
The gist: AI co-scientists could accelerate computational research, but over a long-running study the workflow also has to stay inspectable, resumable and reproducible, which requires persistent computational
Terminology
Abstract
AI co-scientists could accelerate computational research, but over a long-running study the workflow also has to stay inspectable, resumable and reproducible, which requires persistent computational state and provenance. Here we present OpenAI4S, an open-source scientific research agent built around the principle of Code as Action, Science as Sessions. OpenAI4S combines a persistent computing runtime with research-session management: orchestration is handled through structured tool calls, while scientific actions are represented as complete code cells executed in persistent Python and R kernels. An append-only Action Ledger, per-cell execution records, versioned artifacts, environment records, and workspace checkpoints preserve how results were produced and support session recovery, branching, and extension. Configurable sandboxing, permission controls, and code and trajectory screening provide complementary safeguards. We evaluate OpenAI4S on 36 research scenarios spanning retrosynthesis, molecular dynamics, protein binder design, protein mutation, catalyst screening, and mineral spectroscopy, measuring scientific task accuracy, workflow completeness, and reproducibility of the resulting repositories. OpenAI4S achieves an overall score of 7.83, compared with 5.7--6.4 for a general-purpose coding harness evaluated with three frontier models, with the largest gains on long-horizon and computation-intensive workflows. These results suggest that integrating persistent execution with session-level provenance can improve the reliability of AI-assisted scientific workflows. Environment specification and full rerunnability remain weak for every evaluated system, ours included, so reproducibility is still an open problem for scientific agents. The system is available under the MIT license at github.com/PKU-YuanGroup/OpenAI4S.
Sources
- EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery
- The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
- ReAct: Synergizing Reasoning and Acting in Language Models
- Executable Code Actions Elicit Better LLM Agents
- TaskWeaver: A Code-First Agent Framework
- Agentic reinforcement learning empowers next-generation chemical language models for molecular design and synthesis
- MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation
- Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operations
- OmniHuman: A Large-scale Dataset and Benchmark for Human-Centric Video Generation
- CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection