AI's Capability in Assisting Scientific Research in Physics, Astrophysics, and Cosmology II: Project Planning and Proposal Evaluation

arXiv:2607.25881 · cs.CL, astro-ph.CO, astro-ph.IM, cs.HC, gr-qc · Submitted 2026-07-28 · Read on arXiv

Jia Liu, Veena Krishnaraj, Kateryna Vovk, Kosuke Aizawa, Adrian E. Bayer, Linda Blot, Jessica Cowell, Suyog Garg, Jonathan Grée, Anamaria Hell, Ben Horowitz, Masaya Ichikawa, Kanyuni Iemoto, Keigo Kondo, Zacharie Lorsin, Kevin McCarthy, Jamie Robinson, Miguel Ruiz-Granda, Leander Thiele, Ievgen Vovk, Mingshen Zhou

cs.CL, astro-ph.CO, astro-ph.IM, cs.HC, gr-qc

Submitted: 2026-07-28

Comments: 16 pages, 4 figures

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• AGN — 16× in this paper · explained in 1 episode
• Alignment — 4× in this paper · explained in 1 episode
• Accuracy — 3× in this paper · explained in 1 episode
• Large Language Models (LLMs) — 3× in this paper · explained in 6 episodes
• Non-Gaussianity — 3× in this paper · explained in 2 episodes
• Pulsar Timing Array — 3× in this paper
• Ablation — 2× in this paper · explained in 1 episode
• Benchmark — 2× in this paper · explained in 2 episodes
• Trace — 2× in this paper · explained in 2 episodes
• Weak Lensing — 2× in this paper · explained in 2 episodes
• AI Agents — 1× in this paper · explained in 2 episodes
• Anisotropy — 1× in this paper · explained in 3 episodes
• Anti-correlation — 1× in this paper · explained in 3 episodes
• Artificial Intelligence (AI) — 1× in this paper · explained in 2 episodes
• Clustering — 1× in this paper · explained in 2 episodes
• Core — 1× in this paper · explained in 2 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• In-Context Learning — 1× in this paper · explained in 4 episodes
• Iterative Refinement — 1× in this paper · explained in 3 episodes
• N-body simulations — 1× in this paper · explained in 2 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• World Premier International Research Center Initiative — 1× in this paper

Sources

• AstroLLaMA: Towards Specialized Foundation Models in Astronomy
• Mitigating the Bias of Large Language Model Evaluation
• DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
• The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
• LLM Evaluators Recognize and Favor Their Own Generations
• Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
• Self-Preference Bias in LLM-as-a-Judge
• Can large language models provide useful feedback on research papers? A large-scale empirical analysis
• Can AI-Generated Text be Reliably Detected?
• Assisting Research Proposal Writing with Large Language Models: Evaluation and Refinement
• Can Large Language Models Evaluate Grant Proposal Quality? Revisiting the Wenner{\aa}s and Wold Peer Review Data
• Evaluating LLM-Based Grant Proposal Review via Structured Perturbations
• The Denario project: Deep knowledge AI agents for scientific discovery
• LLMs with in-context learning for Algorithmic Theoretical Physics

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering