LLM-Generated or Human-Written? Comparing Review and Non-Review Papers on ArXiv

arXiv:2601.17036 · cs.DL, cs.CL, cs.CY · Submitted 2026-01-19 · Read on arXiv

cs.DL, cs.CL, cs.CY

Submitted: 2026-01-19

Updated: 2026-09-09

Comments: Accepted to Findings of EMNLP 2026

Code: https://github.com/yanaiela/llm-review-arxiv

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• alpha — 40× in this paper · explained in 1 episode
• Computer Science — 20× in this paper
• Accuracy — 3× in this paper · explained in 1 episode
• Adam — 3× in this paper · explained in 2 episodes
• Core — 3× in this paper · explained in 2 episodes
• Accountability — 2× in this paper · explained in 2 episodes
• F1 Score — 2× in this paper · explained in 3 episodes
• Benchmark — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Perplexity — 1× in this paper · explained in 3 episodes
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• GPT-4 Technical Report
• gpt-oss-120b & gpt-oss-20b Model Card
• The Llama 3 Herd of Models
• Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation
• Technical Report on the Pangram AI-Generated Text Classifier
• GPT-4o System Card
• Large language models reshape the language of science
• The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
• Can AI-Generated Text be Reliably Detected?
• The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates
• Gemma 2: Improving Open Language Models at a Practical Size
• EditLens: Quantifying the Extent of AI Editing in Text
• The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search
• Advancing the Scientific Method with Large Language Models: From Hypothesis to Discovery

Related papers

• MTRACE: Multilingual Retrieval-Augmented Generation for Temporally Diverse Text Corpora
• Eleven Years of BRACIS: A Meta-Scientific Study of the Brazilian Conference on Intelligent Systems
• Evidence-Based Scientific Question Discovery: A Framework with Historical Backtesting