Investigating The Smells of LLM Generated Code

arXiv:2510.03029 · cs.SE, cs.AI · Submitted 2025-10-03 · Read on arXiv

cs.SE, cs.AI

Submitted: 2025-10-03

Updated: 2026-09-27

Code: https://github.com/hongzhu6129/EvaluateLLMCodeSmell

Project page: https://google.github.io/styleguide/javaguide.html2https://www.oracle.com/java/technologies/javase/codeconventionsintroduction.html3https://www.designite-tools.com/products-dj

Terminology

• Benchmark — 18× in this paper · explained in 2 episodes
• Abstraction — 13× in this paper · explained in 1 episode
• Alignment — 5× in this paper · explained in 1 episode
• Redundancy — 5× in this paper · explained in 2 episodes
• Smell Violations Per Solution — 5× in this paper
• Large Language Models (LLMs) — 3× in this paper · explained in 6 episodes
• Modularity — 3× in this paper · explained in 9 episodes
• Accuracy — 2× in this paper · explained in 1 episode
• Analyzer — 1× in this paper · explained in 1 episode
• Consistency — 1× in this paper · explained in 3 episodes
• F1 Score — 1× in this paper · explained in 3 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• Generative Model — 1× in this paper · explained in 2 episodes
• Generative Models — 1× in this paper · explained in 2 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
• Measuring Coding Challenge Competence With APPS
• Evaluating Large Language Models Trained on Code
• Exploring Code Analysis: Zero-Shot Insights on Syntax and Semantics with LLMs
• CodeBERT: A Pre-Trained Model for Programming and Natural Languages
• CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation

Related papers

• Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
• GitSkills: A Dataset of Agent Skills on GitHub
• SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
• PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
• IntentCoding: Amplifying User Intent in Code Generation
• Incentives and Outcomes in Bug Bounties