LLMs' Reshaping of People, Processes, Products, and Society in Software Development: A Comprehensive Exploration with Early Adopters

arXiv:2503.05012 · cs.SE, cs.AI, cs.HC · Submitted 2025-03-06 · Read on arXiv

cs.SE, cs.AI, cs.HC

Submitted: 2025-03-06

Updated: 2025-11-23

Terminology

• Prompt Engineering — 31× in this paper · explained in 4 episodes
• Verification — 18× in this paper · explained in 2 episodes
• LLM Integration — 10× in this paper · explained in 2 episodes
• Accuracy — 9× in this paper · explained in 1 episode
• Adaptation — 8× in this paper · explained in 1 episode
• Core — 8× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 5× in this paper · explained in 6 episodes
• Actor — 4× in this paper · explained in 3 episodes
• Generalization — 4× in this paper · explained in 13 episodes
• Generative AI — 4× in this paper · explained in 2 episodes
• Hallucination — 3× in this paper · explained in 3 episodes
• International Conference on Software Engineering — 3× in this paper
• Iterative Refinement — 3× in this paper · explained in 3 episodes
• Alignment — 2× in this paper · explained in 1 episode
• Benchmark — 2× in this paper · explained in 2 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• Data Sovereignty — 2× in this paper · explained in 2 episodes
• Frustration — 2× in this paper · explained in 2 episodes
• Provenance — 2× in this paper · explained in 4 episodes
• Adam — 1× in this paper · explained in 2 episodes
• Adaptability — 1× in this paper · explained in 1 episode
• Agentic System — 1× in this paper · explained in 2 episodes
• FA9550-21-1-0108 and the National Science Foundation — 1× in this paper
• Generalizability — 1× in this paper · explained in 6 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• RAG — 1× in this paper · explained in 3 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Retrieval-Augmented Generation (RAG) — 1× in this paper · explained in 16 episodes
• Scaffolding — 1× in this paper · explained in 3 episodes

Sources

• Evaluating Large Language Models Trained on Code
• An Empirical Study on Challenges for LLM Application Developers
• From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future
• Using AI Assistants in Software Development: A Qualitative Study on Security Practices and Concerns
• Novice Developers' Perspectives on Adopting LLMs for Software Development: A Systematic Literature Review
• From Gains to Strains: Modeling Developer Burnout with GenAI Adoption
• Exploring the Design Space of Cognitive Engagement Techniques with AI-Generated Code for Enhanced Learning
• Understanding the Characteristics of LLM-Generated Property-Based Tests in Exploring Edge Cases
• Deception in LLMs: Self-Preservation and Autonomous Goals in Large Language Models
• SafeGenBench: A Benchmark Framework for Security Vulnerability Detection in LLM-Generated Code
• LiCoEval: Evaluating LLMs on License Compliance in Code Generation
• An Exploratory Study on Upper-Level Computing Students' Use of Large Language Models as Tools in a Semester-Long Project
• ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code
• What's Wrong with Your Code Generated by Large Language Models? An Extensive Study
• SWE-bench Goes Live!
• Agentic Software Engineering: Foundational Pillars and a Research Roadmap

Related papers

• Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
• GitSkills: A Dataset of Agent Skills on GitHub
• SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
• PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
• IntentCoding: Amplifying User Intent in Code Generation
• Incentives and Outcomes in Bug Bounties