The Illusion of Secure LLM Code: Closing the Security Gap via Iterative Reprompting
Ishpuneet Singh, Shreyas Mahajan, Gurjot Singh, Maninder Singh
cs.CR, cs.AI, cs.HC, cs.LG, cs.MA
Submitted: 2026-07-26
Code: https://github.com/ipa-lab/hackingBuddyGPT
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large Language Models (LLMs) are increasingly integrated into software development workflows, yet their ability to autonomously generate secure authentication code remains uncertain.
Terminology
Abstract
Large Language Models (LLMs) are increasingly integrated into software development workflows, yet their ability to autonomously generate secure authentication code remains uncertain. This paper evaluates the security architecture of authentication systems generated by five prominent AI coding assistants through a bi-modal assessment framework combining static code analysis and dynamic penetration testing, mapped to NIST SP 800-63B guidelines. The study examines model behavior across four prompting strategies Basic, Secure, NIST-Based, and Reprompting to reflect varying levels of developer guidance. Empirical results demonstrate that code generated from functional or generically secure prompts consistently omits critical protections, particularly concerning brute-force resistance, session management, and robust password handling. While providing explicit, single-shot NIST context significantly improves compliance, the findings reveal that this remains structurally inadequate. Instead, iterative Reprompting: forcing models into a contextual self-auditing loop is strictly required to achieve a comprehensive, defense-in-depth security architecture. Ultimately, this study proves that current AI coding assistants do not produce secure-by-default applications, dictating that enterprise deployments must transition from single-shot prompt engineering to continuous, standards-driven verification pipelines.
Sources
- Evaluating Large Language Models Trained on Code
- Rethinking the Evaluation of Secure Code Generation
- From Solitary Directives to Interactive Encouragement! LLM Secure Code Generation by Natural Language Prompting
- Can You Really Trust Code Copilots? Evaluating Large Language Models from a Code Security Perspective
- AutoSafeCoder: A Multi-Agent Framework for Securing LLM Code Generation through Static Analysis and Fuzz Testing
- Security Degradation in Iterative AI Code Generation -- A Systematic Analysis of the Paradox
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills
- Token Composition: A Graph Based on EVM Logs