Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attacks
cs.CR
Submitted: 2024-09-12
Updated: 2025-11-25
Comments: 17 pages, 1 figure
Journal ref: Eureka 1, no. 2 (2026) 9-39
DOI: 10.63336/Eureka.46
Code: https://github.com/Huffon/factsumm
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: Large Language Models (LLMs) demonstrate impressive capabilities across various fields, yet their increasing use raises critical security concerns.
Terminology
Abstract
Large Language Models (LLMs) demonstrate impressive capabilities across various fields, yet their increasing use raises critical security concerns. This article reviews recent literature addressing key issues in LLM security, with a focus on accuracy, bias, content detection, and vulnerability to attacks. Issues related to inaccurate or misleading outputs from LLMs is discussed, with emphasis on the implementation from fact-checking methodologies to enhance response reliability. Inherent biases within LLMs are critically examined through diverse evaluation techniques, including controlled input studies and red teaming exercises. A comprehensive analysis of bias mitigation strategies is presented, including approaches from pre-processing interventions to in-training adjustments and post-processing refinements. The article also probes the complexity of distinguishing LLM-generated content from human-produced text, introducing detection mechanisms like DetectGPT and watermarking techniques while noting the limitations of machine learning enabled classifiers under intricate circumstances. Moreover, LLM vulnerabilities, including jailbreak attacks and prompt injection exploits, are analyzed by looking into different case studies and large-scale competitions like HackAPrompt. This review is concluded by retrospecting defense mechanisms to safeguard LLMs, accentuating the need for more extensive research into the LLM security field.
Sources
- A Survey of Large Language Models
- Large Language Models: A Survey
- Generate rather than Retrieve: Large Language Models are Strong Context Generators
- GenRewrite: Query Rewriting via Large Language Models
- Gemma: Open Models Based on Gemini Research and Technology
- Large Multimodal Agents: A Survey
- LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents
- Towards End-to-End Embodied Decision Making via Multi-modal Large Language Model: Explorations with GPT4-Vision and Beyond
- VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks
- Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
- Generating Attractive and Authentic Copywriting from Customer Reviews
- CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents
- AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments
- AutoFlow: Automated Workflow Generation for Large Language Model Agents
- TaskBench: Benchmarking Large Language Models for Task Automation
- Evaluating LLM -- Generated Multimodal Diagnosis from Medical Images and Symptom Analysis
- Gemini Goes to Med School: Exploring the Capabilities of Multimodal Large Language Models on Medical Challenge Problems & Hallucinations
- M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models
- GPT-4 Enhanced Multimodal Grounding for Autonomous Driving: Leveraging Cross-Modal Attention with Large Language Models
- RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills
- Token Composition: A Graph Based on EVM Logs