BiasGym: A Simple and Generalizable Framework for Analyzing and Removing Biases through Injection

arXiv:2508.08855 · cs.CL, cs.AI, cs.LG · Submitted 2025-08-12 · Read on arXiv

cs.CL, cs.AI, cs.LG

Submitted: 2025-08-12

Updated: 2026-09-01

Comments: Accepted at EMNLP Findings 2026

Code: https://github.com/copenlu/BiasGym

License: http://creativecommons.org/licenses/by/4.0/

The gist: Understanding biases and stereotypes encoded in the weights of Large Language Models (LLMs) is crucial for developing effective mitigation strategies.

Terminology

Abstract

Understanding biases and stereotypes encoded in the weights of Large Language Models (LLMs) is crucial for developing effective mitigation strategies. However, biased behavior is often subtle and non-trivial to isolate, even when deliberately elicited, making systematic analysis and debiasing particularly challenging. To address this, we introduce a simple, cost-effective, and generalizable framework BiasGym for reliably injecting, analyzing, and mitigating conceptual associations of biases within LLMs. BiasGym consists of two modules: Inject, which injects specific biases into the model via token-based fine-tuning while keeping the model frozen, followed by two debiasing methods that leverage these injected signals to identify and reliably suppress (Scope) or Steer the components responsible for biased behavior. Our framework enables consistent bias elicitation for better localization of bias conceptual association in the model space, supports targeted debiasing without degrading performance on downstream tasks, and generalizes to biases unseen during fine-tuning. We demonstrate the effectiveness of our proposed framework in reducing real-world stereotypes (e.g., people from Italy being `reckless drivers'), showing its utility for both safety interventions and interpretability research.

Related papers