General Phrase Debiaser: Debiasing Masked Language Models at a Multi-Token Level

arXiv:2311.13892 · cs.CL, cs.AI · Submitted 2023-11-23 · Read on arXiv

cs.CL, cs.AI

Submitted: 2023-11-23

Updated: 2026-08-31

Code: https://github.com/BingkangShi/general-phrase-debiaser

Terminology

• Fine-Tuning — 6× in this paper · explained in 5 episodes
• Benchmark — 5× in this paper · explained in 2 episodes
• Embeddings — 3× in this paper · explained in 2 episodes
• General Language Understanding Evaluation — 2× in this paper
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Diffusion Model — 1× in this paper · explained in 8 episodes
• Diffusion Models — 1× in this paper · explained in 6 episodes
• Distribution Shift — 1× in this paper · explained in 5 episodes
• Jensen-Shannon Divergence — 1× in this paper
• Latent Diffusion Model — 1× in this paper · explained in 2 episodes
• Latent Diffusion Models — 1× in this paper · explained in 2 episodes
• Pruning — 1× in this paper · explained in 4 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Self-Supervised Learning — 1× in this paper · explained in 7 episodes
• Sentence Embedding Association Test — 1× in this paper
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
• DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
• You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient
• IDEAL: Influence-Driven Selective Annotations Empower In-Context Learners in Large Language Models
• HyperTime: Hyperparameter Optimization for Combating Temporal Distribution Shifts
• AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
• MathChat: Converse to Tackle Challenging Math Problems with LLM Agents
• Measuring and Reducing Gendered Correlations in Pre-trained Models
• Refined Coreset Selection: Towards Minimal Coreset Size under Model Performance Constraints
• Decoupled Weight Decay Regularization

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering