Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation
cs.CL
Submitted: 2026-06-16
Updated: 2026-09-21
Comments: EMNLP 2026 Main version: 28 pages, added LLM-as-judge, changed main results to reflect alpha selection based on validation set, moved previous results (alpha per layer) to appendix, and discussed differences in subsection "Using Only Best Alpha Per Layer"
Code: https://github.com/kinit-sk/steering-synth-gen
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Representation Engineering for Large-Language Models: Survey and Research Challenges
- AFRILANGTUTOR: Advancing Language Tutoring and Culture Education in Low-Resource Languages with Large Language Models
- Eliciting Latent Predictions from Transformers with the Tuned Lens
- AugGPT: Leveraging ChatGPT for Text Data Augmentation
- Using GPT-4 to Augment Unbalanced Data for Automatic Scoring
- Use Random Selection for Now: Investigation of Few-Shot Selection Strategies in LLM-based Text Augmentation for Classification
- Scaling and evaluating sparse autoencoders
- Causal Language Control in Multilingual Transformers via Sparse Feature Steering
- Beyond Translation: LLM-Based Data Generation for Multilingual Fact-Checking
- Multilingual Steering by Design: Multilingual Sparse Autoencoders and Principled Layer Selection
- Unsupervised Cross-lingual Representation Learning at Scale
- DFKI-MLT at SemEval-2026 TASK 7: Steering Multilingual Models Towards Cultural Knowledge
- CLaS-Bench: A Cross-Lingual Alignment and Steering Benchmark
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
- Style Vectors for Steering Generative Large Language Model
- Synthetic Eggs in Many Baskets: The Impact of Synthetic Data Diversity on LLM Fine-Tuning
- From Weights to Activations: Is Steering the Next Frontier of Adaptation?
- Gemma 2: Improving Open Language Models at a Practical Size
- Steering Llama 2 via Contrastive Activation Addition
- The Linear Representation Hypothesis and the Geometry of Large Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering