Scalable, Transferable Meta-network for Data Selection Requires a Different Loss (and Why the Obvious Choice is Problematic)
cs.CL, cs.AI, cs.LG
Submitted: 2026-10-01
Updated: 2026-10-01
Code: https://github.com/tatsu-lab/stanford_alpaca
Terminology
Sources
- Evaluating Large Language Models Trained on Code
- Training Verifiers to Solve Math Word Problems
- On the Difficulty of Learning a Meta-network for Training Data Selection
- What is in Your Safe Data? Identifying Benign Data that Breaks Safety
- Large-Scale Data Selection for Instruction Tuning
- Unified Data Selection for LLM Reasoning
- The Llama 3 Herd of Models
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
- GIST: Targeted Data Selection for Instruction Tuning via Coupled Optimization Geometry
- A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)
- Instruction Tuning with GPT-4
- Qwen2.5 Technical Report
- Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective
- Meta-Semi: A Meta-learning Approach for Semi-supervised Learning
- A Generalized Alternating Method for Bilevel Learning under the Polyak-{\L}ojasiewicz Condition
- LLM Data Selection and Utilization via Dynamic Bi-level Optimization
- Data-centric AI: Perspectives and Challenges
- A Survey of LLM $\times$ DATA
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering