Fairshare Data Pricing via Data Valuation for Large Language Models

arXiv:2502.00198 · cs.GT, cs.CL · Submitted 2025-01-31 · Read on arXiv

cs.GT, cs.CL

Submitted: 2025-01-31

Updated: 2025-11-19

Journal ref: Advances in Neural Information Processing Systems 38 (NeurIPS 2025)

DOI: 10.52202/085713-1119

License: http://creativecommons.org/licenses/by/4.0/

The gist: Training data is the backbone of large language models (LLMs), yet today's data markets often operate under exploitative pricing -- sourcing data from marginalized groups with little pay or

Terminology

Abstract

Training data is the backbone of large language models (LLMs), yet today's data markets often operate under exploitative pricing -- sourcing data from marginalized groups with little pay or recognition. This paper introduces a theoretical framework for LLM data markets, modeling the strategic interactions between buyers (LLM builders) and sellers (human annotators). We begin with theoretical and empirical analysis showing how exploitative pricing drives high-quality sellers out of the market, degrading data quality and long-term model performance. Then we introduce fairshare, a pricing mechanism grounded in data valuation that quantifies each data's contribution. It aligns incentives by sustaining seller participation and optimizing utility for both buyers and sellers. Theoretically, we show that fairshare yields mutually optimal outcomes: maximizing long-term buyer utility and seller profit while sustaining market participation. Empirically when training open-source LLMs on complex NLP tasks, including math problems, medical diagnosis, and physical reasoning, fairshare boosts seller earnings and ensures a stable supply of high-quality data, while improving buyers' performance-per-dollar and long-term welfare. Our findings offer a concrete path toward fair, transparent, and economically sustainable data markets for LLM.

Sources

Related papers