You Can Tell Who's Asking: What the Web's Questions Are Made Of, and Where They Come From

arXiv:2609.24106 · cs.CL · Submitted 2026-09-21 · Read on arXiv

cs.CL

Submitted: 2026-09-21

Updated: 2026-09-21

Comments: Accepted to the 13th Web as Corpus Workshop (WaC-13) at EMNLP 2026. 14 pages, 4 figures. Code and data: https://github.com/bodhiumlabs/tell-whos-asking

Code: https://github.com/bodhiumlabs/tell-whos-asking

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

The gist: Questions scraped from the web are used across academia and industry as a proxy for what people want to know.

Terminology

Abstract

Questions scraped from the web are used across academia and industry as a proxy for what people want to know. Across QA training data, retrieval benchmarks, and content strategy, questions on a page are assumed to reflect human intent. We test this assumption at scale by extracting 13.4B question occurrences across 110 FineWeb snapshots (2013-2025), and report three findings. First, you can tell who is asking: provenance (the host/page of questions) leaves a signal in question form, and a logistic model can separate genuine user questions from templated/manufactured ones at AUC 0.725 via length and surrounding context rather than question type, though only 0.554 against commerce FAQ writing. Second, question frequency does not measure demand: the most-frequent questions are boilerplate/templated (over 70% of the top thousand), so occurrence counts measure how often a string was published and not how often it was asked. Third, over twelve years the genuine share of occurrences fell by 79% (42-56% after controlling for crawl composition), with question length and context decreasing. We present the first diachronic, occurrence-level measurement of web question provenance, and find the crawlable web's questions have shifted from being asked by humans toward manufactured for machines to read.

Sources

Related papers