A Training-free Method for LLM Text Attribution

arXiv:2501.02406 · stat.ML, cs.AI, cs.CL, cs.IT, cs.LG, math.IT · Submitted 2025-01-04 · Read on arXiv

stat.ML, cs.AI, cs.CL, cs.IT, cs.LG, math.IT

Submitted: 2025-01-04

Updated: 2026-09-10

Code: https://github.com/yangluo7/CAME

License: http://creativecommons.org/licenses/by/4.0/

The gist: Verifying the provenance of text is increasingly important for firms, educational institutions, and online platforms as Large Language Models (LLMs) produce output that is nearly indistinguishable

Terminology

Abstract

Verifying the provenance of text is increasingly important for firms, educational institutions, and online platforms as Large Language Models (LLMs) produce output that is nearly indistinguishable from human-generated content. We study the problem of determining whether a given text was generated by a particular LLM while controlling the false positive rate. We model LLM-generated text as a sequential stochastic process and develop training-free statistical tests to (i) distinguish between text produced by two known sets of LLMs and (ii) determine whether text was generated by a known LLM or by a distinguishable unknown source, such as a human or another model. We prove that both Type I and Type II errors decay exponentially with text length, establish analogous guarantees for black-box access via sampling, and provide an information-theoretic lower bound showing that there exist model pairs for which no statistical test can make both errors decay faster than exponentially with text length. Numerical experiments empirically evaluate the tests in practical settings and demonstrate strong overall performance, including under many adversarial edits. Our framework provides rigorous guarantees for LLM provenance detection, with applications to content verification, institutional compliance, and misinformation mitigation.

Sources

Related papers