From Outliers to Topics in Language Models: Anticipating Trends in News Corpora

arXiv:2509.22030 · cs.CL · Submitted 2025-09-26 · Read on arXiv

cs.CL

Submitted: 2025-09-26

Updated: 2026-09-19

Comments: presented at ICNLSP 2025; to appear in the ACL Anthology; received the Best Full Paper Award

Code: https://github.com/evangeliazve/outliers-to-topics-icnlsp

Project page: https://maartengr.github.io/BERTopic

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

The gist: This paper examines how outliers, often dismissed as noise in topic modeling, can act as weak signals of emerging topics in dynamic news corpora.

Terminology

Abstract

This paper examines how outliers, often dismissed as noise in topic modeling, can act as weak signals of emerging topics in dynamic news corpora. Using vector embeddings from state-of-the-art language models and a cumulative clustering approach, we track their evolution over time in French and English news datasets focused on corporate social responsibility and climate change. The results reveal a consistent pattern: outliers tend to evolve into coherent topics over time across both models and languages.

Sources

Related papers