Automatic register identification for the open web using multilingual deep learning

This research introduces a sophisticated suite of multilingual deep learning models designed to identify diverse text varieties, or web registers (such as news reports and discussion forums), across 16 different languages. The core contribution lies in the development and validation of the Multilingual CORE corpora, which comprise over 72,000 documents meticulously annotated with a hierarchical taxonomy of 25 distinct registers. This comprehensive annotation scheme is specifically engineered to cover the entirety of the open web text landscape.

The study employs a multi-label classification approach for register identification. The resulting best model achieves an averaged F1 score of 79% across all languages. This performance level is significant, as it matches or surpasses previous studies that utilized simpler, less granular classification schemes. Furthermore, the macro F1 score of 73% with the full 25-class scheme demonstrates robust performance even when employing a fine-grained classification system, which is crucial for applications requiring detailed metadata (as highlighted by Eskelinen et al., 2024).

A critical finding relates to data quality and inherent ambiguity. The authors observe that model performance exhibits a consistent ceiling; specifically, when documents with uncertain labels are removed through data pruning, the performance dramatically increases to over 90% F1. This strongly suggests that the observed performance limitation is not due to inherent shortcomings in the deep learning models themselves, but rather stems from the intrinsic ambiguity present in web registers.

The research underscores a significant advantage of multilingual modeling: multilingual models consistently outperform their monolingual counterparts, particularly for languages that possess limited training data. This cross-lingual transfer capability is vital, as it allows for substantial benefits when training on smaller datasets or rare register classes. Specifically, languages such as Turkish, French, and Swedish showed improvements of 1–3 percentage points with the incorporation of multilingual training data.

However, the study also quantifies the limitations of zero-shot cross-lingual transfer: performance on unseen languages drops by an average of 7%. The authors note that while registers share cross-lingual features, they retain language-specific characteristics that necessitate in-language training data for optimal results.

The paper provides a direct comparison between the detailed hierarchical CORE taxonomy (25 classes) and simpler classification schemes, such as the 9-class X-GENRE scheme (Kuzman and Ljubešic 2023). The authors demonstrate that CORE achieves equivalent performance (77% micro F1) to the simpler scheme while offering much finer granularity. This detailed classification is deemed practically advantageous, enabling applications like those demonstrated by Eskelinen et al. (2024), which leverage specific subregisters to extract structured data from web documents.

A key methodological insight involves the treatment of hybrid texts. The multi-label approach preserves all register information within hybrid documents, preventing them from being collapsed into a single, potentially inaccurate Other category—a common pitfall in simpler models. Furthermore, the analysis reveals that the main challenge in register identification is not classifying hybrids themselves, but rather distinguishing between hybrid and non-hybrid documents. Models trained exclusively on non-hybrids perform poorly when predicting hybrids (55%), and vice versa (59%). This points to a fundamental structural difficulty: web documents often contain passages from different registers, complicating document-level classification.

Among the tested register classifier models, XLM-R Large was identified as offering the best balance between computational speed and accuracy. While multilingual training is beneficial, its effectiveness is heavily dependent on the size of the training data available for specific registers.

Future research directions are clearly outlined:

1. Interpretability: Expanding interpretability analysis to cover all 25 register classes and investigating systematic differences in keyword patterns across languages to better explain performance ceilings.

2. Semantic Analysis: Employing clustering techniques on semantic embeddings of web documents to uncover register relationships not explicitly captured by the CORE taxonomy.

3. Data Augmentation: Mitigating sampling method biases by creating new English register datasets from large-scale, cleaned corpora like HPLT (Burchell et al., 2025).

4. Generative Models: Exploring the use of generative Large Language Models (LLMs) for annotation tasks, though acknowledging their current computational expense for large-scale web data processing.

In conclusion, this work provides a robust framework for systematic and linguistically sound automatic register identification, demonstrating that multilingual deep learning models can effectively capture the vast diversity of web text while providing fine-grained metadata essential for advanced NLP applications.

Here are specific improvements to AI systems derived from this research, detailing what those improved systems can achieve:

) 1. Robust Multilingual Register Identification System (MRIS):

The core improvement is moving beyond simple monolingual or single-language models to a system trained on the comprehensive, hierarchical, multilingual CORE corpora.

• A system fine-tuned with XLM-R Large or BGE-M32048 can achieve a minimum of 77% micro F1 across five major languages (English, Finnish, French, Swedish, Turkish) when tested on non-hybrid documents and the full dataset in a multilingual setting.

• By incorporating the 25-class hierarchical taxonomy (including subregisters), the system can identify not just broad categories but also fine-grained contextual usage (e.g., distinguishing between Recipe and How-to or instructions).

) 2. Uncertainty and Noise Mitigation Module:

The improved system incorporates a data pruning step using the Cleanlab tool, which identifies documents with uncertain labels, outliers, or potential mislabeling based on confidence scores.

• This module can be integrated into the pre-training or fine-tuning pipeline to filter out noisy data before training.

• By removing these uncertain examples (which constitute 24% of the original dataset), the system's performance on challenging edge cases is significantly boosted, reaching over 90% F1 on the pruned test set.

) 3. Hybrid Document Discrimination Engine:

The system can be explicitly designed to differentiate between documents that belong to a single register versus those that are hybrid (combining multiple registers).

• By running separate classification experiments on hybrid and non-hybrid data, the system can achieve high performance (up to 82% F1) on purely non-hybrid documents.

• This capability allows the AI to provide a register purity score for any given text, helping downstream applications determine if a document is likely adhering strictly to one communicative context or if it represents a complex mix of registers.

) 4. Language-Agnostic Cross-Lingual Transfer Capability:

The system leverages multilingual pre-trained models (like XLM-R) to perform zero-shot register identification on 11 additional languages, even without specific training data for those languages.

• This allows the system to immediately classify web text in a language it has never seen during fine-tuning, achieving performance between 50% and 82% F1 (depending on the target language), demonstrating that register features are largely shared across languages.

) 5. Linguistically Interpretable Decision-Making Layer (SACX Integration):

The system includes a mechanism using the SACX explanation method to map classifier decisions back to specific linguistic keywords learned by the model for each register class.

• This provides transparency, allowing researchers to verify that the model is learning linguistically appropriate features (e.g., identifying question or interview keywords for Spoken registers).

• It helps diagnose why a specific classification was made and identifies document-specific terms that might be causing misclassification within a class.

) 6. Optimized Inference Strategy:

The system can utilize the fastest performing models (XLM-R Large, BGE-M3512, ME5) for large-scale deployment while reserving larger models (Mixtral 8x7B) for high-accuracy verification tasks.

• This allows for high throughput in real-time applications where speed is critical, while maintaining the capability to perform deep analysis on complex documents when necessary.

This article presents multilingual deep learning models for identifying web registers -- text varieties such as news reports and discussion forums -- across 16 languages. We introduce the Multilingual CORE corpora, which contain over 72,000 documents annotated with a hierarchical taxonomy of 25 registers designed to cover the entire open web. Using multi-label classification, our best model achieves 79% F1 averaged across languages, matching or exceeding previous studies that used simpler classification schemes. This demonstrates that models can perform well even with a complex register scheme at multilingual scale. However, we observe a consistent performance ceiling across all models and configurations. When we remove documents with uncertain labels through data pruning, performance increases to over 90% F1, suggesting that this ceiling stems from inherent ambiguity in web registers rather than model limitations. Analysis of hybrid texts (those combining multiple registers) reveals that the main challenge lies not in classifying hybrids themselves, but in distinguishing hybrid from non-hybrid documents. Multilingual models consistently outperform monolingual ones, particularly for languages with limited training data. Zero-shot performance on unseen languages drops by an average of 7%, though this varies by language (3--8%), indicating that while registers share features across languages, they also retain language-specific characteristics.