A Large-Scale Dataset for Multi-Class Financial News Classification in Brazilian Portuguese

summary

The gist

The gist A large-scale dataset for multi-class financial news classification in Brazilian Portuguese was developed to address the scarcity of domain-specific datasets hindering Financial Natural

In short

A large dataset of 160,000 Brazilian Portuguese financial news articles was created to help build systems for financial NLP where data is scarce. Researchers tested three similarity-based classification methods—Cosine Similarity, KNN, and FAISS—using semantic embeddings. Findings show that manually labeled data outperforms LLM-generated labels significantly in terms of accuracy and F1-score, proving that high-quality human annotation is crucial for accurate financial text classification.

Key concepts

Financial NLP: This field deals with using Artificial Intelligence to understand and process financial news written in a specific language, like Brazilian Portuguese. The goal is to automatically categorize articles into different financial topics, such as oil or banks, making large amounts of news data usable for financial analysis.
Semantic Embeddings: These are numerical representations of text that capture the meaning or 'semantics' of words and sentences. Instead of just looking at keywords, these embeddings allow computers to understand that two different sentences might mean similar things in a financial context, which is essential for similarity-based classification.
Weak Supervision: This technique involves using less precise or automated labels (like those generated by an LLM) as a starting point for training. The paper explores how effective these weaker labels are compared to high-quality human annotations when classifying complex financial news articles.
FAISS: FAISS is a specialized search algorithm designed for finding the nearest neighbors in massive datasets of vectors, such as semantic embeddings. It is used here to quickly find the most semantically similar articles to a given article, making it efficient for large-scale classification tasks.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Sample Complexity — explained in 20 episodes
Retrieval-Augmented Generation (RAG) — explained in 18 episodes
Federated Learning — explained in 17 episodes
Quantum Fisher Information (QFI) — explained in 16 episodes
Flow Matching — explained in 15 episodes

This episode discusses

A Large-Scale Dataset for Multi-Class Financial News Classification in Brazilian Portuguese · Paper Radio

The paper

A Large-Scale Dataset for Multi-Class Financial News Classification in Brazilian Portuguese · Read on arXiv

Federal University of Paraíba · Federal University of Campina Grande · Federal Rural University of Pernambuco

The development of Financial Natural Language Processing (NLP) systems for Brazilian Portuguese is hindered by the scarcity of domain-specific datasets. To address this, we present a large-scale corpus of approximately 160,000 Brazilian Portuguese financial news articles collected from web sources. To establish initial benchmarks for a weakly supervised multi-class classification approach based on semantic similarity, we annotated subsets of this corpus both manually and using a Large Language Model (LLM). We evaluated three similarity-based methods -- cosine similarity, K-Nearest Neighbors (KNN), and FAISS-based nearest-neighbor search -- using multilingual semantic embeddings. Our experiments reveal that the task is highly sensitive to label quality and class distribution. On the manually labeled subset, KNN achieved the best performance (47.3% accuracy, 31.9% F1-score), closely followed by FAISS. Conversely, on the LLM-annotated subset, while accuracy reached approximately 50%, F1-scores dropped to near 20%, highlighting the negative impact of label imbalance and noisy weak annotations. These findings establish foundational baselines for a challenging task, underscoring the critical need for high-quality manual annotations. Ultimately, this dataset and our preliminary results provide a reference point for future research in Brazilian Portuguese Financial NLP.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "A Large-Scale Dataset for Multi-Class Financial News Classification in Brazilian Portuguese".

Jane: The gist A large-scale dataset for multi-class financial news classification in Brazilian Portuguese was developed to address the scarcity of domain-specific datasets hindering Financial Natural Language Processing (NLP) systems for…

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we’re looking at this paper now about building a huge dataset for financial news in Brazilian Portuguese. The core idea is that there isn't enough data like this to train good AI models for finance in that language, so they created a massive collection of one hundred sixty thousand articles.

Jane: This paper is a huge step forward for giving researchers something concrete to work with when training AI models for finance in that specific language.

Lu: And they also remind us that using AI for annotation at scale requires serious validation first.

Jane: And that means we need to be rigorous about validating those LLM labels before we use them as ground truth.

Lu: The authors also point out that expanding the human-labeled set and moving toward supervised transformer models is the next logical research step for this type of work.

Meng: If you're thinking about deployment, that means we’re talking about using FAISS to keep the classification step fast when you have millions of vectors to search through.

The paper's summary: Tom: So the paper wraps up by suggesting ways to make that dataset even better, mostly focusing on how they handled the labeling process. They noticed that using LLM labels directly needs a lot more scrubbing to get reliable results.

Jane: Right, so they are saying if you use those AI-generated labels as your ground truth, you can't just toss them in and run the model; you need stricter filtering criteria to cut out the noise.

Lu: They want better quality reference data so that when people use weakly supervised methods, the performance metrics are less likely to be skewed by bad labeling.

Meng: From a production standpoint, that makes sense because if your training set is full of garbage labels, your resulting model will just learn garbage patterns at scale.

Tom: They also pointed out the need to improve how they create their manual labels for the smaller subset. They want a better way to capture all those different financial sectors in their manual tagging strategy.

Jane: That means they are looking at making the keyword search strategy for human annotation smarter so it doesn't miss any of those underrepresented industries in the full corpus.

Lu: And they also suggested moving toward using supervised transformer-based models for direct text classification as a next step beyond just similarity search.

Meng: That sounds like a big lift, though I wonder how much extra compute that would cost to run compared to the similarity search method they tested initially.

Tom: The paper is essentially saying that while this dataset is a solid start for testing methods, the path forward involves cleaning up those labels and potentially moving toward models trained directly on labeled data.

Jane: So, the main message here is that building these kinds of domain-specific datasets requires a careful, iterative process of improving both the data collection and the labeling quality before you can trust any classification results.

Lu: It’s about ensuring that when you try to build these AI systems for finance, you're not just feeding them a massive pile of text, but high-quality signals.

Tom: And they are emphasizing that if we want this to actually impact real financial NLP applications, we need to focus on making those labels robust and representative of the real world.

Jane: This leads us nicely into how these models perform when they’re being tested against actual financial data versus just textual similarity.

The paper's improvements: Tom: So we’ve seen how they built this massive dataset for multi-class financial news classification in Brazilian Portuguese, and now we’re wrapping up by looking at what this all means for the field.

Jane: Basically, they created a large corpus of articles to give researchers something concrete to work with when training AI models for finance in that specific language.

Lu: The implication is that we can finally move past just using English benchmarks and start building tools tailored to the Brazilian financial market directly.

Meng: It gives us a foundation, but it doesn't mean we can skip the hard part of fine-tuning those models to actually perform reliably in a production environment.

Lalam: I’m confident that this dataset provides a necessary structure for future research because it directly addresses the scarcity of domain-specific datasets in Brazilian Portuguese finance.

Tom: Yeah, and they showed us that understanding label quality and class distribution is central to getting reliable results for these weak supervision methods.

Jane: It’s a good warning to any researcher: don't just grab a huge dataset; you have to rigorously validate how those labels were created before you use them as ground truth.

Lu: They also highlighted that the choice between different similarity search structures, like FAISS versus KNN, really comes down to how fast and cheap you need the classification step to be during deployment.

Meng: If we're talking about shipping models, efficiency matters because slow searches just mean high latency for real-time decisions.

Tom: So overall, this paper on "A Large-Scale Dataset for Multi-Class Financial News Classification in Brazilian Portuguese" gives us the data and shows us the mechanics of how similarity search works in a new linguistic context.

Jane: It’s a strong foundation, but as Lu mentioned, they still have to figure out how to integrate that textual similarity with external financial data for truly robust decision-making systems.

Lu: And I think the next big move is moving toward supervised transformer models that can learn those complex decision boundaries directly from the text.

Meng: That sounds like a much bigger training investment, but if it gives you direct classification power, then it’s worth exploring when you're ready to push production.

Lalam: I am sure that this work sets a clear direction for how we approach low-resource financial NLP problems going forward.

Tom: Alright, that wraps up our look at this paper on "A Large-Scale Dataset for Multi-Class Financial News Classification in Brazilian Portuguese".

Jane: We’ve covered the construction, the testing against human and AI labels, and what it means for building finance models in Portuguese.

Lu: It’s a solid piece of data for representation learning research.

Meng: It shows us where to focus our attention next if we want to move this from paper to product.

Conclusion: Tom: So we’ve seen how they built this large-scale dataset for multi-class financial news classification in Brazilian Portuguese, and now we’re wrapping up by looking at what all that means for the field.