A Unified BERT-CNN-BiLSTM Framework for Simultaneous Headline Classification and Sentiment Analysis of Bangla News

arXiv:2511.18618 · cs.CL, cs.AI · Submitted 2025-11-23 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "A Unified BERT-CNN-BiLSTM Framework for Simultaneous Headline Classification and Sentiment Analysis of Bangla News".

Jane: A unified framework for Bangla news headline classification and sentiment analysis has been proposed by combining BERT, CNN, and BiLSTM to simultaneously capture both aspects of news content.

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So the team is really buzzing about this paper titled "A Unified BERT-CNN-BiLSTM Framework for Simultaneous Headline Classification and Sentiment Analysis of Bangla News." It claims to tackle two big problems at once with a single system.

Jane: That’s right, Tom; the core idea is that we can look at what the news headline is about, like politics or sports, and also figure out how people feel about it—positive or negative—all in one go. This paper presents a method using a hybrid deep learning model called BERT-CNN-BiLSTM for Bangla news headlines.

Lu: It’s interesting how they combine those elements; we've seen transformer models like BERT work well for understanding context, but integrating them with CNNs and BiLSTMs to capture both local features and long-range dependencies is a clever architectural choice.

Meng: From an engineering standpoint, the paper addresses a real hurdle: handling class imbalance in low-resource languages like Bangla where data isn't always perfectly balanced. That’s something we deal with constantly when training models on smaller datasets.

Lalam: I think what excites me most is how this unified approach moves beyond just classifying headlines or just analyzing sentiment separately; it tries to learn both tasks simultaneously, which has real implications for how we process large volumes of text data in any language.

Tom: Exactly, and that simultaneous learning is what they are pushing for because existing work often focuses on one task at a time. They’re aiming to fill that gap by creating a model that captures headline categorization alongside sentiment analysis of the Bangla news headline (<ref:2511.18618#pg0>).

Jane: It really matters because understanding the emotional tone of news quickly helps people grasp the overall situation, which is crucial for staying informed about current events. This research explores a dataset called BAN-ABSA, which consists of nine thousand fourteen news headlines, and it’s noted as being the first time this specific combination has been experimented with simultaneously in Bengali (<ref:2511.18618#pg0>).

Lu: The paper highlights that previous deep learning solutions like CNN, BiLSTM, and GRU models sometimes struggle with generalization on low-resource conditions or suffer from overfitting when dealing with class imbalance (<ref:2511.18618#pg2>).

Meng: That points to the need for more robust architectures, and this hybrid model is presented as an attempt to raise that bar by combining feature extraction from CNNs with sequential modeling like BiLSTMs (<ref:2511.18618#pg2>).

Lalam: And the authors are specifically looking at how to make the most of those contextual embeddings provided by transformer-based models, which is a key area for future development in language understanding (<ref:2511.18618#pg2>).

Paper summary: Tom: Speaking of that, the methodology involves fusing feature representations from both branches—the CNN branch and the BiLSTM+Attention branch—before feeding them into final classification layers (<ref:2511.18618#pg0>). That fusion step is where the real power of this unified framework comes from.

Jane: That fusion is what allows the model to take the local patterns identified by the CNN and combine them with those long-range dependencies modeled by the BiLSTM (<ref:2511.18618#pg0>). It’s a very structured way of looking at complex text data.

Lu: I find the way they define their input embedding, h(zero)b,t = eb,t(tok) + et(pos) + eb,t(seg) in R dbert, which incorporates token embeddings with positional and segment information from BERT (<ref:2511.18618#pg0>), shows a deep understanding of how to leverage the transformer's output effectively.

Meng: I'm curious about the practical side here; they mention exploring various data balancing strategies to handle the class imbalance in Bangla, and that’s where we need to see if it translates into reliable performance on real-world news feeds.

Lalam: The paper discusses two experimental strategies for dealing with this imbalance: Technique-one which applies undersampling and oversampling to the entire dataset before splitting, and Technique-two which applies it only after splitting on the training set (<ref:2511.18618#pg0>).

Tom: And the results show that Technique-one oversampling provided what they called the "strongest performance" for both headline classification and sentiment analysis (<ref:2511.18618#pg0>). That’s a significant finding when comparing those two approaches.

Jane: It’s interesting to see that the authors found Technique-one to be superior in terms of accuracy, achieving eighty-four point two five percent with an LSTM model and even higher results with their hybrid CNN-LSTM model on comments (<ref:2511.18618#pg2>).

Lu: Considering the performance metrics they report, like the ninety point four nine percent accuracy for their hybrid CNN-LSTM model on sentiment analysis using Word2Vec embedding (<ref:2511.18618#pg2>), it shows that combining these specific components can yield very high results when tailored correctly to the data.

Meng: But they also acknowledge limitations in their work; specifically, they state that some previous DL-based solutions tended to suffer from overfitting and low generalization performance on the low-resource learning condition (<ref:2511.18618#pg2>). So, even with this new framework, we still need to keep an eye on how it performs when deployed across different types of news sources.

Lalam: That limitation is important because it tells us where the current approach stops working; it doesn't guarantee perfect performance everywhere without further fine-tuning specific to new data distributions (<ref:2511.18618#pg0>).

Tom: So, looking at the authors, Mirza Raquib and his team seem very focused on building something that is effective in low-resource settings while maintaining strong semantic understanding through this unified structure. It really speaks to their dedication to NLP for less-resourced languages (<ref:2511.18618#pg0>).

Paper summary: Jane: Indeed, and the conclusion they draw is that the BERT-CNN-BiLSTM model establishes a strong baseline for Bangla text classification in low-resource settings by jointly learning headline categorization and sentiment analysis (<ref:2511.18618#pg0>). That’s a solid starting point for anyone working in this area.

Lu: The implications here are huge, as it suggests that we can build more versatile models for less-resourced languages without having to train entirely new architectures from scratch for every single task (<ref:2511.18618#pg0>). This opens up possibilities for broader language support.

Meng: Practically speaking, if this framework is robust, it means we can deploy systems that can filter and categorize news content with more nuance than before, which has direct relevance for information filtering tools (<ref:2511.18618#pg0>).

Lalam: I think the cultural implication is that as these models get better at understanding the tone and subject matter of news in Bangla, we can develop AI assistants that offer much more context-aware commentary on local issues (<ref:2511.18618#pg0>).

Tom: That’s a big picture idea, Jane; moving from just identifying what's in the headline to truly understanding the feeling behind it changes how we interact with digital media. What's next for this research?

Jane: The paper does mention future work, but the main thrust is solidifying this hybrid structure as a strong baseline (<ref:2511.18618#pg0>). They’re clearly focused on proving that this combination works well together.

Lu: I think the next logical step for someone interested in this is exploring how to adapt these BERT-CNN-BiLSTM components to handle even more complex, nuanced aspects of news discourse beyond just sentiment and classification (<ref:2511.18618#pg0>).

Meng: From a practical deployment view, the next challenge would be optimizing the computational load of this unified model so it can run efficiently in real-time applications rather than just on benchmark datasets (<ref:2511.18618#pg0>).

Lalam: And I see the potential for this architecture to be integrated into larger, more sophisticated language models that already possess strong general knowledge, which could dramatically improve the cultural context awareness of those systems (<ref:2511.18618#pg0>).

Tom: It sounds like this paper gives us a very concrete tool—a unified framework—that’s ready to be tested and built upon by the community for tackling similar challenges in other low-resource languages.

Jane: Precisely, Tom; it provides a clear path forward by showing how combining different AI techniques can lead to better performance when we face data scarcity (<ref:2511.18618#pg0>). We’ll keep watching this space for updates on their next experiments and applications.

Conclusion: Tom: So we've looked at how this BERT-CNN-BiLSTM framework manages to tackle both classifying Bangla news headlines and figuring out the sentiment behind them all at once, and now we're getting to the final thoughts on this paper.

Jane: It really is impressive how they put those different AI layers together—BERT for context, CNN for local patterns, and BiLSTM with attention for long-range dependencies—to handle two complex tasks simultaneously.

Lu: From a theoretical standpoint, combining sequential modeling with convolutional feature extraction to capture both short-term features and long-term relationships is a very elegant way to structure the information flow in NLP.

Meng: I'm still thinking about how this structure actually scales; getting those different branches to communicate effectively without creating too much computational overhead is something we need to watch closely for practical deployment.

Lalam: This unified approach suggests a more holistic understanding of text where the subject matter and the emotional tone are intrinsically linked in a single processing stream.

Tom: Exactly, and when you look at the title, "A Unified BERT-CNN-BiLSTM Framework for Simultaneous Headline Classification and Sentiment Analysis of Bangla News," it really lays out exactly what this research has achieved.

Jane: It tells us that they haven't just built two separate systems; they’ve engineered one cohesive model designed to understand the subject and the feeling of a piece of Bangla news at the same time.

Lu: The implication here is that we can move toward more integrated NLP tools where understanding *what* is being reported and *how* it's being felt are handled in one unified process, which opens up new avenues for complex information retrieval.

Meng: From an engineering standpoint, this means we have a solid baseline architecture to start building on when we need to process Bangla news streams where both categorization and emotional scoring are required immediately.

Lalam: And for culture, this means that as AI gets better at capturing the nuances of how people feel about local news in Bangla, we can build tools that offer far more context-aware commentary on social issues.

Tom: That's a big picture idea, Lalam; it’s about moving beyond simple tagging to true contextual comprehension of local media. So, this unified framework isn't just a technical curiosity; it's a step toward smarter digital content interaction.

Jane: It’s definitely more than just technical curiosity when you consider the real-world impact on how we process and understand news in a specific language.

Lu: Indeed, and this structure provides a robust foundation that researchers can use to explore even deeper aspects of Bangla discourse later on, moving beyond the current scope of classification and sentiment.

Mirza Raquib, Munazer Montasir Akash, Tawhid Ahmed, Saydul Akbar Murad, Farida Siddiqi Prity, Mohammad Amzad Hossain, Asif Pervez Polok, Nick Rahimi

Department of CCE, International Islamic University Chittagong, Chattogram, Bangladesh · Department of ICE, Noakhali Science and Technology University, Noakhali, Bangladesh · Department of CSE, Banladesh University of Engineering and Technology, Dhaka, Bangladesh · Dept. of CSE, IIUC ChatagongtogrammeghatgrammeghatgrammeghatgrammeghatgrammeghatgrammeghatgrammeghatgrammeghatgrammE · School of CSE, University of Southern Mississippi, Hattiesburg, MS, USA · Department of ICE, NSTU, Noakhali, Bangladesh · mPower Social Enterprise · School of Computing Sciences and Computer Engineering, University of Southern Mississippi

cs.CL, cs.AI

Submitted: 2025-11-23

Updated: 2026-10-02

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 74/100

The gist: A unified framework for Bangla news headline classification and sentiment analysis has been proposed by combining BERT, CNN, and BiLSTM to simultaneously capture both aspects of news content.

Key concepts

BERT Layer
This layer uses pre-trained BERT to create deep, contextualized representations of words. It considers the meaning of a word based on both its left and right context simultaneously. These rich embeddings serve as the foundation for capturing complex linguistic nuances in the news text.
CNN Layer
The 1D convolutional branch is used to extract local n-gram features from the BERT embeddings. It scans the text using multiple kernel sizes to identify specific local patterns and features that might indicate certain aspects or sentiments within a headline.
BiLSTM Layer with Attention
This component uses a Bidirectional LSTM to understand long-range dependencies across the entire sequence of words. An attention mechanism is added to weigh the most informative tokens, allowing the model to focus on crucial parts of the text when making its final classification.

Terminology

Summary

A unified framework for Bangla news headline classification and sentiment analysis has been proposed by combining BERT, CNN, and BiLSTM to simultaneously capture both aspects of news content. This research addresses the challenge of handling class imbalance in a low-resource language like Bangla by exploring various data balancing strategies and demonstrating that the hybrid model achieves state-of-the-art results.

The gist

The proposed model BERT-CNN-BiLSTM significantly outperforms all baseline models in classification tasks, achieving new state-of-the-art results for Bangla news headline classification and sentiment analysis.

Model Architecture Overview

The framework is a hybrid deep learning approach that fuses transformer embeddings with CNN and BiLSTM networks. The overall architecture begins with a BERT layer to generate contextualized embeddings from input sequences, which are then processed by two parallel branches: a CNN module that captures local n-gram features and a BiLSTM+Attention module that models long-range dependencies. The feature representations from both branches are fused and passed through fully connected layers for the final classification.

The BERT layer utilizes pre-trained BERT to produce contextualized representations of words by considering both left and right contexts simultaneously. These embeddings are input into the CNN and BiLSTM models with attention to capture local patterns and global dependencies. The input embedding for a token at position t in sample b is defined as:

h(0)b,t = eb,t(tok) + et(pos) + eb,t(seg) ∈ R dbert

CNN Layer

After obtaining contextual embeddings from BERT, the CNN layer is employed to extract local n-gram features and n-gram features that may be indicative of specific aspects or sentiments in news headlines. This is achieved using a 1D convolutional branch with multiple kernel sizes. During convolution, the filter count is 128, ReLU activation is employed, L2 regularization is added, and batch normalization and dropout (p=0.35) follow. Global MaxPooling is then used to extract the most notable characteristics from each convolution, resulting in a pooled feature vector of size R(B×128). This CNN representation is denoted as CNN ∈ R B×256.

BiLSTM Layer with Attention Layer

The BiLSTM layer is designed to capture long-range dependencies across the entire sequence. It uses a Bidirectional LSTM (BiLSTM) combined with an attention mechanism. The BiLSTM encodes sequential dependencies in both forward and backward directions, while attention highlights the most informative tokens for the classification task. The final attended representation is calculated as:

BiLSTMatt = X Σ T t=1 αt · H′ b[t], where αt = exp(u⊤ t w), indicating the weighted sum of hidden states with dropout. This BiLSTM branch outputs a feature vector of size R(B×2dh).

Feature Combination and Classification Layer

The strengths of both branches are complementary, as the CNN features cooperate with the BiLSTM+Attention features. The CNN feature vector (∈ R B×dcnn) and the BiLSTM-attention feature vector (∈ R B×2dh) are concatenated to form a unified feature vector: F = Dropout BatchNorm Concat(CNN, BiLSTMatt) ∈ R B×(dcnn+2dh). This combined representation is then passed through fully connected layers for final classification. The model is trained with the AdamW optimizer and categorical cross-entropy loss with label smoothing (ϵ=0.2).

Experimental Strategies and Performance Analysis

The research experimented with two experimental strategies to handle class imbalance:

  1. Technique-1: Applying undersampling and oversampling to the entire dataset before splitting, which provided the strongest performance, both headline and sentiment.

  2. Technique-2: Applying undersampling and oversampling only after splitting on the training set.

The results showed that Technique-1 oversampling yielded superior outcomes compared to Technique-2 when trained directly on the original imbalanced dataset, achieving 81.37% test accuracy for headlines and 64.46% for sentiment analysis respectively. In contrast, in Technique-2, training directly on the original imbalanced dataset also delivered high results (81.37% and 64.46%). The paper concludes that the BERT-CNN-BiLSTM model establishes a strong baseline for Bangla text classification in low-resource settings by jointly learning headline categorization and sentiment analysis.

Model Interpretability

Local Interpretable Model-agnostic Explanations (LIME) are used to provide interpretability of model predictions. For headline classification, LIME visualizations show that the highlighted words strongly contributed to the politics class, indicating the model identifies domain-specific political terms as decisive features in classification. For sentiment analysis, LIME shows that the highlighted tokens made the strongest contributions toward the neutral class, suggesting reliance on descriptive or abstract terms to infer neutrality.

Improvements for AI systems

Here are specific improvements to AI systems derived from this research, detailing what these improved systems can do:

  1. Improved Bangla News Understanding System: The proposed BERT-CNN-BiLSTM framework will enable an AI system to simultaneously perform two critical tasks on raw Bengali news headlines:

  2. Headline Categorization (Classification): Assigning the headline to specific topics such as Politics, Religion, or Sports.

  3. Sentiment Analysis (Sentiment Classification): Determining the emotional tone of the headline (Positive, Negative, or Neutral).

  4. Robust Handling of Imbalanced Data: The system can be trained effectively on imbalanced datasets (like BAN-ABSA) by employing advanced data balancing techniques—specifically Technique 1 (global undersampling/oversampling) and Technique 2 (split-based balancing)—leading to significantly higher performance metrics in both classification tasks compared to standard methods.

  5. State-of-the-Art Performance for Low-Resource Languages: The model establishes a strong baseline and achieves new state-of-the-art results for Bangla news headline classification and sentiment analysis, making it highly effective for low-resource language NLP applications where annotated data is scarce.

  6. Enhanced Interpretability through XAI: By integrating Local Interpretable Model-agnostic Explanations (LIME), the improved AI system can provide transparent decision support. It can highlight which specific words or phrases in a headline most strongly influenced the model's classification (e.g., identifying key political terms for topic classification or descriptive words for sentiment).

  7. Superior Feature Fusion Capability: The hybrid architecture explicitly combines three distinct feature extraction methods:

  8. Contextual Embeddings from BERT (for deep semantic understanding).

  9. Local N-gram Features from CNN (for capturing short-range lexical patterns).

  10. Global Sequential Dependencies from BiLSTM with Attention (to model long-range context and focus on salient tokens).

  11. Improved Generalization and Stability: The use of 5-fold cross-validation, especially when utilizing the oversampled dataset under Technique 1, ensures the resulting AI system is robust and generalizes well across different data partitions, minimizing overfitting.

  12. Optimized Training Regimes: The system can dynamically choose between training on the original imbalanced set (Technique 2) or a balanced synthetic set (Technique 1), allowing developers to select the optimal strategy based on whether preserving natural distribution or achieving perfect class balance is prioritized for a specific deployment scenario.

  13. Versatility Across Datasets: The model demonstrates robustness by performing well on external, larger corpora (like the Potrika Dataset) and complex sentiment datasets, confirming its ability to maintain high performance when tested against novel data distributions.

Abstract

In our daily lives, newspapers are an essential information source that impacts how the public talks about present-day issues. However, effectively navigating the vast amount of news content from different newspapers and online news portals can be challenging. Newspaper headlines with sentiment analysis tell us what the news is about (e.g., politics, sports) and how the news makes us feel (positive, negative, neutral). This helps us quickly understand the emotional tone of the news. This research presents a state-of-the-art approach to Bangla news headline classification combined with sentiment analysis applying Natural Language Processing (NLP) techniques, particularly the hybrid transfer learning model BERT-CNN-BiLSTM. We have explored a dataset called BAN-ABSA of 9014 news headlines, which is the first time that has been experimented with simultaneously in the headline and sentiment categorization in Bengali newspapers. Over this imbalanced dataset, we applied two experimental strategies: technique-1, where undersampling and oversampling are applied before splitting, and technique-2, where undersampling and oversampling are applied after splitting on the In technique-1 oversampling provided the strongest performance, both headline and sentiment, that is 78.57% and 73.43% respectively, while technique-2 delivered the highest result when trained directly on the original imbalanced dataset, both headline and sentiment, that is 81.37% and 64.46% respectively. The proposed model BERT-CNN-BiLSTM significantly outperforms all baseline models in classification tasks, and achieves new state-of-the-art results for Bangla news headline classification and sentiment analysis. These results demonstrate the importance of leveraging both the headline and sentiment datasets, and provide a strong baseline for Bangla text classification in low-resource.

Sources

Related papers