Multiclass Sentiment Analysis for Identifying Political Viewpoints

The paper investigates multiclass sentiment analysis of political viewpoints on social media, specifically for Tamil language tweets, using two machine-learning approaches: XGBoost and BERT. The dataset, provided for the DravidianLangTech@NAACL 2025 shared task, consists of 4,352 training samples, 544 development samples, and 544 unlabeled test samples, with seven sentiment classes: Substantiated, Sarcastic, Opinionated, Positive, Negative, Neutral, and None of the above. Preprocessing involved removing punctuation, emojis, and user mentions using Python's regular expression module. Feature extraction used TF-IDF for the XGBoost model and BertTokenizer for the BERT model. The XGBoost model achieved a macro F1-score of 0.2835, while the BERT-based model (bert-base-uncased, with learning rate 1e-5, 5 epochs, batch size 32, and softmax activation) achieved a macro F1-score of 0.2806 on the test set. The results demonstrate the challenge of classifying complex and contextualized political discourse sentiment. The confusion matrices show that 'Positive' was the most correctly classified label (110 times for XGBoost, 109 times for BERT), while 'None of the above' was poorly classified (2 times for BERT). Common misclassifications included 'Negative' misclassified as 'Positive' (56 times in both models) and 'Substantiated' misclassified as 'Positive' (43 times in both models). The paper concludes that XGBoost outperformed BERT in this use case, and suggests future research should extend to more underrepresented languages, incorporate additional algorithms, and increase dataset size and diversity for more robust and generalizable models.

Improvements to AI Systems:

1. Domain-Adaptive Sentiment Calibration: Implement a post-processing layer that uses confusion-matrix-derived priors (e.g., penalizing over-prediction of 'Positive' for 'Negative' and 'Substantiated' classes) to re-weight softmax outputs. This would reduce systematic misclassifications observed (56 and 43 errors, respectively).

2. Hierarchical Multi-Task Learning: Train a shared BERT backbone with two auxiliary heads: one for sarcasm detection and one for stance detection (substantiated vs. opinionated). This forces the model to learn contextual cues that differentiate sarcastic/opinionated from literal positive/negative, addressing the poor performance on 'None of the above' (only 2 correct).

3. Code-Mixed Tokenization Enhancement: Replace standard BertTokenizer with a custom tokenizer that preserves Tamil-English code-mixed morphemes and handles transliteration variants. This would improve feature extraction for underrepresented linguistic patterns, potentially boosting macro F1 beyond 0.28.

4. Ensemble with Confidence-Based Voting: Combine XGBoost (TF-IDF) and BERT predictions using a meta-learner (e.g., logistic regression on their softmax probabilities). Since XGBoost slightly outperformed BERT, the ensemble can exploit complementary strengths—XGBoost for lexical patterns, BERT for contextual nuances—to improve overall macro F1.

5. Active Learning for Minority Classes: Use uncertainty sampling (e.g., entropy of BERT's predictions) to selectively label the most ambiguous test samples, specifically targeting 'None of the above' and 'Sarcastic' classes. This would expand the training set (currently 4,352) with high-value examples, directly addressing the dataset size limitation.

What the Improved AI System Can Do:

• Achieve higher macro F1-score (target >0.35) on Tamil political sentiment by reducing systematic positive-bias and improving rare-class recall.

• Correctly identify sarcastic and substantiated statements by leveraging sarcasm/stance auxiliary tasks, rather than collapsing them into 'Positive' or 'Negative'.

• Handle code-mixed Tamil-English text more robustly, making it deployable for real-world social media monitoring in multilingual regions.

• Provide calibrated confidence scores, allowing downstream systems to flag uncertain predictions for human review, especially for 'None of the above' cases.

• Generalize better to unseen political discourse by learning from a more balanced, actively curated dataset, reducing overfitting to majority classes.

The rapid growth of social media has created vast amounts of political discourse, which provides valuable opportunities to analyze public opinions and identify different political perspectives. Sentiment Analysis (SA) is a core task in Natural Language Processing (NLP) that allows the computational study of attitudes and opinions in textual data, and has become increasingly important for understanding political discourse. In this work, we investigate multiclass sentiment analysis of political view- points on social media, that is to automatically discriminate multiple sentiment classes over political issues and figures. To solve this task we design and evaluate two machine-learning approaches based on XGBoost and BERT. We train and evaluate the models on a labeled dataset of political social media posts using standard classification metrics. The experimental results show that the XGBoost model reaches an F1-score of 0.2835 and the BERT- based model reaches an F1-score of 0.2806 on the test set. These results demonstrate the challenge of classifying complex and contextualized political discourse sentiment and provide a baseline for future research in multiclass political sentiment analysis.