AI Wizards at EXIST 2026: Hierarchical Soft-Label Learning for Multimodal Sexism Identification in Memes

summary

Video file (mp4)

The gist

The gist The system presents a hierarchical multi-task learning approach for multimodal sexism identification in memes by modeling tasks as conditional soft-label predictions over empirical annotator

In short

The AI Wizards system identifies sexism in memes using a hierarchical approach modeled as conditional soft-label predictions over annotator distributions. It uses fixed Gemini embeddings processed by a lightweight Gated MLP to generate separate predictions for three tasks: binary identification, source intention, and fine-grained categorization. The model achieved top performance on the hardest task (2.3), proving that predicting disagreement distributions is superior to predicting single labels.

Key concepts

Hierarchical Soft-Label Learning
This approach treats the three sexism detection tasks as nested problems where each task's prediction is conditioned on the others. Instead of predicting a single correct answer, the model predicts a probability distribution over what different human annotators might label the meme, allowing it to learn from collective disagreement.
Learning with Disagreement (LeWiDi)
This paradigm requires AI systems to predict the full empirical distribution of labels given by multiple human annotators, rather than just guessing a single majority label. This forces the model to capture nuanced uncertainty and consensus across different human perspectives on sexist content.
Homoscedastic Uncertainty Weighting
This technique allows the model to dynamically adjust its focus between the three tasks during training. When one task is difficult, its associated variance parameter increases, effectively telling the model to pay less attention to that specific task's loss while still allowing it to learn from it.

Terminology used across episodes

This episode discusses

The paper

AI Wizards at EXIST 2026: Hierarchical Soft-Label Learning for Multimodal Sexism Identification in Memes · Read on arXiv

Matteo Fasulo, Antonio Gravina, Luca Tedeschini, Luca Babboni

Swiss Data Science Center, ETH Zurich · Everest Systems GmbH · Villanova.ai S.p.A

We present the AI Wizards submission to EXIST 2026 for multimodal sexism identification in memes. The task is composed of three, increasingly harder subtasks. We model them hierarchically as conditional soft-label prediction over empirical annotator distributions. Our system maps fixed Gemini Embedding 2 vision-language representations through a lightweight Gated MLP trained with KL divergence and homoscedastic uncertainty weighting. Our submissions ranked first on Task 2.3 and fourth on Tasks 2.1 and 2.2 on the official Soft-Soft leaderboards. The code is available at https://github.com/NLP-AI-Wizards/EXIST-2026

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "AI Wizards at EXIST 2026".

Jane: The gist The system presents a hierarchical multi-task learning approach for multimodal sexism identification in memes by modeling tasks as conditional soft-label predictions over empirical annotator distributions,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: What’s interesting here is how they handled training. They train against empirical soft targets using KL divergence, which penalizes the model when it places mass on categories that annotators collectively rejected.

Jane: That addresses the problem of models just guessing a majority label when there's real disagreement among human raters.

Lu: They also used homoscedastic uncertainty weighting to balance the three heterogeneous tasks instead of fixed scalar weights.

Meng: So, if one task loss gets too high, they increase a variance parameter sigma i to reduce that task’s contribution and prevent the model from completely ignoring it.

Tom: And they masked losses for Tasks two point two and two point three to only propagate gradients for instances where downstream annotations actually exist <ref:2607.04410#pg1>.

Jane: That sounds like a smart way to handle the complexity of having three different levels of prediction, from simple identification up to fine-grained categorization.

Conclusion: Tom: So, looking at the AI Wizards submission to EXIST two thousand twenty-six the title points directly at their approach: hierarchical soft-label learning for multimodal sexism identification in memes <ref:2607.04410#pg1,hierarchical soft-label learning for multimodal sexism identification in memes>.

Jane: It suggests they are focusing on how to handle ambiguity and disagreement in complex visual data like memes, using a structured learning method.

Lu: The authors claim that their system maps fixed Gemini Embedding two vision-language representations through this lightweight Gated MLP trained with KL divergence and homoscedastic uncertainty weighting <ref:2607.04410#pg1,lightweight Gated MLP trained with KL divergence and homoscedastic uncertainty weighting>.

Meng: And the results show they ranked first on Task two point three and fourth on Tasks two point one and two point two on the official Soft-Soft leaderboards, which is a significant performance metric for this type of learning task <ref:2607.04410#pg1,ranked first on Task 2.3 and fourth on Tasks 2.1>.

Tom: It implies that predicting the full annotator distribution is actually better than just predicting a single label for these kinds of nuanced problems.

Jane: For someone listening here, it means we're moving past models that just say "this meme is sexist" to models that can tell you *why* and which specific types of sexism are present, even when humans disagree on the exact category.

More episodes

← Home