M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Dataset
cs.CL
Submitted: 2025-10-27
Updated: 2026-09-01
Comments: Camera-ready version accepted at EMNLP Findings 2026. Code and data available at: https://github.com/UKPLab/M4FC
Code: https://github.com/UKPLab/M4FC
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: Existing real-world datasets for multimodal fact-checking have multiple limitations: they contain few instances, cover only one or two languages, focus on a single task, or rely on external news
Terminology
Abstract
Existing real-world datasets for multimodal fact-checking have multiple limitations: they contain few instances, cover only one or two languages, focus on a single task, or rely on external news article sets to source true claims. To address these shortcomings, we introduce M4FC, a new real-world dataset comprising 4,982 images paired with 6,980 claims. The images, verified by professional fact-checkers from 22 organizations, represent a diverse range of cultural and geographic contexts. Each claim is available in one or two out of ten languages. M4FC spans six multimodal fact-checking tasks: visual claim extraction, claimant intent prediction, fake image detection, image contextualization, location verification, and verdict prediction. We provide baseline results for all tasks and analyze how combining intermediate tasks affects verdict prediction performance. We make our dataset and code publicly available.
Sources
- Qwen2.5-VL Technical Report
- Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
- AMMeBa: A Large-Scale Survey and Dataset of Media-Based Misinformation In-The-Wild
- Multimodal Large Language Models to Support Real-World Fact-Checking
- Towards Explainable Bilingual Multimodal Misinformation Detection and Localization
- The Llama 3 Herd of Models
- VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checking
- NewsRECON: News Article Retrieval for Image Contextualization
- XFacta: Contemporary, Real-World Dataset and Evaluation for Multimodal Misinformation Detection with Multimodal LLMs
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering