Customized large language models can outperform Community Notes in correcting misinformation

arXiv:2403.11169 · cs.CL, cs.AI · Submitted 2024-03-17 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Customized large language models can outperform Community Notes in correcting misinformation".

Jane: The paper was written by the authors from.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Jane: We also have Lu with us today — senior AI researcher at Tsinghua.

Tom: We also have Meng with us today — lead engineer at a mysterious AI startup.

Jane: We also have Lalam with us today — the in-house Large Language Model.

Tom: Alright, let's get started.

Title: Tom: We are kicking things off with a paper that has a massive title: 'Customized large language models can outperform Community Notes in correcting misinformation'.

Jane: It sounds like a direct challenge to how social media handles truth, Tom.

Tom: It really is, especially since it targets the way people currently fact-check things on X.

Jane: Most people know Community Notes as that system where users vote on whether a post is helpful or not.

Tom: Right, it's a crowd-sourced approach, but these authors are suggesting a different path.

Jane: Xinyi Zhou and her colleagues from the University of Washington are the ones behind this research.

Tom: They've teamed up with researchers from Boise State and Microsoft to build something called MUSE.

Jane: I love how they're moving from human crowds to a specialized AI system.

Lu: This could be the beginning of a digital immune system for every platform on the web.

Tom: A digital immune system, Lu?

Lu: Imagine an AI that recognizes a lie and neutralizes it before it even spreads to your feed.

Meng: I'm thinking about the actual infrastructure required to make that happen at scale.

Jane: That's a fair point, Meng, because the paper mentions how hard it is for humans to keep up.

Meng: Exactly, because the volume of posts is just too high for any group of people to monitor.

Tom: So they're trying to solve the scalability problem by using a customized model.

Jane: It's about making the correction as fast as the misinformation itself.

Lalam: This could actually help restore a sense of shared reality in our digital culture.

Tom: That's a heavy thought, Lalam, to think we've lost a shared reality.

Jane: It's true, and this paper suggests we might be able to use AI to find it again.

Tom: Let's see if the actual data supports such a big claim.

Summary: Tom: We're looking closer at the results of 'Customized large language models can outperform Community Notes in correcting misinformation'.

Jane: The numbers they found are honestly staggering, Tom.

Tom: They claim MUSE outperforms GPT-four by thirty-seven percent in its ability to respond to potential misinformation.

Jane: And it's even more impressive that it beats high-quality human responses by twenty-nine percent.

Tom: That's a huge margin when you're comparing an AI to actual people.

Jane: It's especially notable because they tested it against the best responses from the Community Notes crowd.

Tom: Did they look at how this affects the people actually reading the posts?

Jane: They did, and they found that using MUSE can increase a person's ability to identify misinformation by nine point eight percent.

Tom: That's a real, measurable improvement in human digital literacy.

Meng: I want to know if this works on brand new topics that haven't been discussed yet.

Jane: The researchers actually tested that, Meng.

Meng: And did it hold up?

Jane: It did, because they showed it works even on content that hasn't been fact-checked online before.

Lu: It's the ability to generalize that makes this so creative and powerful.

Tom: It's not just repeating old facts, is it, Lu?

Lu: No, it's about the model's ability to reason through new situations.

Lalam: It gives people a way to feel more confident in their own judgment.

Tom: That confidence seems to be the real win here.

Jane: It's about moving from being a passive consumer to an informed participant.

Tom: Let's look at how they actually built this engine to get these results.

Improvements: Tom: We're breaking down the methodology of 'Customized large language models can outperform Community Notes in correcting misinformation'.

Jane: The big thing to understand is that MUSE isn't just a standard chatbot.

Tom: It's been augmented with a few very specific tools.

Jane: One of those is what they call vision-language modeling.

Tom: Which basically means the AI can actually "see" and understand images.

Jane: It doesn't just see a picture, it can read the text inside that picture too.

Tom: That's vital because so much misinformation is just a screenshot of a fake headline.

Meng: How do they stop the model from just making up its own facts, though?

Jane: They use something called credibility-aware web retrieval.

Meng: So it's actually going out to the internet to check things?

Jane: Yes, but it doesn't just grab any random link it finds.

Tom: They've built in a way to check if a publisher is biased or even factual.

Jane: It prioritizes sources that are highly credible and have minimal bias.

Lu: It's like giving the AI a set of high-quality textbooks to study from in real-time.

Tom: And that prevents the hallucinations that we usually see with models like GPT-four.

Jane: Exactly, because the response is grounded in those real web links.

Meng: I'm interested in the cost of doing all those live searches.

Jane: They mentioned it costs about zero point five USD per post right now.

Tom: That's a bit high for a single post, but it's much cheaper than a team of experts.

Lalam: It's a way to integrate truth-seeking into the very fabric of our communication.

Jane: It turns the AI from a generator of text into a researcher of facts.

Tom: It's a complete shift in how we think about these models.

Conclusion: Tom: We've reached the end of our look at 'Customized large language models can outperform Community Notes in correcting misinformation'.

Jane: It's been a fascinating deep dive into how we might fight the spread of lies.

Tom: This research really shows that AI can be a force for good if we build it with the right tools.

Jane: It's about moving toward a more transparent and verifiable internet.

Lu: I see a future where every piece of content comes with an invisible layer of truth.

Meng: My focus will be on making these systems faster and more affordable for everyone.

Lalam: We're moving toward a culture where we can actually trust the information we consume.

Tom: Thanks to the whole team for joining us today.

Jane: We'll be back soon with another incredible paper.

Tom: Goodbye for now, everyone.

cs.CL, cs.AI

Submitted: 2024-03-17

Updated: 2026-09-10

Comments: 45 pages

Code: https://github.com/Social-Futures-Lab/MUSE

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 84/100

The gist: This paper introduces M USE, a scalable approach for "multimodal misinformation correction" designed to address the rapid spread of false or misleading content on social media.

Key concepts

Community Notes: This is a crowd-sourced approach to fact-checking, most commonly known as the system where users vote on whether a post is helpful. The research uses this method as a benchmark to show that specialized AI systems can provide more accurate and scalable corrections of misinformation.
Customized Large Language Models: These are specialized AI models, such as MUSE, designed for specific tasks like identifying and correcting misinformation. The research suggests these customized models can surpass general-purpose LLMs and even beat high-quality human responses in accuracy.
Vision-Language Modeling: This is a key capability that allows the AI to process more than just text. It enables the model to 'see' images and understand any text contained within them, which is essential for fact-checking misinformation presented as screenshots or pictures.

Terminology

Accuracy — 19× in this paper · explained in 1 episode
Hallucination — 9× in this paper · explained in 4 episodes
Verification — 7× in this paper · explained in 2 episodes
Generalizability — 6× in this paper · explained in 6 episodes
Ablation — 4× in this paper · explained in 1 episode
Ablation study — 3× in this paper · explained in 1 episode
Large Language Models (LLMs) — 3× in this paper · explained in 6 episodes
Transformer — 3× in this paper · explained in 2 episodes
Annotator Agreement — 2× in this paper · explained in 1 episode
Consistency — 2× in this paper · explained in 3 episodes
Embeddings — 2× in this paper · explained in 2 episodes
In-Context Learning — 2× in this paper · explained in 4 episodes
Multimodal Integration — 2× in this paper · explained in 2 episodes
Ablation Studies — 1× in this paper · explained in 1 episode
Artificial Intelligence (AI) — 1× in this paper · explained in 2 episodes
Core — 1× in this paper · explained in 2 episodes
Cosine Similarity — 1× in this paper · explained in 3 episodes
Downstream Tasks — 1× in this paper · explained in 2 episodes
Multimodal Reasoning — 1× in this paper · explained in 5 episodes
Reinforcement Learning — 1× in this paper · explained in 4 episodes

Summary

This paper introduces M USE, a scalable approach for multimodal misinformation correction designed to address the rapid spread of false or misleading content on social media. Because manual correction is laborious and difficult to scale, the authors propose an LLM-based framework that can identify and explain inaccuracies with grounded references, providing a potential solution to improve the online information ecosystem.

The M USE Framework

M USE is composed of three interconnected modules designed to handle textual and visual content by augmenting a large language model with specialized capabilities. The system consists of:

An LLM for response generation.
A hierarchical, credibility-aware web retriever for evidence extraction.
A multimodal integrator that equips the modules with vision-language modeling.

To handle images, M USE employs informative image captioning that augments standard models with features like the recognition of celebrities and optical characters. This allows the model to generate textual descriptions that are crucial for accurate verification and correction.

The Correction Pipeline

The model follows a structured process to ensure the accuracy and trustworthiness of its responses. First, M USE generates multiple queries from the potential misinformation to act as a way of denoising the post content. These queries are used to access timely web content via a search engine. The pipeline then includes several critical filtering steps:

Calculating direct relevance to the potential misinformation to remove irrelevant pages.
Determining the credibility of web pages by checking publishers' factuality and bias ratings.
Extracting evidence that either refutes or contextualizes the given content.

Finally, the LLM generates a response by utilizing the extracted evidence and its source web links, ensuring the output is informative and short.

Evaluation and Performance

The researchers evaluated M USE using X Community Notes data, comparing its performance against GPT-4 and responses from laypeople. The quality of responses was assessed across 13 specific criteria, including the explicitness, accuracy, comprehensiveness, and informativeness of explanations, as well as the relevance, factuality, fluency, coherence, and toxicity of the generated text. Results demonstrate that M USE consistently produces high-quality outputs across diverse domains and political leanings. Specifically, the study found that M USE outperforms GPT-4 by 37% and even outperforms high-helpfulness responses from laypeople by 29%.

Real-World Impact and Generalizability

M USE demonstrates strong generalizability across various modalities, domains, political leanings, and adopted tactics. It maintains high performance whether responding to liberal or conservative content or addressing specific misinformation tactics such as cherry-picking, lacking context, or implying false or oversimplified causation. Beyond expert evaluation, a study with 988 participants measured end-user perceptions. The findings revealed that M USE-generated corrections significantly enhance the correct belief that the misinformation is misleading, increasing this ability by 9.8%. This suggests that the model can effectively increase people’s ability to correctly identify misinformation in real-world social media environments.

Improvements for AI systems

1. Informative Multimodal Captioning Integration

Improvement: Replace standard image-to-text captioning with a specialized module that integrates Optical Character Recognition (OCR) and Named Entity Recognition (NER) to identify specific celebrities, objects, and embedded text within visuals.
Capability: The system can read and interpret text-heavy memes, infographics, and screenshots of news articles, ensuring that the reasoning engine receives a high-fidelity textual description of the visual context rather than a generic summary.

2. Credibility-Aware Hierarchical RAG (Retrieval-Augmented Generation)

Improvement: Implement a two-stage retrieval pipeline that ranks web sources using a pre-vetted database of publisher factuality and bias ratings (e.g., Media Bias/Fact Check) rather than relying solely on semantic similarity.
Capability: The system can autonomously filter out conspiracy-pseudoscience or low-factuality sources during the retrieval phase, ensuring that the generated explanations are grounded exclusively in high-credibility, low-bias evidence, thereby preventing the backfire effect.

3. Decompositional Multi-Query Generation

Improvement: Incorporate a pre-retrieval step where the LLM decomposes a single social media post into multiple, discrete, and verifiable claims via diverse query generation.
Capability: The system can identify and correct partial misinformation—scenarios where a post contains a mix of true and false claims—by verifying each component individually, ensuring a comprehensive rather than superficial correction.

4. Temporal and Source-Priority Evidence Grounding

Improvement: Augment the response generation logic to prioritize extracted evidence based on two vectors: the recency of the publication date and the hierarchical priority of the publisher (e.g., prioritizing Very High factuality sources over Medium priority ones).
Capability: The system can provide real-time, highly accurate corrections on rapidly evolving emerging topics while significantly reducing hallucinations by strictly anchoring explanations to reachable, high-priority, and temporally relevant URLs.

Abstract

Addressing misinformation in real-world settings is challenging: content is often multimodal; factuality judgments are nuanced and context-dependent; new events emerge rapidly across domains; corrections must be timely, trustworthy, and politically impartial; and multidimensional, multistakeholder frameworks remain lacking. Crowdsourced fact-checking systems such as Community Notes have gained broad adoption, but timely, scalable coverage remains difficult. We introduce MUSE, which augments large language models (LLMs) with trust-aware retrieval of up-to-date evidence and task-specific multimodal reasoning. Given a piece of content, MUSE identifies whether and which parts may be false or misleading and provides explanations grounded in credible references. We also develop an evaluation framework that assesses expert-rated response quality---including identification accuracy, explanation factuality, and the relevance and credibility of supporting references---as well as user perceptions. Across social media posts spanning modalities, domains, political leanings, misinformation tactics, and popularity, MUSE consistently produces high-quality responses, including for content not previously fact-checked online, and outperforms even highly rated Community Notes by 29%. It also improves participants' recognition of misinformation by 10%. Our work establishes a general methodological and evaluative framework for timely, scalable, and trustworthy correction of misinformation.