Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs

arXiv:2510.14242 · cs.CL, cs.LG · Submitted 2025-10-16 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs".

Jane: The gist: Flip-Flop Consistency (F2C) is an unsupervised training method that improves Large Language Model robustness to prompt perturbations by aligning representations without sacrificing performance,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: Let's talk about who wrote this stuff. The paper is "Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs," and it’s authored by Parsa Hejabi Elnaz Rahmati, Alireza S. Ziabari, Morteza Dehghani from the University of Southern California.

Jane: It’s good to see research coming from places like USC on these kinds of fundamental model issues. The authors are looking at how to fix that inconsistency using unsupervised training methods instead of needing a ton of human labels.

Lu: The authors are clearly trying to build something that works without needing those expensive, time-consuming supervised fine-tuning datasets they mentioned earlier in the paper, which is a huge practical step forward.

Meng: Unsupervised is always appealing because labeled data can be scarce or very expensive to generate for these kinds of consistency tasks. It sounds like they are trying to find a way to make models robust without needing massive amounts of human annotation work.

Lalam: I think the focus on unsupervised training means they are building a method that learns from the model's own internal structure and its responses, rather than relying on external data points to teach it what's consistent.

The paper's summary: Tom: So, what is F2C actually doing? Basically, they introduce Flip-Flop Consistency. They say this method has two main parts: Consensus Cross-Entropy and a representation alignment loss.

Jane: That sounds technical, but the simple idea is that they use a majority vote across different prompt variations to create some kind of hard pseudo-label, and then they pull the less confident answers toward that consensus.

Lu: The Consensus Cross-Entropy part sounds like it’s establishing what the "correct" answer should be by seeing which prompt variations agree most strongly on a single answer.

Meng: And the representation alignment loss is where they take those answers, and if an input variation isn't in that majority group, they adjust its internal representation to match the consensus.

Lalam: So it’s not just picking the majority vote; it’s actively teaching the model how to handle inputs that are slightly different but should all lead to similar outcomes.

The paper's improvements: Tom: The results they show are pretty compelling when you look at how this works in practice. They tested it on eleven datasets across four different NLP tasks, and the average agreement observed went up by eleven point six two percent <ref:2510.14242#pg1>.

Jane: That’s a solid number, but what really stands out is that the mean F1 score improved by nearly nine percent on nine of those datasets—that's a big boost in actual task performance.

Lu: They also showed that this method reduces the variance across different formats by three point two nine percent on average, which means it makes the model more predictable when you change how you ask it things <ref:2510.14242#pg1>.

Meng: That reduction in variance is important for deployment because it means we can trust the output more, knowing it won't swing wildly depending on a minor formatting change in the prompt.

Lalam: They also noted that when testing on data that wasn't used during training, this method still performs well and actually increases agreement while decreasing variance compared to the base model.

Conclusion: Tom: Alright, so to wrap up this discussion on "Flip-Flop Consistency," the big picture here is that we can significantly improve robustness without needing those massive amounts of labeled data for every single prompt variation.

Jane: The paper shows that by using internal consensus to pull representations toward a shared understanding, we get better agreement and higher performance in many scenarios. It seems like leveraging the model’s own internal signals is a powerful way to boost reliability.

Lu: It suggests that much of the inconsistency we see in LLMs from just changing how we phrase things can be managed by letting the model find its own consensus, even without any human labels guiding it.

Meng: From an engineering standpoint, this means less overhead in prompt optimization because we are training the model to handle variations intrinsically rather than having to constantly engineer perfect prompts for every single use case.

Lalam: I think this work points toward a future where models can be more naturally consistent when interacting with users, making them much more trustworthy for complex tasks.

Tom: That’s what we have with Flip-Flop Consistency—a way to teach models consistency through their own internal dialogue across different inputs. We’ll be watching how this method integrates into the broader landscape of AI development.

Parsa Hejabi Elnaz Rahmati Alireza S. Ziabari Morteza Dehghani

University of Southern California

cs.CL, cs.LG

Submitted: 2025-10-16

Updated: 2026-10-05

Comments: 17 pages, 7 figures, 7 tables, and 1 algorithm. ACL 2026. Revised after peer review; added Llama-3.2-3B-Instruct experiments, majority pseudo-label quality analysis, representation alignment analysis, and expanded implementation details

Journal ref: Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1571-1587 (2026)

DOI: 10.18653/v1/2026.acl-long.71

Code: https://github.com/huggingface/accelerate

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 73/100

The gist: The gist: Flip-Flop Consistency (F2C) is an unsupervised training method that improves Large Language Model robustness to prompt perturbations by aligning representations without sacrificing

Key concepts

Consensus Cross-Entropy (CCE)
This component uses a majority vote across different prompt variations to create a 'hard pseudo-label.' Only examples with a strict majority contribute to the loss calculation. This establishes the most likely correct answer based on what most prompts agree on.
Representation Alignment Loss
This loss pulls lower-confidence predictions toward the consensus established by high-confidence, majority variations. It uses methods like Jensen-Shannon divergence and KL divergence to ensure that different prompt styles lead to similar internal model representations, improving overall consistency.
Flip-Flop Consistency (F2C)
F2C is an unsupervised training algorithm designed to improve robustness against prompt perturbations. It combines CCE for pseudo-labeling with alignment losses to force the model's outputs and internal understandings to agree across different phrasing styles, enhancing reliability.
Prompt Perturbations
These are changes made to a prompt that do not change its core meaning but alter its surface form. Examples include changing formatting, casing, adding separators, paraphrasing the text, or reordering items in a few-shot example.

Terminology

Summary

The gist: Flip-Flop Consistency (F2C) is an unsupervised training method that improves Large Language Model robustness to prompt perturbations by aligning representations without sacrificing performance, showing significant gains in agreement, F1 score, and variance across various settings.

Introduction and Motivation

Large Language Models often produce inconsistent answers when faced with different phrasings of the same prompt because semantic consistency is a core requirement for trustworthy models in high-stakes settings like law and medicine The paper addresses the issue where LLM predictions can vary sharply under prompt perturbations such as formatting, casing, separators, paraphrasing, item ordering in few-shot settings, and other surface changes While existing methods like prompt engineering add computational overhead and supervised fine-tuning is limited by labeled data availability, F2C targets the gap of improving consistency in the absence of supervision while preserving task performance

Flip-Flop Consistency (F2C) Components

F2C is an unsupervised training algorithm that improves robustness to prompt perturbations by aligning their representations without sacrificing the performance It is composed of two key components:

  1. Consensus Cross-Entropy (CCE): This uses a majority vote across prompt variations to create a hard pseudo-label The instance-level CCE is defined as LCCE(i) = 1/ni,c⋆i > V/2 λCCE 1/V X V v=1 li,v, where only examples with a strict majority contribute to the loss.

  2. Representation Alignment Loss: This pulls lower-confidence and non-majority predictors toward the consensus established by high-confidence, majorityvoting variations. This is implemented through a combination of JSD loss within the Consensus set (Case 2) and KL divergence loss between the non-confident (NC) set and the Consensus Confidence (CC) set in Case 3.

Training Objective Formulation

The total loss per example is defined by a piecewise function based on the consensus state: LFF(i) = 0, Case 1, LCCE(i) + Ljsd(i), Case 2, LCCE(i) + Ljsd(i) + Lflip(i), Case 3. In Case 3, the alignment loss is defined as KLq⋆i,s∥ q¯T⋆i for the NC set Si and JSD loss within the CC set Ti to encourage agreement. The algorithm selects a Consensus-Confident (CC) set Ti and a Non-Confident (NC) set Si based on confidence margins, controlling the alignment intensity with wflip.

Evaluation and Results

F2C was evaluated on 11 datasets spanning four NLP tasks, showing that it significantly raises observed agreement (Po) on average by 11.62%, whereas swarm distillation slightly decreases it (−0.38%), showing a +12.00% agreement margin over swarm. Furthermore, F2C improves F1 on 9 out of 11 datasets with an average gain of +8.94% and reduces across-format σF1 by 3.29% on average. In out-of-domain evaluations, F2C successfully generalizes to the OOD data, yielding higher F1 on 74/80 train→test pairs and increasing Po on 64/80 compared to the base model.

Generalization Capabilities

The method demonstrates strong generalization to unseen prompt perturbations when trained on a subset of formats. For instance, for RTE, training with the first 5 formats and evaluating on the remaining 5 shows an F1 gain of 8.077 and an agreement increase of 3.255. This indicates that mats improves both performance and agreement on the held-out formats, and reduces across-format variance.

Conclusion

F2C consistently improves agreement, increases mean F1, and reduces variance across prompt formats, outperforming both swarm distillation and the CCE-only variant. The results suggest that much of the inconsistency from prompt phrasing can be mitigated by leveraging the model’s own internal consensus, without gold labels. Future work includes extending F2C to open-ended generation and exploring adaptive selection of high-confidence variations beyond top-K.

Limitations

Limitations include the fact that the method was fine-tuned on a 3B instruction-tuned model, which limits scalability across different models. Additionally, the evaluation is focused on classification tasks with discrete labels and does not study open-ended generation. The perturbations considered are non-adversarial template variants drawn from PromptSource, and the method assumes multiple semantically equivalent templates per instance. Finally, the effectiveness may degrade in settings with scarce or low-quality templates.

Ethical Considerations

The researchers used only publicly available datasets and openweight models, ensuring no new human data collection occurred. All datasets used are established benchmarks obtained via the HuggingFace datasets library or their official repositories, complying with original license terms. The corpora are designed for evaluating language understanding and reasoning and contain de-identified, nonsensitive text.

References

The references listed include works by OpenAI (2025), Brown et al. (2020), and Zhou et al. (2025). The paper also cites works such as Sclar et al. (2024) and Yang et al. (2024). The references cover various aspects of LLM consistency, such as prompt source and model editing techniques, and the evaluation metrics used in the study. The paper also cites works like Zhou et al. (2022) and Huang et al. (2023). The references cover various aspects of LLM consistency, such as prompt source and model editing techniques, and the evaluation metrics used in the study. The paper also cites works like Zhou et al. (2022) and Huang et al. (2023). The references cover various aspects of LLM consistency, such as prompt source and model editing techniques, and the evaluation metrics used in the study. The paper also cites works like Zhou et al. (2022) and Huang et al. (2023). The references cover various aspects of LLM consistency, such as prompt source and model editing techniques, and the evaluation metrics used in the study. The paper also cites works like Zhou et al. (2022) and Huang et al.

Improvements for AI systems

  1. Improved LLM robustness to prompt perturbations by implementing Flip-Flop Consistency (F2C), which uses a majority vote across prompt variations to create a hard pseudo-label and pull[s] lower-confidence and non-majority predictors toward the consensus established by high-confidence, majorityvoting variations. This allows the AI system to maintain semantic consistency when faced with different phrasings of the same prompt.

  2. Enhanced task performance by combining Consensus Cross-Entropy (CCE) with a representation alignment objective, resulting in F2C not only enhances consistency but also improves task performance as shown by an average gain of +8.94% in mean F1 across 9 out of 11 datasets. This makes the AI system more reliable for high-stakes applications like law and medicine.

  3. Improved generalization to unseen prompt variations by training on a subset of prompt formats and evaluating on held-out formats, where F2C consistently improves both performance and agreement while reducing variance. This allows the AI system to perform better when encountering novel phrasing it was not explicitly trained on.

Abstract

Large Language Models (LLMs) often produce inconsistent answers when faced with different phrasings of the same prompt. In this paper, we propose Flip-Flop Consistency (F 2C), an unsupervised training method that improves robustness to such perturbations. F 2C is composed of two key components. The first, Consensus Cross-Entropy (CCE), uses a majority vote across prompt variations to create a hard pseudo-label. The second is a representation alignment loss that pulls lower-confidence and non-majority predictors toward the consensus established by high-confidence, majority-voting variations. We evaluate our method on 11 datasets spanning four NLP tasks, with 4-15 prompt variations per dataset. On average, F 2C raises observed agreement by 11.62%, improves mean F 1 by 8.94%, and reduces performance variance across formats by 3.29%. In out-of-domain evaluations, F 2C generalizes effectively, increasing and agreement while decreasing variance across most source-target pairs. Finally, when trained on only a subset of prompt perturbations and evaluated on held-out formats, F 2C consistently improves both performance and agreement while reducing variance. These findings highlight F 2C as an effective unsupervised method for enhancing LLM consistency, performance, and generalization under prompt perturbations. Code is available at https://github.com/ParsaHejabi/Flip-Flop-Consistency-Unsupervised-Training-for-Robustness-to-Prompt-Perturbations-in-LLMs.

Sources

Related papers