CCQA: Generating Question from Solution Can Improve Inference-Time Reasoning in SLMs

arXiv:2509.18536 · cs.CL, cs.AI · Submitted 2025-09-23 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "CCQA: Generating Question from Solution Can Improve Inference-Time Reasoning in SLMs".

Jane: Cycle-Consistency in Question Answering (CCQA) is a novel reasoning method designed to improve inference-time reasoning accuracy for Small Language Models (SLMs) by generating and comparing questions derived from each candidate…

Tom: First, who's behind it and why it matters.

Title and authors: Jane: So, we covered the setup, and now let's look deeper into what CCQA actually does according to the paper. It explains that when a model gives multiple answers, it first checks if there is a clear majority response through majority voting. If that doesn't happen, they trigger the CCQA cycle.

Tom: Right, and page two details the exact flow of this process: if we hit what they call Low Confidence Voting or LCV—meaning the model’s answers are inconsistent with no clear winner—that’s when things get interesting. They regenerate questions from each candidate solution instead of just voting.

Lu: That LCV condition is where the real power seems to come into play, because it forces a deeper comparison than just looking at a simple majority vote; it checks the quality of every single path. It’s about evaluating reasoning paths rather than just picking an output based on popularity.

Meng: So, instead of relying solely on the model's internal confidence score or majority count when things are messy, they are actively probing each path with a new question to see if it aligns with our initial intent. That sounds like a more rigorous way to handle uncertainty.

Lalam: And that rigor is exactly what I think will make a big difference for the practical impact of these models. When we deploy AI systems, we need methods that don't just guess; they need verification steps like this to ensure reliability in real-world scenarios.

Tom: The paper emphasizes that under this LCV condition, the selection process shifts: they compare the generated questions against the original question and select the solution whose generated question has the highest similarity score. This is their final decision-making step for that specific situation.

Jane: So, if we put it simply, when we're unsure if any single answer is right, CCQA doesn't just guess; it asks each potential answer a question and sees which one answers our original query the best in terms of similarity.

The paper's summary: Tom: Now let’s talk about what they actually suggest as improvements to existing methods. They are pointing out that traditional strategies like self-consistency can fall apart when SLMs produce highly inconsistent outputs, which is a big concern for smaller models.

Lu: The authors propose that instead of just stopping at a vote, we should construct this cycle—the original question, the solution, and the new question—to leverage that principle of reversibility they mentioned earlier to improve overall quality.

Meng: The real improvement here is moving away from just relying on surface-level consistency to actually verifying the reasoning path itself through this generated question mechanism. It’s about validating *why* a model arrived at an answer, not just *what* the answer is.

Lalam: And from my view, that focus on reasoning path verification is significant because it addresses a fundamental weakness in how we currently evaluate inference time performance for these smaller models. It shows us where the real bottlenecks are.

Jane: They also introduce a specific way to score this comparison, using a combination of lexical overlap and semantic meaning to determine the best match. They set weights for this scoring system, with zero point four for BLEU score and zero point six for cosine similarity when they are doing backward question generation in LCV situations.

Tom: That weighted scoring system sounds very practical; balancing structure with actual meaning is smart because it ensures we’re selecting a solution that is both structurally sound and conceptually accurate relative to the original intent of the query.

The paper's improvements: Jane: So, wrapping up this discussion on CCQA, the main implication is that we have a new, lightweight technique that significantly boosts inference-time reasoning accuracy for SLMs by introducing a self-checking loop when things get uncertain.

Tom: Precisely, Jane; it’s about providing a robust way to handle those tricky situations where smaller models might otherwise just give us garbage outputs. The experimental results they shared show that CCQA outperforms existing state-of-the-art methods across most benchmarks and SLMs tested.

Lu: I think the biggest implication is that this framework gives us a more sophisticated tool for probing the reasoning capabilities of SLMs in ways that traditional voting methods simply can't handle effectively. It opens up new avenues for testing model intelligence on these smaller architectures.

Meng: From my side, it’s good to see a method that offers substantial accuracy gains without demanding an enormous increase in computational resources compared to other complex inference strategies we already use. That resource balance is something I really value in practical deployment.

Lalam: Overall, I feel very positive about this work; the paper "CCQA: Generating Question from Solution Can Improve Inference-Time Reasoning in SLMs" gives us a concrete strategy for enhancing the reasoning capabilities of these models efficiently and reliably, which is exactly what we need moving forward.

Conclusion: Tom: So we've been diving deep into CCQA: Generating Question from Solution Can Improve Inference-Time Reasoning in SLMs, and to wrap up, it seems this method gives SLMs a really smart way to self-verify their answers when things get tricky.

Jane: Exactly, Tom; the core idea is that instead of just relying on a simple vote when the model is inconsistent, CCQA forces it to generate a new question from its reasoning path and checks how well that new question matches our original query.

Lu: I think what’s fascinating here is how they manage to balance lexical similarity with semantic understanding through that weighted scoring system; it’s a really creative way to make the AI check its own work in a meaningful way.

Meng: From an engineering standpoint, the efficiency of using a lightweight model like Flan-T5 for that question generation step is pretty impressive, especially since we need strategies that don't add too much overhead to our deployment pipelines.

Lalam: I think this work has huge implications for how we design trust in AI systems; if models can self-validate their reasoning paths rigorously, it really helps build a more reliable culture around the outputs we get from these powerful tools.

Tom: It really does; when we see SLMs performing better on those complex math and commonsense tasks, it shows that inference-time strategies don't have to be overly complicated to get significant gains in accuracy.

Jane: That’s the key point, Tom; CCQA proves that a focused approach on validating the reasoning steps is much more effective than just relying on broad consistency checks.

Lu: Think about the future possibilities here; if we can scale this cycle-consistency idea across different types of generative tasks, imagine how robust our complex world models could become.

Meng: I wonder what kind of real-world applications emerge next; right now, it’s strong on benchmarks, but how does this translate when a model is trying to solve a highly specific engineering problem?

Lalam: I see this advancing the culture of AI development by showing us that focusing on the quality of internal logic, rather than just the final output score, is where we should be spending our design efforts.

Tom: Exactly; so while CCQA offers a solid improvement for current SLMs, we’ve got plenty more to explore in how these verification techniques can evolve.

Jane: And that brings us to our next topic: let's look at the recent work on persistent tri-state message passing and see how it tackles structural uncertainty in learning.

Jin Young Kim, Ji Won Yoon

Department of Artificial Intelligence, Chung-Ang University

cs.CL, cs.AI

Submitted: 2025-09-23

Updated: 2025-09-23

Code: https://github.com/scai-research/ccqa_official

Importance score: 90/100

The gist: Cycle-Consistency in Question Answering (CCQA) is a novel reasoning method designed to improve inference-time reasoning accuracy for Small Language Models (SLMs) by generating and comparing questions

Key concepts

Cycle-Consistency in Question Answering (CCQA)
CCQA is a novel method where a new question is created from every potential answer or reasoning step generated by an SLM. It then compares these new questions to the original one to pick the best solution. This creates a cycle between the original query, the model's output, and a derived question to assess quality.
Backward Question Generation
When standard voting methods don't yield a clear winner, CCQA uses this technique. It takes each reasoning path separately and feeds it into a specialized model to generate a new question. This helps evaluate the quality of individual reasoning steps in low-confidence scenarios.
Similarity Measurement
The method measures how well the newly generated question matches the original one using two metrics: BLEU score (measuring word overlap) and cosine similarity (measuring semantic meaning). A weighted combination of these scores determines which candidate solution is the most relevant to the initial query.

Terminology

Summary

Cycle-Consistency in Question Answering (CCQA) is a novel reasoning method designed to improve inference-time reasoning accuracy for Small Language Models (SLMs) by generating and comparing questions derived from each candidate solution. This approach addresses the performance degradation observed when conventional self-feedback or voting strategies fail on smaller models, establishing CCQA as a new practical baseline for efficient SLM reasoning across mathematical and commonsense benchmarks.

The gist

CCQA generates a question from each reasoning path and answer, evaluates each by its similarity to the original question, and then selects the candidate solution with the highest similarity score as the final response.

Motivation for CCQA

Prior studies on inference-time reasoning strategies like chain-of-thought (CoT) and self-consistency (SC) have shown performance degradation when applied to SLMs. This degradation stems from two main factors: first, smaller models struggle to understand complex inputs and follow instructions; second, voting-based approaches like SC become ineffective when SLMs produce highly inconsistent outputs, as majority voting fails to evaluate the quality of reasoning paths. CCQA is proposed as a solution inspired by cycle consistency (Hoffman et al., 2018), constructing a cycle between the original question, the solution produced by the SLM, and a new question generated from that solution.

The CCQA Framework

The overall process of CCQA involves several key steps:

  1. The SLM first receives the original question and produces multiple candidate solutions, including reasoning paths (RPs) and answers.

  2. When there is no dominant response during majority voting, CCQA generates a new question from each candidate solution using a lightweight Flan-T5 model specialized for question generation.

  3. It measures both lexical and semantic similarity between each generated question and the original one to assign a similarity score.

  4. The candidate solution whose generated question most closely matches the original is selected as the final response, without requiring the model to process any additional complex input.

Backward Question Generation and Similarity Measurement

To evaluate reasoning path quality in Low Confidence Voting (LCV) situations—defined when no clear majority exists—CCQA employs backward question generation. In these cases, each reasoning path (RPi) is used as input to the fine-tuned T5 model to generate a question (GQi). The final selection is based on a weighted sum of BLEU score and cosine similarity:

score(GQi, OQ) = α · BLEU(GQi, OQ) + β · cosine(GQi, OQ).

The authors determined optimal weights by setting α to 0.4 and β to 0.6. This balanced combination of lexical structure (BLEU) and semantic meaning (cosine similarity) provides the most effective similarity measure for identifying accurate reasoning paths.

Experimental Validation

Extensive experiments were conducted on six reasoning benchmarks, including four mathematical tasks (GSM8K, SVAMP, Multi-Arith) and four commonsense tasks (CSQA, StrategyQA, ARC-Challenge), across eight SLMs ranging from 135M to 3B parameters. The results consistently verified that CCQA outperforms existing state-of-the-art (SOTA) reasoning methods across most SLMs and benchmarks. Notably, CCQA with Llama3.2-3B on GSM8K achieved 69.60% accuracy compared to USC’s 53.83%, demonstrating its effectiveness in enhancing SLM reasoning capabilities, especially under LCV conditions where SC performed poorly.

Conclusion and Contributions

The main contributions of the paper are:

: Our paper introduces a novel inference-time reasoning technique for SLMs, namely CCQA, that evaluates the quality of each reasoning path and its answer by regenerating a question and measuring its similarity to the original. To the best of our knowledge, this is the first attempt to investigate the inferencetime reasoning capabilities of SLMs and to improve them. 2.

: We leverage a lightweight Flan-T5 model to generate questions from candidate solutions. Compared to conventional SLMs, our finetuned Flan-T5 is computationally efficient and produces higher-quality questions. 3.

: Our extensive experiments across diverse benchmarks and SLMs demonstrate that CCQA consistently outperforms SOTA reasoning methods, substantially improving reasoning capabilities of SLMs. 4.

CCQA provides a robust performance-resource balance, achieving significant accuracy gains with only marginal additional computational cost compared to other inference-time strategies. While the effectiveness depends on the quality of the backward question generator, its lightweight design and substantial performance gains make it an essential strategy for enhancing SLM reasoning.

Improvements for AI systems

As a fastidious researcher, I have analyzed the provided paper on Cycle-Consistency in Question Answering (CCQA) for Small Language Models (SLMs). Based on my understanding of the proposed methodology, here are the specific improvements that can be implemented in AI systems and what those improved systems will be capable of:

The core improvement lies in shifting from relying solely on majority voting or self-feedback mechanisms—which fail when SLMs produce inconsistent outputs—to a mechanism that validates reasoning paths by testing their coherence against the original problem statement.

Here are the specific improvements and capabilities:

  1. Maturity of Inference-Time Reasoning for SLMs:

  2. CCQA (Cycle-Consistency in Question Answering) is introduced as a novel, lightweight inference-time reasoning framework specifically designed to enhance Small Language Models (SLMs).

  3. Robustness Under Low Confidence Voting (LCV): The system can reliably perform well even when the SLM outputs highly varied and inconsistent answers.

  4. Path-Quality Verification: Unlike conventional methods that ignore the quality of internal reasoning steps, CCQA explicitly evaluates each generated solution by regenerating a question from its reasoning path and measuring its similarity to the original question.

  5. Lightweight Question Generation Module: The system leverages a fine-tuned Flan-T5 model to efficiently generate high-quality questions from candidate solutions. This addresses the known limitation of SLMs in generating accurate questions from their own reasoning paths.

  6. Optimized Similarity Scoring: The similarity metric is a weighted combination of both surface-level lexical overlap (BLEU) and deep semantic correspondence (embedding-based cosine similarity), optimized by empirically determined weights (e.g., 0.4 for BLEU, 0.6 for cosine). This ensures that the system selects solutions that are not only structurally similar but also conceptually correct relative to the original intent.

The improved AI system will be capable of the following specific tasks:

  1. Improved Accuracy on Reasoning Benchmarks (GSM8K, SVAMP, ARC-Challenge): The system will consistently outperform state-of-the-art methods across mathematical and commonsense reasoning tasks on SLMs (e.g., achieving up to 69.60% accuracy on GSM8K for Llama3.2-3B).

  2. Enhanced Performance in Inconsistent Output Scenarios: The system will mitigate the performance degradation observed when SLMs produce diverse outputs, as demonstrated by its superior handling of Low Confidence Voting (LCV) conditions compared to Self-Consistency (SC).

  3. Effective Reasoning with Smaller Models: The framework allows for significant performance gains on resource-constrained SLMs (e.g., SmolLM2 variants), demonstrating that sophisticated inference strategies can be effectively applied where traditional methods fail.

  4. Efficient Reasoning Under Inconsistency: The system provides a favorable performance-resource balance, achieving substantial accuracy gains with only marginal additional computational overhead compared to more complex inference strategies like Chain-of-Thought (CoT) or multiple forward passes of other methods.

  5. Superior Question Generation Capability: By integrating the fine-tuned Flan-T5, the system ensures that every candidate solution is rigorously vetted by generating a question specifically tailored to confirm if that solution actually solves the original problem as intended.

Sources

Related papers