Last Layer Logits to Logic: Empowering LLMs with Logic-Consistent Structured Knowledge Reasoning

arXiv:2511.07910 · cs.CL · Submitted 2025-11-11 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Last Layer Logits to Logic".

Jane: Large Language Models (LLMs) struggle to maintain logic consistency in structured knowledge reasoning tasks like Knowledge Graph Question Answering (KGQA) due to representational differences between unstructured and structured knowledge,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we’ve talked about what this paper is trying to do generally, and now let’s look at who did it. The title, "Last Layer Logits to Logic: Empowering LLMs with Logic-Consistent Structured Knowledge Reasoning," really tells you exactly where the focus is—on the logits layer.

Jane: And it’s interesting that they list authors from different groups, including Ant Group and Zhejiang University, which suggests a strong collaboration between model development and deep knowledge graph research.

Lu: I find it telling that they are building this framework to address the representational differences between unstructured text and structured knowledge; that gap is where most current reasoning methods hit their wall.

Meng: The authors’ focus on moving beyond input-level guidance, as mentioned in the abstract, tells me they recognize that existing work, like ToT or GoT, isn't actually fixing the fundamental issue of output logic consistency.

Lalam: That distinction is important because it means this approach isn't just another prompt trick; it’s a structural improvement to how we make these AI systems reason over data.

Tom: Right, so the title points to a specific technical fix targeting the last layer of generation, and that gives us a clear idea of what they are focusing on for this discussion.

Jane: It really emphasizes that the goal is to instill logic consistency directly into the model's probabilistic predictions rather than just steering it through intermediate reasoning steps in natural language.

Lu: Thinking about the authors, it shows they’ve put together a team capable of bridging both the theoretical modeling of logical paths and the practical application within large knowledge graphs.

Meng: It makes sense that they are looking at structure constraints because if we don't enforce KG rules, any model will eventually start hallucinating connections, which is a huge risk for us when deploying AI agents.

Lalam: If we can make the reasoning process inherently structured by these modules, it helps create a culture of higher quality output that reduces the need for constant manual fact-checking downstream.

The paper's summary: Tom: Okay, let's get into the meat of what this paper actually proposes. They introduce the Logits-to-Logic framework, which centers around three key stages: Logic Compiling, Logits Strengthening (Zs), and Logits Filtering (Zf).

Jane: In simple terms, they take the logical constraints from a knowledge graph and compile them into a format called a Non-deterministic Finite Automaton or NFA. This NFA essentially maps out all the legal paths that can exist in the KG for any given question.

Lu: That’s where I get really excited; modeling the entire set of legal reasoning paths as an NFA is a powerful way to capture all possible correct trajectories, which is much more robust than relying on linear chain-of-thought methods.

Meng: So, the next step they propose is using sentence transformers to score these legal paths, which helps them determine which logical steps are semantically relevant to the question's actual meaning. That’s a nice way to filter out irrelevant reasoning ideas early on.

Lalam: And then they use those scores in the strengthening module to boost the probability of selecting those correct, logic-aligned paths when generating tokens. It’s like giving the model a strong nudge toward what is actually true about the data we are querying.

Tom: Then there's filtering, where they use that NFA's transition function to actively stop the model from picking tokens that don't belong on any legal path, setting those illegal tokens’ probability to negative infinity. That’s a very direct way to enforce structure.

Jane: So, the summary boils down to building a system that compiles logic into an NFA, strengthens the correct paths with learned scores, and then filters out any output that deviates from those pre-defined legal paths within the knowledge graph.

Lu: It’s a clever way to bridge the gap between the LLM’s fluent language generation and the rigid constraints of structured data representation.

The paper's improvements: Tom: Now let's talk about what they claim these improvements actually achieve in practice. They report that this method significantly boosts logic consistency across multiple Knowledge Graph Question Answering benchmarks, achieving state-of-the-art performance on tasks like Multi-hop QA and Single-hop QA.

Jane: The experimental results show that the framework improves LLMs’ logic consistency, and they also showed it’s directly transferable to different knowledge graphs and even different types of reasoning tasks without needing a complete overhaul.

Lu: The ablation studies are quite compelling; they show that both the strengthening module and the filtering module are necessary for good performance, indicating that you need both components working together to achieve this level of alignment.

Meng: I see what you mean; removing the strengthening module caused performance drops of one point five percent and six point nine percent on CWQ and WebQSP, which suggests that without it, the model just isn't focusing enough on the question's semantic logic within those paths.

Lalam: And taking out the filtering module caused a drop in alignment with structured KG logical distributions, which really proves that enforcing structural rules is critical for preventing outright errors.

Tom: So what’s the practical implication of these results? It means this method isn't just theoretical; it actually delivers state-of-the-art performance on real, complex reasoning benchmarks.

Jane: This has huge implications because it suggests we can build AI systems that perform better at retrieving and synthesizing information from structured knowledge, which is exactly what we need for many enterprise applications.

Conclusion: Tom: So to wrap things up on "Last Layer Logits to Logic," the paper provides a flexible and transferable framework by unifying the LLM's generation process with the KG structure through that NFA approach.

Jane: It achieves precise logical reasoning by making sure every token generated adheres to a verifiable path within the knowledge graph, which is a really elegant way to handle structured data constraints.

Lu: The main limitation they acknowledge is that because the search space for correct reasoning paths is so large, even with beam search, there's still an inevitable number of incorrect reasoning paths that can slip through.

Meng: That’s a fair caution; we have to be prepared for those occasional errors in deployment, but the computational efficiency gains are also significant when comparing it to methods like ToG or DoG.

Lalam: And it’s important to remember that this method is flexible enough to work across different KGs and tasks, which makes it a very versatile tool for our AI ecosystem.

Tom: Fantastic overview, team. It sounds like we have a really solid new direction for making structured reasoning in AI much more reliable by looking at the logits distribution from the output side.

Jane: It certainly does, Tom. We’ll keep an eye on this paper closely as they explore how to push these capabilities even further in future work.

Songze Li, Zhiqiang Liu, Zhaoyan Gong, Xiaoke Guo, Zhongpu Bo, Zhengke Gui, Lei Liang

University of Zhejiang University

cs.CL

Submitted: 2025-11-11

Updated: 2026-10-02

Importance score: 91/100

The gist: Large Language Models (LLMs) struggle to maintain logic consistency in structured knowledge reasoning tasks like Knowledge Graph Question Answering (KGQA) due to representational differences between

Key concepts

Logic Drift
This occurs when an LLM's reasoning output does not match the actual logical possibilities within a Knowledge Graph or question intent. It manifests as either generating steps that don't exist in the KG or producing steps that are semantically unrelated to the user's query, causing inconsistent answers.
NFA (Non-deterministic Finite Automaton)
An NFA is used to model all possible legal reasoning paths within a Knowledge Graph. Each state represents a valid step in the reasoning process. This structure helps define exactly what constitutes a logically correct sequence of actions or tokens for an LLM to follow when generating an answer.
Logits Strengthening (Zs)
This module boosts the probability scores (logits) of tokens that align with the question's semantic logic within the defined legal paths. It uses differentiation and scaling techniques to make correct reasoning steps more likely to be chosen by the LLM during generation.
Logits Filtering (Zf)
This process constrains the LLM by setting transition function values for tokens that are not part of any legal path in the NFA. By assigning extremely low scores (like negative infinity) to illegal tokens, it forces the model to only generate sequences that follow valid KG logic.

Terminology

Summary

Large Language Models (LLMs) struggle to maintain logic consistency in structured knowledge reasoning tasks like Knowledge Graph Question Answering (KGQA) due to representational differences between unstructured and structured knowledge, leading to Logic Drift where LLMs output reasoning paths that are either semantically irrelevant or logically inconsistent with the question intent or KG structure. This paper proposes the Logits-to-Logic framework, which targets the logits output from autoregressive generation by incorporating logits strengthening and filtering as core modules to correct logical defects in LLM outputs, achieving state-of-the-art performance on multiple KGQA benchmarks.

The gist

The Logits-to-Logic framework aligns LLMs’ last-layer logits distribution with the logical distributions of the question and knowledge graph through three stages: Logic Compiling, Logits Strengthening (Zs), and Logits Filtering (Zf), fundamentally addressing Logic Drift from the output perspective.

Problem Definition and Logic Drift

The key challenge identified is the inconsistency between LLMs’ outputs and the logical distributions of KG and question. This manifests in two forms of Logic Drift: (1) LLMs often output reasoning paths that do not exist in the KG (KG-Inconsistent Logic Drift), and (2) LLMs generate reasoning paths that are semantically irrelevant to the question logic (Question-Inconsistent Logic Drift). The paper models State Transfer Reasoning as a process where LLM outputs depend on previous sequences, which can be modeled using a Non-deterministic Finite Automaton (NFA) where accepting states represent all legal reasoning paths in the KG.

Logits-to-Logic Framework

The framework proceeds in three stages:

  1. Logic Compiling: This stage involves compiling legal paths in the KG into an NFA and scoring these paths using a sentence transformer to obtain scores indicate reasoning paths similar to question semantic logic. This prepares for aligning LLMs’ outputs with the logical distributions of KG and question.

  2. Logits Strengthening (Zs): This module enhances logits values that align with the question’s semantic logic in the NFA using differentiation and scaling techniques to boost the probability of correct answer paths, aiming to make the logits distribution closer to question logic.

  3. Logits Filtering (Zf): This module constrains logits values corresponding to tokens that do not belong to legal paths in the NFA by setting transition function values, specifically using NFA’s transition function δ to guide LLMs in generating legal paths, such as setting δ for illegal tokens like art and award to −∞.

Key Contributions and Results

The major contributions include:

We propose the Logits-to-Logic framework and design logits strengthening and logits filtering modules to fundamentally address the Logic Drift from the output perspective.

"Extensive experimental results on multiple KGQA benchmarks demonstrate that our method significantly improves LLMs’ logic consistency in structured knowledge reasoning and achieves state-of-the-art performance, while being directly transferable to different KGs and tasks."

Experimental results show that Logits-to-Logic outperforms baselines across multiple datasets (CWQ, WebQSP, GrailQA) and tasks (Multi-hop QA, Single-hop QA, Slot Filling). Ablation studies confirm the necessity of both modules: Removing Zs decreases performance by 1.5% and 6.9% on WebQSP and CWQ respectively, indicating that logits strengthening is crucial for aligning outputs with question logic, while Removing Zf prevents LLMs outputs from aligning with structured KG logical distribution.

Computational Efficiency

The framework demonstrates significant computational overhead advantages compared to other methods. The paper shows that Logits-to-Logic achieves remarkable reductions of 89%, 96%, and 84% in token consumption on the CWQ dataset when compared to ToG, DoG, and PoG. Furthermore, it requires no additional Model API calls beyond a single call per question when benchmarked against GCR, resulting in substantial improvement over methods that necessitate multiple iterative calls. The computational complexity for constructing NFAs is analyzed as NT × RD (NT tokens per path length times the average number of paths), with experimental analysis showing feasibility on large-scale KGs.

Conclusion and Limitations

The paper concludes that Logits-to-Logic provides a flexible and transferable logic-consistency reasoning framework by unifying the LLM’s autoregressive generation and the KG’s structure within an NFA, thereby achieving precise logical reasoning. The primary limitation noted is that due to the excessively large search space for correct reasoning paths, even with beam search, there is an inevitable number of incorrect reasoning paths introduced. However, this method maintains robustness and flexibility across different KGs and tasks.

Improvements for AI systems

Here are specific improvements for AI systems based on the Logits-to-Logic framework, along with what those improved systems can achieve:

  1. The core improvement lies in shifting logic consistency enforcement from expensive, input-level prompt engineering (like ToT or DoG) to a low-latency, output-level mechanism operating directly on the model's last layer logits.

  2. This allows for the creation of a Logic Correction Layer that acts as a lightweight, dynamic filter/strengthener during the autoregressive decoding phase, rather than relying solely on complex pre-prompted agentic workflows.

  3. The improved system can maintain high logical fidelity when reasoning over structured data (Knowledge Graphs) because it explicitly maps the LLM's probabilistic output distribution onto the mathematically defined logical distributions of both the question and the KG structure (the NFA).

  4. Specifically, the system can:

Ease complex multi-hop Knowledge Graph Question Answering (KGQA) by ensuring every generated token adheres to a verifiable, legal path within the graph structure. It will eliminate hallucinated paths (Gray highlights in Figure 2) and semantically irrelevant reasoning steps (Orange highlights), leading to significantly higher precision and recall on benchmarks like CWQ and WebQSP.

  1. The system can achieve superior computational efficiency compared to agent-based methods by requiring only a single LLM API call per question, drastically reducing token consumption (up to 89% reduction over ToG/DoG) and lowering operational costs for large-scale deployment.

  2. The system can adapt flexibly across diverse knowledge graphs and tasks (Multi-hop QA, Single-hop QA, Slot Filling) without requiring task-specific workflow redesigns or extensive fine-tuning of agent roles, due to its reliance on the general structure of the KG and question logic within a transferable NFA framework.

  3. The system can be optimized for deployment using smaller, more efficient LLM backbones (like LLaMA-3.1-8B) while achieving performance comparable to or exceeding larger proprietary models (like GPT-4), making high-quality structured reasoning accessible on consumer hardware with manageable memory footprints.

  4. The system provides a robust diagnostic capability: by analyzing the logits distribution, it can pinpoint exactly where logic drift occurs (i.e., whether the error is due to question misalignment or KG structural inconsistency), allowing developers to fine-tune either the input scoring model or the filtering rules specifically for that failure mode.

Sources

Related papers