Beyond Pixels: A Vector-to-Graph Framework for Reliable Schematic Auditing

arXiv:2602.11678 · cs.AI, cs.CV · Submitted 2026-02-12 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "Beyond Pixels: A Vector-to-Graph Framework for Reliable Schematic Auditing".

Tom: Multimodal Large Language Models (MLLMs) suffer from structural blindness when analyzing engineering schematics because their pixel-driven paradigm discards explicit vector-defined relations necessary for topological reasoning.

Jane: First, who's behind it and why it matters.

Title and authors: Tom: The title itself, "Beyond Pixels," really tells you that this work is about moving past just interpreting the raw image data and getting to a deeper level of understanding the schematic structure. The authors are Chengwei Ma, Zhen Tian, Zhou Zhou, Zhixian Xu, Xiaowei Zhu, Xia Hua, Si Shi, and F. Richard Yu from institutions like Guangdong Laboratory of Artificial Intelligence and Digital Economy and Carleton University.

Jane: It’s interesting that they're using a vector-to-graph approach because it tackles the problem head-on by converting those visual diagrams into something mathematically structured—a property graph—where connections are defined by explicit rules rather than just visual proximity.

Lu: The implications here are huge; if you can build a system that understands topology symbolically, you unlock a new level of reasoning for AI in complex technical fields where understanding the 'how' is as important as understanding the 'what'.

Meng: I’m thinking about how this impacts practical engineering workflows. If we can integrate this into design review tools, it could automate checks that currently require highly specialized human auditors to find subtle wiring errors.

Lalam: I see a massive cultural implication here; by making the underlying structure of systems auditable and machine-readable, we are moving toward a generation of AI that operates with verifiable integrity rather than just generating plausible outputs based on visual cues alone.

The paper's summary: Tom: Basically, the core idea is that current models fail because they are pixel-driven, discarding vector definitions necessary for topology, so this paper proposes a Vector-to-Graph pipeline that turns CAD diagrams into property graphs where nodes are components and edges show connectivity.

Jane: So it’s not just about identifying "a resistor" in an image; it’s about mapping out exactly which terminals are connected to which, and the paper shows they use deterministic Graph Signal Processing operators to verify those connections mathematically.

Lu: The methodology involves a few stages: first parsing with ezdxf, then using an LLM pipeline to extract nodes, edges based on geometric heuristics and interpretation, and finally using an MLLM planner that turns natural language rules into structured queries against this graph.

Meng: That sounds like a very solid plan for handling complex relational data; the idea of combining LLM planning with deterministic verification seems like a way to balance flexibility with precision.

Lalam: I think the summary really highlights how they are bridging the gap between high-level visual understanding and low-level symbolic logic by creating this explicit graph structure as an intermediary step.

The paper's improvements: Tom: The authors highlight several key enhancements, specifically focusing on how they design that diagnostic probe to isolate topological reasoning failures, showing exactly where current models fall short in tasks like multi-point grounding checks.

Jane: They show that the V2G framework is designed to preserve relational information lost during pixel processing by explicitly encoding topology into the graph structure, which allows for verification through spectral graph tools.

Lu: One of the specific improvements they detail is how the MLLM planner interprets natural language compliance rules like "Every CT secondary must connect to exactly one ground" into structured queries, which is a significant step in making model behavior predictable.

Meng: From an implementation view, I find that mapping those linguistic rules to specific verification functions within a defined library F makes the system far more controllable than relying on the MLLM to just "guess" the correct connection pattern every time.

Lalam: The improvement mentioned regarding how they handle spatial perturbations, like rotation or translation of schematics, is really significant because it means their approach is invariant to those visual changes; it focuses on symbolic vector relationships instead of exact pixel matching.

Conclusion: Tom: To wrap up this discussion on "Beyond Pixels," we see that converting schematics into property graphs and verifying compliance with deterministic Graph Signal Processing offers a reliable way to overcome structural blindness in multimodal AI for engineering tasks. The accuracy gains they reported across various error categories are quite substantial.

Jane: It confirms that making structure explicit through a Vector-to-Graph pipeline is essential when you need reliable auditing in domains where structural constraints are paramount, moving us toward systems that can reason about logic rather than just recognizing visual patterns.

Lu: I think the real impact lies in how this opens up avenues for creating AI agents that don't just see objects but actively reason over the symbolic structure of the world they are interacting with.

Meng: For me, it’s about moving from probabilistic checks to mathematically verifiable ones, which is a huge step for any system that needs to be trusted in a professional setting.

Lalam: I feel like this work sets a new standard for how we should think about multimodal AI; focusing on explicit structure ensures that the resulting systems are not just smarter visually but also fundamentally more reliable and auditable.

Tom: Fantastic insights, everyone. We’ve covered a lot about how this Vector-to-Graph Framework tackles the structural blindness in schematic auditing. We'll be looking forward to seeing how this technique evolves in future work.

Chengwei Ma, Zhen Tian, Zhou Zhou, Zhixian Xu, Xiaowei Zhu, Xia Hua, Si Shi

Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Guangdong Power Grid Co., Ltd. · Shanghai University · Carleton University

cs.AI, cs.CV

Submitted: 2026-02-12

Updated: 2026-09-29

Comments: 4 pages, 3 figures. Published in ICASSP 2026

DOI: 10.1109/ICASSP55912.2026.11460425

Code: https://github.com/gm-embodied/V2GAudit

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 86/100

The gist: Multimodal Large Language Models (MLLMs) suffer from structural blindness when analyzing engineering schematics because their pixel-driven paradigm discards explicit vector-defined relations

Key concepts

Vector-to-Graph Pipeline
This is the core methodology where CAD diagrams are converted into property graphs. In this structure, components are nodes and connections are edges. This process moves beyond raw image interpretation to create a mathematically structured representation of the schematic's topology.
Property Graph
A mathematical structure used to represent data where nodes (like components) have properties, and edges (like connectivity) explicitly define the relationships between them. This allows for symbolic reasoning about how parts are connected, rather than just visual proximity.
Graph Signal Processing
This is a set of deterministic operators used to verify connections within the graph structure mathematically. It is employed to check if the defined topological relationships in the schematic are correct, providing a precise, verifiable method for auditing.
Structural Blindness
A limitation in multimodal large language models where their pixel-driven approach discards explicit vector definitions necessary for topological reasoning. This causes them to fail at understanding the underlying structural relationships in engineering schematics.

Terminology

Summary

Multimodal Large Language Models (MLLMs) suffer from structural blindness when analyzing engineering schematics because their pixel-driven paradigm discards explicit vector-defined relations necessary for topological reasoning. This paper proposes a Vector-to-Graph (V2G) pipeline that converts CAD diagrams into property graphs, enabling structure-aware verification through deterministic Graph Signal Processing (GSP), thereby providing a reliable path toward practical deployment of multimodal AI in engineering domains where structural constraints are paramount.

Exposing Structural Blindness

The paper first demonstrates the limitations of current MLLMs by designing a targeted diagnostic probe to isolate topological reasoning failures, inspired by benchmarks like CLEVR and SpatialEval. The core failure mode highlighted is the inability to perform multi-point grounding, such as ensuring a Current Transformer (CT) secondary connects to exactly one ground symbol. While leading high-resolution MLLMs can correctly identify components like current transformers and grounding symbols, they consistently fail to determine if two ground points are connected to the same circuit, often producing confident but incorrect answers or hedging with ambiguous regions. This behavior exemplifies structural blindness: MLLMs act as “bag-of-objects” recognizers lacking the ability to recover the graph structure implicit in vectordefined schematics.

Proposed Framework: Vector-to-Graph with Graph-Level Verification

The V2G framework addresses pixel-based methods by converting CAD schematics into a property graph and integrating an MLLM planner with deterministic GSP verification. This process is divided into four stages:

  1. V2G Transformation: CAD schematics are parsed using the ezdxf library to obtain low-level primitives, which are then processed by an LLM-driven pipeline to sequentially extract nodes (components), edges (connectivity inferred via geometric heuristics and LLM interpretation), and attributes (terminal IDs, polarity). The result is a property graph G = (V, E, X) that preserves both symbolic semantics and explicit topology.

  2. MLLM Planner: Compliance rules are expressed in natural language prompts. The MLLM interprets these rules into structured queries of the form at = (Rt, ft), where Rt is a relevant subgraph of G and ft is a verifier function from a library F. This involves Rule Interpretation, Region Selection (aligning keywords with node types), and Function Selection (retrieving the appropriate verification operator).

  3. GSP Verifier: For the selected subgraph Rt = (Vt, Et, Xt), connectivity is verified using spectral graph tools. Connectivity is determined by checking if the number of connected components equals the multiplicity of the zero eigenvalue, c = multλ=0(Lt), or equivalently rank(Lt) = Vt − c. Specific compliance checks are performed using defined functions: grounding uniqueness is tested by g(Rt) = X v∈Vt 1[type(v) = Ground], and polarity/phase consistency is enforced through attribute constraints catt(Rt).

  4. Compliance Report: The outputs are aggregated as O = (ϕj, oj), providing structured JSON with violation flags and natural language summaries for human auditors.

Intropy Analysis

The paper introduces Intropy, a measure quantifying intelligence as adaptive efficiency, defined by dL = δS/R, where δS is meaningful discrepancy reduction and R denotes internal resistance such as uncertainty or representation mismatch. Pixel-based schematic understanding exhibits low Intropy due to high structural resistance because topological relations remain implicit. The proposed V2G framework reduces this resistance by explicitly encoding schematic structure as graphs, increasing δS while lowering R, thus yielding more efficient and robust auditing outcomes.

Experimental Validation

The framework was tested on a benchmark of approximately N≈900 test instances derived from real engineering schematics, covering connection labeling, grounding (multi-point), and wiring checks across three categories. The results show significant performance gains: overall accuracy increased from 12% to 47% when using the +V2G setting across six tested VLMs. The largest improvements were observed in Conn. (+61%), Ground. (+27%), and Wiring (+20%). Ablation studies confirmed that topology, attributes, and planner-guided subgraph selection each contribute to the full performance gain, demonstrating that explicit structure is essential for reliable multimodal auditing in engineering domains.

Conclusion

The V2G framework successfully overcomes the structural blindness of MLLMs by transforming CAD drawings into property graphs and verifying compliance via deterministic GSP functions. This approach confirms that making structure explicit is essential for achieving reliable multimodal auditing in engineering domains, as evidenced by substantial accuracy improvements across tested models on a realistic diagnostic benchmark.

References

[1] Yifan Hou, Buse Giledereli, Yilei Tu, and Mrinmaya Sachan, “Do vision-language models really understand visual language?,” arXiv preprint arXiv:2410.00193, 2024.

Improvements for AI systems

Here are the specific improvements for AI systems based on the proposed Vector-to-Graph (V2G) framework:

  1. Replace reliance on pixel-based vision encoders for schematic auditing with a hybrid pipeline that first converts CAD diagrams into explicit property graphs and then uses these graphs to guide reasoning.

  2. Implement a deterministic Graph Signal Processing (GSP) verification module to replace probabilistic or learned relational checks, ensuring compliance decisions are mathematically verifiable and auditable.

  3. Augment Multimodal Large Language Models (MLLMs) with the structural property graph representation as explicit input context, shifting their role from purely visual parsers to reasoning planners that operate over symbolic topology.

  4. Enable MLLM-driven Rule Interpretation where natural language compliance rules are translated directly into structured queries against the graph, allowing the model to systematically search for topological violations rather than relying on learned heuristics for complex connectivity checks (e.g., grounding uniqueness).

This improved AI system can perform:

  1. Precise detection of multi-point grounding violations by explicitly counting ground symbols connected to a single circuit in the property graph structure.

  2. Accurate verification of wiring integrity, including identifying open circuits and polarity reversals by analyzing spectral features (like eigenvalue multiplicity) of the adjacency matrix derived from the component graph.

  3. Robust handling of spatial perturbations (rotation, translation, noise) on schematics, as the V2G framework is invariant to these changes because it relies on symbolic vector relationships rather than exact pixel matching.

  4. Generation of fully auditable compliance reports in structured JSON format detailing which specific rule (e.g., CT secondary must connect to exactly one ground) was violated, providing deterministic evidence for human auditors or automated regulatory checks.

Sources

Related papers