EvoDesign: Agentic Editable Diagram Creation via Design Expertise Evolution

arXiv:2604.09568 · cs.HC, cs.CL, cs.CV · Submitted 2026-02-20 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "EvoDesign: Agentic Editable Diagram Creation via Design Expertise Evolution".

Tom: The gist: EvoDesign introduces an agentic framework that generates object-level editable diagrams via an intermediate canvas schema, bridging the representation gap between automated generation and intuitive human editing.

Jane: First, who's behind it and why it matters.

Paper summary: Tom: So we're looking at the paper EvoDesign: Agentic Editable Diagram Creation via Design Expertise Evolution and the authors are really tackling that gap between automated generation and actual human editing. They introduce this agentic framework that generates diagrams through an intermediate canvas schema, which is basically a way to separate what the diagram means from how it actually looks.

Jane: And what’s interesting is their design knowledge evolution mechanism. They aren't just building the agents once; they have a system that distills execution traces into a hierarchy of domain guidelines and universal principles, which lets the agents learn and get more context-aware over time.

Lu: The authors identify this representation gap as the lack of formats that handle both autonomous machine generation and intuitive human editing simultaneously, pointing out that in real workflows, people constantly tweak things for clarity when they’re aligning their intent.

Meng: So what's the main push here? They are proposing this canvas schema because it lets you unify the precision you get from code with the flexibility you need in a user interface, allowing direct manipulation through that canvas environment while keeping the structure intact.

Lalam: And they set up an iterative refinement loop where the system actually perceives what’s rendered on that canvas to resolve any conflicts it finds right then and there during creation. It's this closed-loop operation where the system corrects itself as it goes.

Tom: It sounds like they’re using this combination of separating intent from rendering logic, plus that evolving knowledge base, to make creating high-fidelity diagrams something that anyone can actually do.

Jane: They also brought in CanvasBench to test this framework, using a VLM-as-a-judge to score the resulting diagrams on content integrity, visual presentation, and cognitive utility.

Lu: The results showed EvoDesign did well across all three areas they tested—content integrity, visual presentation, and cognitive utility—which they found to be a balanced performance profile on that specific benchmark dataset.

Meng: So the numbers suggest that this system isn't just good at making pretty pictures; it’s good at creating visuals that are actually meaningful and usable for people who need them.

Lalam: And the authors also mention how this framework helps human and AI work together better by supporting those real-world collaborative workflows through this interactive canvas idea.

Tom: So to wrap up EvoDesign, the paper presents a framework that builds editable diagrams using an agentic system grounded in a canvas schema and driven by evolving design expertise. It’s about taking complex information and making it visually accessible and interactive.

Jane: The authors are pushing this idea toward what they call Democratic Design because they want to lower the barrier for non-experts to communicate complicated things through visuals that stay structurally sound.

Lu: A key implication is that we're moving toward a logic of design where those specific professional heuristics get distilled into reusable principles, meaning we can build tools that adapt better when you switch between different types of design tasks.

Meng: From an engineering standpoint, this suggests a direction for AI development where systems don't just spit out static images but instead systems that understand how to maintain and iterate on the structure of the diagram itself.

Lalam: And this work lays groundwork for future research by providing a rigorous way to study how we can build interactive tools that support real-world human and AI co-creation in actual workflows.

Conclusion: Tom: So, to wrap up this paper by EvoDesign, they're proposing a framework that builds editable diagrams using an agentic system grounded in a canvas schema and driven by evolving design expertise. It’s about taking complex information and making it visually accessible and interactive.

Jane: And the authors are pushing for this to be Democratic Design because they want to lower the barrier for non-experts to communicate complex things through high-fidelity visuals that stay structurally sound.

Lu: The main implication is that we're moving toward a logic of design where heuristics are distilled into reusable principles, which means we can build tools that adapt better to different domains.

Meng: From an engineering viewpoint, this suggests a direction for AI development where systems don't just output static images but systems that understand how to maintain and iterate on the structure of the diagram itself.

Lalam: And this work lays groundwork for future research by providing a rigorous way to study how we can build interactive tools that support genuine human-AI co-creation in real-world workflows.

Tom: They title it EvoDesign: Agentic Editable Diagram Creation via Design Expertise Evolution, and what this really means is they're moving beyond just generating pictures. They’re talking about a way to build diagrams where the structure itself is editable by humans, not just a static image for viewing.

Jane: Exactly. Think about how much time people spend fixing bad diagrams after an AI spits one out. EvoDesign aims to get rid of that friction by letting the human actually tweak the logic inside that visual representation directly on a canvas.

Lu: The name itself, "EvoDesign," hints at the knowledge evolution part. It suggests the system isn't just following fixed rules; it’s learning from its own mistakes and successes in design to get better at understanding what professional-grade diagrams actually look like and feel like.

Meng: So if we think about practical impact, this means that instead of a designer spending hours manually cleaning up an AI-generated flow chart, they can interact with the AI's output in a way that feels more like collaborating on a shared document.

Lalam: That shifts the focus from just generating an artifact to enabling real-world collaboration. It’s about making these powerful visualization tools usable for everyone, not just people who know how to code or design complex systems.

Tianfu Wang, Leilei Ding, Ziyang Tao, Yi Zhan, Zhiyuan Ma, Wei Wu, Yuxuan Lei, Yuan Feng, Junyang Wang

Hong Kong University of Science and Technology (Guangzhou) · University of Science and Technology of China

cs.HC, cs.CL, cs.CV

Submitted: 2026-02-20

Updated: 2026-10-05

Code: https://github.com/AuraX-AI/EvoDiagram

Importance score: 92/100

The gist: The gist: EvoDesign introduces an agentic framework that generates object-level editable diagrams via an intermediate canvas schema, bridging the representation gap between automated generation and

Key concepts

Canvas Schema D=(G, S, L)
This is the core representation of a diagram. It unifies code precision with UI flexibility by defining three parts: G (semantic graph for meaning), S (style schema for aesthetics), and L (layout configuration for placement). This structure allows the system to generate diagrams that are both logically sound and visually flexible.
Design Policy π=(A, K)
This policy dictates how the system operates. 'A' represents the coordinated team of specialized agents working together, while 'K' consists of context-aware heuristics retrieved from a distilled knowledge memory. This combination ensures that autonomous actions are always grounded in expert design knowledge.
Design Knowledge Evolution M
This mechanism distills professional design rules into reusable insights. It organizes memory into three tiers: raw triplets (Ks), contextual rules (Kg), and universal axioms (Kp). By summarizing traces and aggregating guidelines, the system learns general principles like 'Functional Modularity' for better design decisions.
Agentic Components
The system uses specialized agents to handle different parts of the diagram creation pipeline. The Semantic Structure Agent builds the logical blueprint, the Visual Style Agent defines aesthetics, and the Spatial Layout Agent maps elements onto a 2D coordinate system. These distinct roles allow for complex, coordinated generation.

Terminology

Summary

The gist: EvoDesign introduces an agentic framework that generates object-level editable diagrams via an intermediate canvas schema, bridging the representation gap between automated generation and intuitive human editing.

How it works

EvoDesign employs a coordinated multi-agent system to decouple semantic intent from rendering logic, which resolves conflicts across heterogeneous design layers. The framework is grounded in a canvas schema that unifies the precision of code with the intuitive flexibility of a UI. This schema defines an object-level editable representation D = (G, S, L) comprising a semantic graph G, a style schema S, and a layout configuration L.

The system's operation is governed by a design policy π = A and K where A represents the coordinated team of specialized agents and K denotes context-aware heuristics retrieved from the distilled knowledge memory M. This policy unifies autonomous execution with design expertise. The system operates in a closed-loop, performing iterative refinement by perceiving the rendered canvas to resolve encountered conflicts.

Agentic Components

The agentic system A decomposes the pipeline into specialized agents with distinct focuses. The primary components include:

  1. SEMANTIC STRUCTURE AGENT: This agent distills source content C into a structured semantic graph G = Astr(Ksty, I) to generate a logical structure. It performs entity-first, relation-last construction to transform the abstract blueprint into a precise symbolic instantiation G ready for styling.

  2. VISUAL STYLE AGENT: This agent defines aesthetic properties S = Asty(Ksty, I, G) to ensure a cohesive and professional visual narrative. It translates the abstract specification S˜ into a deterministic set of key-value pairs S compatible with canvas.

  3. SPATIAL LAYOUT AGENT: This agent uses knowledge Klay to produce geometric instantiation L = Alay(Klay, I, G, S) by mapping styled semantic elements onto a two-dimensional coordinate system. It employs a tool-augmented layout process to prioritize physical feasibility through a toolaugmented layout process.

Design Knowledge Evolution

To navigate latent heuristics of professional design, EvoDesign proposes a design knowledge evolution mechanism that distills domain-specific design priors from self-collected experience. This mechanism organizes the design memory M = M = 3 tiers: Ks (raw triplets), Kg (contextual rules), and Kp (universal axioms).

The distillation process involves three steps:

  1. Trajactory-to-Strategy Summarization: Translating raw execution traces into high-utility sample strategies Ks by filtering logs to preserve core reasoning paths and feedback loops.

  2. Instance-to-Guideline Aggregation: Identifying commonalities across strategies within specific domain tags to distill domain guidelines Kg into contextual rules that remain constant within a field.

  3. Guideline-to-Principle Abstraction: Performing cross-domain synthesis to identify shared structural patterns across the library of guidelines into a general principle Kp of ”Functional Modularity,” facilitating a transition to a universal ”logic of design”.

Evaluation and Benchmarking

EvoDiagram is evaluated on CanvasBench, which is a dataset filtered for canvas-recoverability. The evaluation protocol uses a VLM-as-a-judge to assign scores across three primary axes: Content Integrity (CCF), Visual Presentation (VVA), and Cognitive Utility (GCE) The results show EvoDiagram exhibits a balanced and superior performance profile across all evaluation axes in CanvasBench.

Conclusion

The framework successfully bridges the representation gap by generating artifacts that are not only semantically faithful and aesthetically professional but also inherently interactive. EvoDesign empowers non-experts to communicate complex information through high-fidelity visuals while ensuring the final output is aligned with human intent>. This framework promotes Democratic Design by lowering the barrier to creating high-fidelity, structurally sound visuals. It enhances Human-AI Co-creation through an interactive canvas paradigm that supports real-world collaborative workflows. Finally, it facilitates Expertise Accessibility via a Design Knowledge Evolution mechanism that distills professional heuristics into reusable insights. This evolution ensures artifacts are semantically faithful and stylistically professional. The system is designed to support rigorous research in this new paradigm. This paper introduces EvoDesign, an agentic framework that bridges the representation gap in automated diagramming by generating object-level editable diagrams via a canvas-based schema.

Improvements for AI systems

  1. Bold header: Multi-agent Coordinated System Refinement

This improvement involves operationalizing a coordinated multi-agent system where specialized agents for semantic parsing, visual style, and spatial layout decouple semantic intent from rendering logic. This resolves conflicts across heterogeneous design layers by ensuring each agent follows a unified spec-to-instance paradigm (Morgan, 1988) grounded in a shared symbolic schema D.

  1. Bold header: Design Knowledge Evolution Mechanism

This mechanism allows agents to adaptively retrieve context-aware expertise by distilling execution traces into a hierarchical memory of domain guidelines, enabling agents to retrieve context-aware heuristics during synthesis. This prevents cascading errors by grounding coordination in evolvable design knowledge which includes raw experiences (Ks), domain rules (Kg), and universal axioms (Kp).

  1. Bold header: Closed-Loop Refinement Agent

The system gains a crucial feedback loop via the Refiner agent which performs a coordinated perception-action cycle. This agent generates a natural language critique (Visual Diagnosis) and invokes precision tools (Tool-Augmented Correction) to rectify identified issues without necessitating a full regeneration of the global schema, ensuring cross-layer structural consistency.

  1. Bold header: CanvasBench Benchmark for Object-Level Editing

The introduction of CanvasBench provides a rigorous evaluation suite consisting of both data and metrics for canvas-based diagramming, focusing on object-level editability and cognitive utility. This allows for assessment beyond mere semantic correctness to measure dimensions like Content Fidelity (CCF), Visual Aesthetics (VVA), and Cognitive Ease (GCE).

  1. Bold header: Human-Centered Web Application Interface

The framework supports a fluid transition between AI generation and human intervention via a web application that treats diagrams as object-level editable entities. This interface allows users to perform direct UI operations, such as click to refine the visual hierarchy, ensuring the final output is inherently interactive and UI-friendly.

Sources

Related papers