The GUI Is Not the State: Diagnosing State Aliasing in GUI World Models

arXiv:2609.32679 · cs.LG, cs.AI, cs.CL, cs.CV · Submitted 2026-09-26 · Read on arXiv

cs.LG, cs.AI, cs.CL, cs.CV

Submitted: 2026-09-26

Updated: 2026-09-26

Terminology

• World Model — 56× in this paper · explained in 7 episodes
• Benchmark — 38× in this paper · explained in 2 episodes
• State Aliasing — 20× in this paper · explained in 2 episodes
• Distillation — 16× in this paper · explained in 2 episodes
• Fidelity — 16× in this paper · explained in 2 episodes
• Multi-Teacher Distillation — 13× in this paper
• Provenance — 8× in this paper · explained in 3 episodes
• Robustness — 8× in this paper · explained in 5 episodes
• Distribution Shift — 7× in this paper · explained in 5 episodes
• Benchmarking — 4× in this paper · explained in 3 episodes
• Trace — 4× in this paper · explained in 2 episodes
• Ablation — 3× in this paper · explained in 1 episode
• Agent Memory — 3× in this paper · explained in 1 episode
• Accuracy — 2× in this paper · explained in 1 episode
• Adaptation — 2× in this paper · explained in 1 episode
• Aggregation — 2× in this paper · explained in 1 episode
• Decode — 2× in this paper · explained in 2 episodes
• Generalization — 2× in this paper · explained in 13 episodes
• Generative AI — 2× in this paper · explained in 2 episodes
• Grounding — 2× in this paper · explained in 3 episodes
• Reinforcement Learning — 2× in this paper · explained in 4 episodes
• Semantic Similarity — 2× in this paper · explained in 2 episodes
• Structured Reasoning — 2× in this paper · explained in 3 episodes
• Action Primitives — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Hallucination — 1× in this paper · explained in 3 episodes
• Reasoning Traces — 1× in this paper · explained in 3 episodes
• Tokenization — 1× in this paper · explained in 2 episodes

Sources

• MobileDreamer: Generative Sketch World Model for GUI Agent
• Chain-of-Memory: Enhancing GUI Agents for Cross-Application Navigation
• HalluClear: Diagnosing, Evaluating and Mitigating Hallucinations in GUI Agents
• Kimi K3: Open Frontier Intelligence
• MobileWorldBench: Towards Semantic World Modeling For Mobile Agents
• PAL-UI: Planning with Active Look-back for Vision-Based GUI Agents
• UI-TARS: Pioneering Automated GUI Interaction with Native Agents
• Qwen3-VL Technical Report
• Qwen3.8-Omni: Towards Native Omni-Modal Agents
• UISim: An Interactive Image-Based UI Simulator for Dynamic Mobile Environments
• AppDeltaWorld: Transition-Grounded Delta Code World Model for Mobile GUI Agents
• How Mobile World Model Guides GUI Agents?
• Mobile-Agent-v3: Fundamental Agents for GUI Automation
• OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks
• Code2World: A GUI World Model via Renderable Code Generation

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks