WebCraftBench: Evaluating Web Application Generation from a Software Testing Perspective

arXiv:2609.15387 · cs.SE, cs.AI · Submitted 2026-09-14 · Read on arXiv

cs.SE, cs.AI

Submitted: 2026-09-14

Updated: 2026-09-20

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Alignment — 27× in this paper · explained in 1 episode
• Benchmark — 27× in this paper · explained in 2 episodes
• Trace — 15× in this paper · explained in 2 episodes
• Abstraction — 13× in this paper · explained in 1 episode
• Core — 6× in this paper · explained in 2 episodes
• Robustness — 5× in this paper · explained in 5 episodes
• Benchmarking — 3× in this paper · explained in 3 episodes
• Aggregation — 2× in this paper · explained in 1 episode
• Consistency — 2× in this paper · explained in 3 episodes
• Reinforcement Learning — 2× in this paper · explained in 4 episodes
• Verification — 2× in this paper · explained in 2 episodes
• Ablation — 1× in this paper · explained in 1 episode
• Adaptation — 1× in this paper · explained in 1 episode
• Agentic AI — 1× in this paper · explained in 1 episode
• Agentic Coding — 1× in this paper · explained in 1 episode
• Alignment Scores — 1× in this paper · explained in 1 episode
• Deep Reinforcement Learning — 1× in this paper · explained in 2 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• LLM judge — 1× in this paper · explained in 2 episodes
• Representation Learning — 1× in this paper · explained in 2 episodes
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• WebSight: A Vision-First Architecture for Robust Web Agents
• I-WebGenBench : Evaluating Interactivity in LLM-Generated Scientific Web Applications
• Deep Reinforcement Learning for Automated Web GUI Testing
• WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models
• Temac: Multi-Agent Collaboration for Automated Web GUI Testing
• Understanding Automated Web GUI Testing: An Empirical Study Across Exploration Strategies and State Abstractions
• WebRISE: Requirement-Induced State Evaluation for MLLM-Generated Web Artifacts
• FullFront: Benchmarking MLLMs Across the Full Front-End Engineering Workflow
• Multimodal graph representation learning for website generation based on visual sketch
• FronTalk: Benchmarking Front-End Development as Conversational Code Generation with Multi-Modal Feedback
• Web-Bench: A LLM Code Benchmark Based on Web Standards and Frameworks
• Cookie-Bench: Continuous On-screen Key Interaction Evaluation for Web Generation
• AWorld: Orchestrating the Training Recipe for Agentic AI
• ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation
• WebGameBench: Requirement-to-Application Evaluation for Coding Agents via Browser-Native Games
• MiniAppBench: Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants
• FrontendBench: A Benchmark for Evaluating LLMs on Front-End Development via Automatic Evaluation

Related papers

• Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
• GitSkills: A Dataset of Agent Skills on GitHub
• SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
• PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
• IntentCoding: Amplifying User Intent in Code Generation
• Incentives and Outcomes in Bug Bounties