CAPTCHAs in the Agentic Era: Solvers That Learn from Every Encounter

arXiv:2609.02393 · cs.CR · Submitted 2026-09-02 · Read on arXiv

cs.CR

Submitted: 2026-09-02

Updated: 2026-09-02

Code: https://github.com/microsoft/OmniParser

Project page: https://www.kaggle.com/datasets/pooriamst/website-screenshots

Terminology

• Accuracy — 55× in this paper · explained in 1 episode
• Robustness — 21× in this paper · explained in 5 episodes
• Verification — 14× in this paper · explained in 2 episodes
• Distillation — 11× in this paper · explained in 2 episodes
• Fine-Tuning — 7× in this paper · explained in 5 episodes
• Grounding — 7× in this paper · explained in 3 episodes
• Adversarial Robustness — 5× in this paper · explained in 3 episodes
• Vision-language models — 5× in this paper · explained in 2 episodes
• Alignment — 4× in this paper · explained in 1 episode
• Adversarial Attacks — 3× in this paper · explained in 1 episode
• Adversarial Examples — 3× in this paper · explained in 2 episodes
• Adversarial Training — 3× in this paper · explained in 1 episode
• Perplexity — 3× in this paper · explained in 3 episodes
• Vision-Language Models (VLMs) — 3× in this paper · explained in 6 episodes
• AdamW Optimizer — 2× in this paper · explained in 1 episode
• Benchmark — 2× in this paper · explained in 2 episodes
• Catastrophic Forgetting — 2× in this paper · explained in 5 episodes
• Computer Vision and Pattern Recognition — 2× in this paper
• Core — 2× in this paper · explained in 2 episodes
• Data Augmentation — 2× in this paper · explained in 7 episodes
• Document Object Model — 2× in this paper
• F1 Score — 2× in this paper · explained in 3 episodes
• Instruction Tuning — 2× in this paper · explained in 2 episodes
• Sample Efficiency — 2× in this paper · explained in 4 episodes
• Transferability — 2× in this paper · explained in 4 episodes
• Transformer — 2× in this paper · explained in 2 episodes
• Visual Grounding — 2× in this paper · explained in 2 episodes
• Ablation — 1× in this paper · explained in 1 episode
• Agentic Framework — 1× in this paper · explained in 6 episodes
• Aggregation — 1× in this paper · explained in 1 episode
• Attention Mechanism — 1× in this paper · explained in 3 episodes
• Benchmarking — 1× in this paper · explained in 4 episodes
• Chain-of-Thought Prompting — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Generative Model — 1× in this paper · explained in 2 episodes
• Generative Models — 1× in this paper · explained in 3 episodes
• Hallucination — 1× in this paper · explained in 4 episodes
• Hybrid Approach — 1× in this paper · explained in 4 episodes
• Instance Segmentation — 1× in this paper · explained in 3 episodes
• Tokenization — 1× in this paper · explained in 2 episodes
• Transformer Architecture — 1× in this paper · explained in 2 episodes
• You Only Look Once — 1× in this paper
• Zero-Shot Generalization — 1× in this paper · explained in 5 episodes

Sources

• Open CaptchaWorld: A Comprehensive Web-based Platform for Testing and Benchmarking Multimodal LLM Agents
• Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V
• OmniParser for Pure Vision Based GUI Agent
• Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
• Surfer-H Meets Holo1: Cost-Efficient Web Agent Powered by Open Weights
• Distilling the Knowledge in a Neural Network
• On Tiny Episodic Memories in Continual Learning
• Intriguing properties of neural networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Robustness May Be at Odds with Accuracy

Related papers

• SoK: AI-Augmented Binary Reversing
• Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
• Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
• Efficient Fuzzy PSI under One-Sided Assumptions
• Sealing the Audit-Runtime Gap for LLM Skills
• Token Composition: A Graph Based on EVM Logs