When Does Corruption Break the Plan? A Degradation-Tolerance Benchmark for Camera-Only End-to-End Driving

summary

The gist

The gist The DriveDegrade benchmark targets detection or bird’s-eye-view perception, not the planning output that drives the car Introduction and Motivation Camera-only end-to-end (E2E) driving

In short

DriveDegrade is a benchmark testing how much camera degradation affects autonomous driving planning, not just perception. It injects sixteen types of image corruption into camera feeds and evaluates fifteen different driving policies. The findings show that planners are sensitive to certain degradations like blur but robust to others like weather, and they distinguish between tolerating bad pixels versus losing information entirely.

Key concepts

DriveDegrade: A benchmark designed specifically to measure the tolerance of camera-only end-to-end driving policies against image quality degradation. It systematically injects various corruption types into camera feeds to find the breaking point for planning performance.
Corruption Families: Sixteen distinct categories of image corruption, such as blur, noise, JPEG artifacts, and raindrop damage. These are defined by a single physical parameter and severity level (S1 to S5), allowing researchers to test how different types of visual interference affect a driving policy.
Quality vs. Removal: The paper separates corruption into two axes: those that degrade the image quality (like blur) and those that remove information entirely (like occlusion). Planners behave differently on these axes, showing they lose accuracy when information is deleted, regardless of how much quality is lost.
Ego-Status Shortcut: A measure derived from how planners respond to image degradation. It quantifies the difference between a planner that tolerates bad pixels but not missing ones and one that ignores the camera entirely, revealing which specific types of visual faults are most critical for robust planning.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Sample Complexity — explained in 18 episodes
Federated Learning — explained in 17 episodes
Retrieval-Augmented Generation (RAG) — explained in 16 episodes
Interpretability — explained in 15 episodes
Quantum Fisher Information (QFI) — explained in 15 episodes

This episode discusses

The paper

When Does Corruption Break the Plan? A Degradation-Tolerance Benchmark for Camera-Only End-to-End Driving · Read on arXiv

Haohua Que Handong Yao

University of Georgia

Camera-only end-to-end (E2E) driving models are nearing deployment, where the camera stream is degraded by blur, noise, low light, weather, frame loss, and memory faults. How much a policy tolerates before its driving breaks is unclear. Corruption-robustness benchmarks target detection or bird's-eye-view perception, not the planning output that drives the car. We present DriveDegrade, a benchmark for image-degradation tolerance in camera-only E2E driving. Sixteen corruption families at five severities are injected on the fly inside the image loader, one operator reaching fifteen policies, and we evaluate open-loop planning on nuScenes and NAVSIM plus a CARLA closed-loop anchor. First, mild degradation barely affects planning, and the families that break it have a clear threshold at mid severity. Second, fragility is corruption-dependent: blur, JPEG, and raindrop damage planning most, while weather and bit error are tolerated far into the range. Third, a flat curve is ambiguous, so we separate corruptions that degrade the image from those that remove it. A planner that reads its camera must lose accuracy when information is deleted, whatever it does under quality loss. On these two axes the planners separate sharply, quantifying the ego-status shortcut without mistaking indifference for robustness. A released vision-language-action planner is flat on both axes, and blinding all six of its cameras costs it only 11.5 percent.

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "When Does Corruption Break the Plan? A Degradation-Tolerance Benchmark for Camera-Only End-to-End Driving".

Dev: The gist The DriveDegrade benchmark targets detection or bird’s-eye-view perception,

Rosa: First, who's behind it and why it matters.

Title and authors: Rosa: So let's talk about who wrote this and what the title itself says. The paper is "When Does Corruption Break the Plan? A Degradation-Tolerance Benchmark for Camera-Only End-to-End Driving."

Dev: It’s written by Haohua Que Handong Yao from the College of Engineering at the University of Georgia, USA. The title really frames it as a benchmark, meaning they built a standardized test for this kind of image degradation tolerance.

Taro: A benchmark is useful because it lets us compare different systems against the same set of challenges, rather than just seeing how one system does in isolation.

Rosa: Exactly. So instead of just saying "this model works," we can say "this model performs at severity level four under a specific type of noise."

Dev: It moves the conversation away from simple accuracy metrics and toward measuring how much uncertainty or failure a driving policy can absorb before it starts making bad decisions.

Taro: That’s interesting because most existing robustness papers focus on perception, which is the 'seeing' part, but this one focuses on the 'planning' part.

Rosa: Precisely. It explicitly states that corruption-robustness benchmarks target detection or bird's-eye-view perception rather than the planning output that drives the car.

Dev: That distinction is huge for deployment because a system can still perceive things, but if its planning fails under degradation, it’s not safe to use on the road.

The paper's summary: Rosa: So what does the summary of this paper actually boil down to? It says they present DriveDegrade as a benchmark for image-degradation tolerance in camera-only E2E driving models.

Dev: They inject sixteen corruption families at five severities, and they test fifteen policies on nuScenes, NAVSIM, plus CARLA closed-loop. This gives them a very specific experimental setup to test the limits of these systems.

Taro: I like that they are using an open-loop planning evaluation alongside a closed-loop anchor in CARLA. That combination lets you see how the system behaves when it has feedback versus when it's just trying to make decisions on its own.

Rosa: That closed-loop anchor is important because the paper confirms that camera loss remains load-bearing under feedback, which is a big reassurance for safety cases.

Dev: That sounds like they’re saying that even if the visual input gets degraded severely, as long as the feedback loop is working, the planning accuracy stays relatively stable.

Taro: So it's not just about having a good perception system; it's about how resilient that perception system is when it feeds into a decision-making loop.

The paper's improvements: Rosa: The authors suggest several improvements, and I want to unpack those for you. One big one is the vision-reliance diagnostic, which they use to resolve the flatcurve ambiguity.

Dev: They use this diagnostic to separate quality degradation from information removal. This turns the degradation curve into an interpretable measure of camera reliance instead of just a robustness score that might reward a planner for ignoring its input.

Taro: So it gives us a way to distinguish between a planner that's actually struggling with bad pixels versus one that's completely missing data, which is crucial for diagnosis.

Rosa: And they also discuss the ambiguity when quality loss and information removal are separated, showing that a planner must lose accuracy when information is deleted whatever it does under quality loss.

Dev: That implies that if we can separate those two failure modes, we can better understand the planner's internal logic and where its vulnerabilities truly lie.

Taro: I think this helps us design better safety cases because we move from a vague measure of robustness to a specific understanding of what's failing.

Conclusion: Rosa: So to wrap up, this paper presents DriveDegrade as a way to turn camera degradation from just a perception question into measuring planning-level tolerance.

Dev: They’ve located clear breakpoints for blur, JPEG, and noise where planning breaks most fragile, while weather and bit errors are often tolerated far into the range.

Taro: The CARLA anchor confirms that camera loss remains load-bearing under feedback, which is a really strong finding when you think about real-world operational safety.

Rosa: They also released an artifact repository at github.com/UGAMOBILITY-LAB/DriveDegrade, making it accessible for reproducibility on CPU with no raw sensor data.

Dev: The implications are that we can now target mitigation strategies more effectively; for instance, focusing engineering resources on things like raindrop because it was identified as the most damaging family for camera-reliant planners.

Taro: It shows us exactly where to put our efforts when we're trying to make these camera-only systems more reliable in the real world.

Rosa: And that’s how this paper ends, giving us a solid foundation for evaluating camera resilience in driving policies through DriveDegrade.

More episodes

← Home