A Comprehensive Assessment Benchmark for Rigorously Evaluating Deep Learning Image Classifiers

The scientific paper introduces a new comprehensive assessment benchmark designed for rigorously evaluating deep learning image classifiers, addressing the critical need for more reliable and robust models in real-world scenarios. Current evaluation protocols are insufficient because they typically rely on limited types of test data, failing to comprehensively assess performance across various generalisation challenges. This work advocates for benchmarking performance using a wide range of different types of data and employing a single metric applicable to all such data types to produce a consistent evaluation.

Need for Robustness Assessment

The success of deep learning models often overlooks their brittleness and insufficient reliability for deployment in domains requiring security or safety concerns. The paper highlights that deep learning is poor at correctly generalising to novel data/situations. To be dependable, methods must be able to make accurate predictions on data not used for training. This necessity leads to defining various forms of robustness, including:

1. Generalisation performance assessed using a test or validation set (in-distribution generalisation).

2. Out-of-distribution (OOD) generalisation, which requires coping with changes in appearance like orientation, scale, viewpoint, lighting, and blur. This can be assessed using clean samples that differ from the training data or IID test images that are synthetically modified.

3. Adversarial robustness assessed by testing with adversarially perturbed samples, which are modifications that cause significant changes in prediction despite being virtually imperceptible to a human observer.

4. Ability to reject samples from unseen categories, termed open-set recognition (OSR) or OOD detection/rejection.

Issues with Current Assessment Methods

Previous research has often considered one form of generalisation in isolation, leading to models that are poor at other types of testing. The paper notes that most work focuses solely on clean test data, resulting in models that are highly susceptible to adversarial attacks and prone to finding short-cuts to improve clean accuracy at the expense of learning more generally useful information. Furthermore, existing metrics often fail because they do not account for the rejection of samples. For unknown class rejection, standard metrics like False Positive Rate (FPR@X%) use separate thresholds for each unknown data-set and do not evaluate the accuracy with which accepted samples are classified. This means a model could score highly by rejecting many samples while assigning random labels to the accepted ones, rendering it virtually useless in any practical application.

Proposed Comprehensive Assessment Benchmark

The proposed benchmark evaluates performance using five distinct types of test data, illustrated in Figure 1:

1. Clean data from the standard test set.

2. Corrupt data from the standard test set that have been manipulated to simulate changes in viewing conditions (e.g., noise, blur).

3. Adversarial samples produced using AutoAttack (AA), constrained by both l∞ and l2-norm attacks with a specified perturbation budget (ϵ).

4. Novel classes, which include objects from classes unseen during training.

5. Unrecognisable images, which are synthetically generated using four methods: Phase (randomizing phase in the Fourier domain), Scramble (random permutation of all pixels), Blobs, and Uniform.

Proposed Metric and Evaluation Strategy

The benchmark utilizes a metric derived from Zhu et al. (2024) that is an extended version of the Detection Error Rate (DER). This metric counts false positives and false negatives as a proportion of all test samples, considering both whether a sample is correctly accepted or rejected, and whether the predicted class label is correct. The proposed benchmark reports this as detection accuracy rate (DAR) rather than DER to correlate higher scores with improved performance. Results are summarized by averaging performance across each data-set type to obtain a single summary metric for overall robustness, balancing the importance of each data type.

Key Findings and Conclusion

The results demonstrate that existing methods of training DNNs, including those claiming state-of-the-art robustness, fail to perform as well as previous, less comprehensive assessments would suggest. The assessment reveals trade-offs: models trained for one type of data may suffer poor performance on others. For instance, networks trained with adversarial training (AT) are most resistant to adversarial attacks but show reduced performance on unknown class rejection. This suggests that current efforts to increase robustness in one area may result in a decrease in robustness elsewhere. The paper concludes that the lack of comprehensive assessment means models are highly susceptible to malicious attacks and, by appropriate choice of data type, can be fooled into producing the wrong predictions with high probability, necessitating a change in evaluation practices to test models against all types of data.

Training Methods Evaluated

The study evaluates various training-time data augmentation techniques, including:

1. Baseline: Simple geometric augmentations like 4-pixel random cropping and random horizontal flipping.

2. Noise Augmentation: Adding Gaussian noise to the training images, with different methods for choosing the standard deviation range [σl, σu].

Based on the comprehensive assessment benchmark proposed in this paper, here are the specific improvements that can be made to current deep learning image classifiers, and what those improved systems will be capable of doing:

The primary improvement is shifting from a narrow focus (e.g., clean accuracy) to a holistic understanding of model reliability across all possible real-world data scenarios by implementing the proposed comprehensive assessment framework.

Here are the specific improvements and resulting capabilities:

1. ​-Implement Robust, Multi-faceted Training Regimes: Instead of relying on single training methods, develop models trained using a combination of augmentation techniques (e.g., baseline geometric augmentations + noise injection + adversarial training (PGD10) + pixmix/regmixup).

2. ​-Adopt Comprehensive Evaluation Protocols: Utilize the proposed benchmark which tests classifiers against five distinct data types:

• Clean data (IID generalization).

• Corrupt data (testing resilience to noise, blur, and geometric transformations).

• Adversarial samples (testing resistance to imperceptible, gradient-based attacks).

• Novel classes (testing ability to reject unseen objects from other domains/classes).

• Unrecognisable images (testing rejection of semantically meaningless or highly distorted inputs like random noise or extreme scrambles).

3. ​-Employ a Unified Performance Metric: Replace disparate metrics with the proposed Detection Accuracy Rate (DAR), which measures the proportion of correctly processed samples across all data types, calculated using Maximum Softmax Probability (MSP) and a fixed rejection threshold (e.g., 95% acceptance rate for clean data).

The resulting improved AI systems will possess the following specific capabilities:

1. ​-Enhanced Real-World Reliability in Diverse Environments: The system will be significantly more reliable when deployed in complex, uncurated environments (e.g., surveillance, autonomous vehicles) because it is explicitly trained to handle a wide variety of input variations—from subtle lighting changes and image compression artifacts (corrupt data) to malicious adversarial perturbations and completely novel objects (novel/unrecognisable classes).

2. ​-Superior Security Against Malicious Attacks: By training on adversarial samples and testing against them, the system will exhibit significantly higher resistance to evasion attacks. It will be much less likely to be fooled by imperceptible noise or targeted manipulation designed to force a misclassification.

3. ​-Effective Out-of-Distribution (OOD) Detection and Rejection: The system will not only attempt to classify objects it knows but will actively reject inputs that fall outside its learned distribution, including entirely new object categories or images so distorted they contain no meaningful semantic information. This prevents the model from making high-confidence, incorrect predictions on unknown entities.

4. ​-Consistent and Honest Performance Reporting: Unlike current models that report high clean accuracy while failing catastrophically on corrupt or novel data, this system will provide a single, consistent performance score (DAR) that reflects its true utility across all critical testing scenarios. This prevents the deployment of systems that give a false sense of security.

5. ​-Improved Decision Boundary Integrity: The evaluation process forces the development of models whose decision boundaries are well-positioned across feature space, ensuring they correctly separate known classes while explicitly learning to ignore or reject samples far from those known regions, leading to more principled and trustworthy classification decisions in practice.

Reliable and robust evaluation methods are a necessary first step towards developing machine learning models that are themselves robust and reliable. Unfortunately, current evaluation protocols typically used to assess classifiers fail to comprehensively evaluate performance as they tend to rely on limited types of test data, and ignore others. For example, using the standard test data fails to evaluate the predictions made by the classifier to samples from classes it was not trained on. On the other hand, testing with data containing samples from unknown classes fails to evaluate how well the classifier can predict the labels for known classes. This article advocates benchmarking performance using a wide range of different types of data and using a single metric that can be applied to all such data types to produce a consistent evaluation of performance. Using the proposed benchmark it is found that current deep neural networks, including those trained with methods that are believed to produce state-of-the-art robustness, are vulnerable to making mistakes on certain types of data. This means that such models will be unreliable in real-world scenarios where they may encounter data from many different domains, and that they are insecure as they can be easily fooled into making the wrong decisions. It is hoped that these results will motivate the wider adoption of more comprehensive testing methods that will, in turn, lead to the development of more robust machine learning methods in the future. Code is available at: https://codeberg.org/mwspratling/RobustnessEvaluation