Exploring convolutional neural networks with transfer learning for diagnosing Lyme disease from skin lesion images

The gist Lyme disease which manifests itself in most cases with erythema migrans (EM) skin lesions can be diagnosed using convolutional neural networks (CNNs) for early diagnosis and referral to expert dermatologists

Dataset Creation and Preparation

The study created an EM dataset by collecting images from the internet and Clermont-Ferrand University Hospital Center (CF-CHU) of France, resulting in 866 images assigned to the EM class and 806 assigned to the Confuser class The dataset was subdivided into five-folds using stratified five-fold cross-validation to ensure each fold maintained the original class ratio Data augmentation techniques were applied only to the training set to expand it twenty times, using augmentations such as flip (vertical or horizontal), rotation, brightness, contrast, and saturation

CNN Architectures Benchmarked

Twenty-three CNN architectures were benchmarked against the dataset in terms of predictive performance, computational complexity, and statistical significance The architectures tested included VGG, ResNet, DenseNet, MobileNet, Xception, NASNet, and EfficientNet Specific architectures analyzed were VGG architecture [27], Inception architecture [28], ResNet architecture [31], Dense Convolutional Network (DenseNet) [33], MobileNet Architecture [34] including MobileNetV2 and MobileNetV3, Xception architecture [40], NASNet architecture [41] including NASNetMobile, and EfficientNet architecture

Transfer Learning Strategy

To improve performance on the limited dataset, custom transfer learning from ImageNet pre-trained models was used as well as pretraining the CNNs with the skin lesion dataset HAM10000 The best performing configuration across all architectures involved pretraining only the unfrozen part of an ImageNet pre-trained model with HAM10000 data before fine-tuning U layers with our Lyme disease dataset This approach was applied to ResNet50, resulting in the best accuracy of 84.42%

Performance and Model Selection

The customized ResNet50 architecture achieved the best classification accuracy of 84.42% ±1.36, with an AUC of 0.9189 ±0.0115 A lightweight model customized from EfficientNetB0 also performed well with an accuracy of 83.13% ±1.2 and an AUC of 0.9094 ±0.0129 The study utilized various predictive performance measures, including accuracy, recall/sensitivity/hit rate/TPR, specificity/selectivity/TNR, precision/PPV, NPV, MCC, Cohen’s kappa coefficient (κ), and Area under the receiver operating characteristic (AUC) metrics The best performing model overall was ResNet50-IMGHAMPP-FT141 with an accuracy of 84.42%

Model Complexity and Explainability

Model complexity was measured by total number of model parameters, FLOPs, average training time per epoch, disk and GPU memory usage, and average inference time per image The study provided guidelines for model selection based on predictive performance and computational complexity Explainability was achieved using Gradient-weighted Class Activation Mapping (Grad-CAM) to visualize the regions of the input image that are significant for predictions The discussion indicated that EfficientNetB0-187 is a good choice with reasonable accuracy for resource-constrained mobile platforms

Conclusion and Guidelines

The study confirmed the effectiveness of even some lightweight CNNs for building Lyme disease pre-scanner mobile applications to assist people with an initial self-assessment and referring them to expert dermatologist for further diagnosis The authors made all the trained models publicly available which can be used by others for transfer learning and building pre-scanners for Lyme disease Resource intensive models like ResNet50 can be effective for building computer applications to assist non-expert practitioners with identifying EM The best performing trained models of each architecture are presented in ModelName-U format, where U represents the no of unfrozen layers during transfer learning fine-tuning

How it works

The study utilized transfer learning as the Lyme dataset is not huge enough to obtain good performance by training large CNNs from scratch They started with a CNN already pretrained on ImageNet dataset and after removing the original ImageNet classification head our EM classification head consisting of Global Average Pooling (GAP) layer, dropout layer, and a fully connected layer with sigmoid activation for binary classification was added as shown in Figure 14 Fine-tuning the whole CNN architecture after training the classifier head with our Lyme dataset performed poorly compared to the partial fine-tuning of several layers at the end of the CNN while keeping rest of the layers frozen

Model Selection Guidelines

The study provided guidelines for model selection based on predictive performance and computational complexity The experimental result described above makes it evident that CNNs have great potential to be used for Lyme disease pre-scanner application For resource-constrained mobile platforms, EfficientNetB0-187 is a good choice with reasonable accuracy If resource constraint is not a problem, then RestNet50-141 can be used for the best accuracy Resource intensive models like ResNet50 can be effective for building computer applications to assist non-expert practitioners with identifying EM

Limitations

A limitation is that dark-skinned samples are underrepresented in our dataset A limited number of samples in the dataset with skin hair artifact over the EM lesion is also a concern Although hair removal algorithms can be used for removing skin hair it will increase the computational complexity of the real-time mobile application

Future Directions

Existing works including this study on AI-based Lyme disease analysis only utilize images but including patients’ metadata can be a great way of strengthening the analysis Another limitation is that dark-skinned samples are underrepresented in our dataset We trained the CNNs with whole images without EM lesion segmentation The effect of the EM lesion segmentation on the predictive performance of CNNs can be an interesting study A better alternate can be augmenting the training dataset with artificial skin hair

Conclusion

In this study, we benchmarked and extensively analyzed twenty-three well-known CNNs based on predictive performance, complexity, significance tests, and explainability using a novel Lyme disease dataset to find out the effectiveness of CNNs for Lyme disease diagnosis from EM images The study found that even the lightweight models like EfficientNetB0 performed well suggesting the application of CNNs for Lyme disease pre-scanner mobile applications which can help people with an initial assessment of the probability of Lyme disease and referring them to expert dermatologist for further diagnosis Resource intensive models like ResNet50 can be effective for building computer applications to assist non-expert practitioners with identifying EM We also made all the trained models publicly available, which can be utilized by others for transfer learning and building prescanners for Lyme disease The study confirms the effectiveness of even some lightweight CNNs for building Lyme disease pre-scanner mobile applications to assist people with an initial self-assessment and referring them to expert dermatologist for further diagnosis

Acknowledgments

This research was funded by the European Regional Development Fund, project D

1. textbfImprove Mobile Deployment Capability with EfficientNetB0-187 for Remote Diagnosis: The improved system can be directly deployed in mobile devices without requiring an internet connection for processing the lesion image in a remote server. This enables initial self-assessment of Lyme disease probability even in areas without good internet facilities, addressing the constraint mentioned in the discussion about resource-constrained platforms.

2. textbfOptimize Model Selection via Complexity and Accuracy Trade-off: The system can utilize Figure 18 shows a bubble chart reporting model accuracy vs FLOPs to guide selection, favoring models like EfficientNetB0-187 is a good choice with reasonable accuracy for resource-constrained mobile platforms. This ensures the deployed application balances diagnostic performance with limited computational resources.

3. textbfEnhance Diagnostic Reliability through Transfer Learning Strategy: The system should employ the configuration that pretraining only the unfrozen 141 layers of an ImageNet pre-trained model with HAM10000 data before fine-tuning as this performed best in terms of most of the metrics (7 out of 11) for ResNet50, leading to the highest reported accuracy of 84.42%.

4. textbfIncrease Model Interpretability for Clinical Trust: Implement Grad-CAM visualization to provide clinicians with visual evidence by showing the regions of input that are significant to the CNNs for making predictions, which is crucial as AI tools are used to assist non-expert practitioners in identifying EM.

5. textbf Strengthen Data Robustness through Synthetic Augmentation: To address limitations like underrepresented samples, the system should utilize augmentation techniques such as augmenting the training dataset with artificial skin hair instead of relying solely on existing augmentations, which helps mitigate issues where dark-skinned samples are underrepresented in our dataset.