Deep Feature Pyramid Convolutional Networks with In-Place Activated Batch Normalization for Automated Skin Lesion Boundary Segmentation

arXiv:1812.00877 · cs.CV, cs.LG, stat.ML · Submitted 2018-11-23 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Deep Feature Pyramid Convolutional Networks with In-Place Activated Batch Normalization for Automated Skin Lesion Boundary Segmentation".

Jane: The paper was written by Glib Kechyn from.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Summary: Tom: So, in this segment, we want to take a deeper look at what the paper actually managed to achieve using its core methods. The authors used a U-net architecture as their primary model.

Jane: It’s essentially taking that standard U-net structure and applying it specifically to tackle the difficult task of finding these precise boundaries in skin images.

Tom: They trained this network on the ISIC dataset, which is a massive collection of dermatoscopic images, giving the AI a lot of examples to learn from.

Meng: Training on that volume of data makes sense if you want the model to be robust enough to handle real-world clinical variability in image quality and skin structures.

Lu: I think the fact that they are using a sophisticated architecture like U-net is really paving the way for automated analysis in many other complex segmentation problems, not just dermatology.

Lalam: The summary shows us a clear step toward achieving high precision, meaning we are seeing how AI can move towards providing instant, reliable diagnostic support globally.

Tom: But knowing what it achieved is one thing; how it actually functions is another question entirely.

Improvements: Tom: Now, let's look at the improvements that the authors suggest in Deep Feature Pyramid Convolutional Networks with In-Place Activated Batch Normalization for Automated Skin Lesion Boundary Segmentation. This is where they really differentiate their approach from standard U-nets.

Jane: They aren' are just using a basic U-net; they’re integrating advanced components like Wide ResNet38 and DPN backbones to act as the encoders, which help capture complex patterns better.

Tom: And the concept of In-Place Activated Batch Normalization, or InPlace-ABN, is also used to speed up training and save memory—that's a major engineering win.

Meng: Saving twenty-five percent of memory consumption via that InPlace-ABN technique is a huge practical improvement for me; it means we can run these massive models on less hardware.

Lu: This use of feature pyramid networks alongside the DPN backbone opens up so many possibilities for finding subtle, complex features that were previously overlooked by standard architectures.

Lalam: By optimizing the training process and improving the capture of detail, this work is helping to democratize high-level diagnostic tools for people in resource-limited settings.

Tom: That efficiency combined with better pattern recognition is truly a powerful combination.

Discussion of Enhancements: Tom: We've seen the core methods and the key improvements, but let's talk more about the specific techniques they used to enhance prediction quality in Deep Feature Pyramid Convolutional Networks with In-Place Activated Batch Normalization for Automated Skin Lesion Boundary Segmentation.

Jane: They used various strategies like cycle learning rate scheduling and snapshot ensembling, which helps stabilize the training process over time.

Tom: And instead of just one model, they are combining predictions from multiple models, using an ensemble approach to get better generalization across different architectures.

Meng: The post-processing step is also critical; using marker-based watershed after running the test time augmentations and then averaging them gives us a very robust final result.

Lu: This systematic approach—layer upon layer of refinement and combination—suggest that we can build far more reliable AI systems for complex medical tasks.

Lalam: The ability to improve prediction certainty through these sophisticated methods is essential for ensuring that the culture of medical research moves toward high trust and verifiable accuracy.

Tom: It's all about making sure the predictions are as reliable as possible, which is crucial when dealing with something as serious as melanoma.

Conclusion: Tom: As we wrap up our discussion on Deep Feature Pyramid Convolutional Networks with In-Place Activated Batch Normalization for Automated Skin Lesion Boundary Segmentation, we've seen some incredible progress in automated segmentation.

Jane: We’ve moved from understanding the basic U-net concept to seeing how the specialized backbones and optimization techniques elevate this really quite a significant shift.

Tom: It seems clear that by pushing the boundaries of AI architecture and training methodology, we have found a powerful way forward for reliable lesion boundary detection.

Meng: I am very optimistic about how these optimized models will be deployed in real-world clinical settings to assist doctors globally.

Lu: The path is wide open for future research, suggesting that this is just the beginning of a truly integrated era of AI in medical imaging.

Lalam: We are hopeful that the integration of AI like this will ultimately improve the quality and accessibility of healthcare worldwide.

Tom: So, as we conclude Deep Feature Pyramid Convolutional Networks with In-Place Activated Batch Normalization for Automated Skin Lesion Boundary Segmentation, thank you all for joining us on this fascinating piece of research.

Jane: It's a testament to the ongoing advancements in AI that it is not just a theoretical concept but something that can actually provide practical solutions today.

Tom: We're excited to see what the future holds for these kinds of automated systems!

Glib Kechyn

cs.CV, cs.LG, stat.ML

Submitted: 2018-11-23

Updated: 2026-08-25

Importance score: 90/100

The gist: This paper presents a method for "automatic lesion boundary detection in dermoscopy" using deep neural networks to address the "time-consuming and tedious" nature of manual segmentation performed by

Key concepts

U-net architecture
The authors used the U-net structure as their primary model for segmentation. This standard architecture is applied specifically to find precise boundaries in skin images, allowing the AI to analyze complex patterns and achieve high precision.
InPlace Activated Batch Normalization (InPlace-ABN)
This technique is a major engineering improvement used in the network. It helps speed up the training process and significantly reduces memory consumption by twenty-five percent, enabling efficient deployment of large models on less hardware.
Ensemble approach
To improve prediction quality, the researchers combined predictions from multiple models. This strategy provides better generalization across different architectures, ensuring a more robust and reliable final result for complex medical tasks.

Terminology

Summary

This paper presents a method for automatic lesion boundary detection in dermoscopy using deep neural networks to address the time-consuming and tedious nature of manual segmentation performed by pathologists. By adapting U-net architectures, the research aims to improve diagnostic accuracy for melanoma, the most dangerous type of skin cancer, despite the challenges posed by high variability in medical images.

The Proposed Architecture

The approach utilizes a U-net style network as the primary model for segmenting skin boundaries, employing a method based on ensembling deep convolutional networks with different architectures and snapshots for better generalization. To better capture complex patterns, the model utilizes deep encoders, specifically Wide ResNet38 and DPN backbone networks that are pretrained on ImageNet.

The architecture features a FPN based decoder and several specific structural modifications:

  • An initial block using a 7x7 kernel and stride 2 convolution followed by max-pooling with stride 2.

  • The inclusion of a 1 x 1 convolution operation in each decoder block to reduce filters.

  • The implementation of In-Place Activated BatchNorm (InPlace-ABN), which serves as a memory efficient replacement for BatchNorm + Activation step and saves up to 25 percent of memory consumption.

Data Preprocessing and Augmentation

The research utilizes the ISIC dataset, comprising 2594 training images, 100 validation images, and 1000 test images with resolutions ranging from 576x768 to 6748x4499. Preprocessing involves resizing the image to 224x224 and normalizing it with the mean and standard deviation estimated from the training set. To ensure robust training, the training set is separated into 5 stratified folds to preserve class distribution.

To enhance model performance, approximately 14 random augmentations are applied to each image, including:

  • MotionBlur, MedianBlur, and RandomContrast

  • RandomBrightness and ShiftScaleRotate

  • CLAHE and IAASharpen

  • Distort, HueSaturation, and ToGray

Training and Post-processing

The training process is optimized through a cycle learning rate, snapshot ensembling and hypercolumn. The loss function is defined as "w1BCE + w2(1 - dice) with weights of 0.5 for each component. This combination uses Binary cross-entropy - for more certain body of the object and dice to make the boundary more precise."

In the post-processing stage, a Marker based watershed is applied, where a connected component was used as a marker with some dilation and erosion. Additionally, the final results are derived from test time augmented predictions, where 7 augmentations for each model, were ensembled by taking the mean of the predicted masks.

Results and Conclusions

The experimental results demonstrate the efficacy of the approach, with the best model achieved results with 0.752. Other experiments utilizing different architectures or single models yielded results between 0.700 and 0.750.

The paper concludes that the U-net architecture serves as an effective solver of the current lesion boundary segmentation task. The author suggests that the model could be further optimized by reducing and selecting only optimal augmentations and by using deeper architectures following with the ensembles.

Improvements for AI systems

1. Multi-Scale Patch-Based Inference Engine

  • Improvement: Replace the fixed 224 times 224 resizing strategy with a high-resolution sliding-window patch-based inference approach, utilizing overlapping tiles and Gaussian weighted blending for reconstruction.

  • Capability: This allows the system to process images at their native resolution (up to 6748 times 4499), enabling the detection of microscopic irregularities and fine-grained textural changes at the lesion boundary that are currently lost during aggressive downsampling.

2. Attention-Gated Skip Connections and Boundary-Aware Loss Function

  • Improvement: Integrate Attention Gates (AGs) into the U-Net skip connections and augment the loss function with a Boundary-Weighted Dice Loss and a Distance Transform-based loss (e.g., Hausdorff Distance loss).

  • Capability: The system will suppress irrelevant feature activations in non-lesion regions and prioritize the high-frequency spatial information at the edges, resulting in significantly higher precision for the lesion's perimeter and reduced bleeding of the segmentation mask into healthy tissue.

3. Domain-Specific Self-Supervised Pretraining (SSL)

  • Improvement: Shift from ImageNet-based pretraining to a self-supervised pretext task (such as Masked Autoencoding or Contrastive Learning) performed on a massive, unlabeled corpus of dermatoscopic images.

  • Capability: This aligns the encoder’s feature space with the specific spectral and textural characteristics of skin morphology, allowing the model to learn dermatological nuances (e.g., pigment networks, globules) far more effectively than a model trained on generic natural objects.

4. Bayesian Uncertainty Quantification

  • Improvement: Implement Monte Carlo (MC) Dropout or Deep Ensembles to provide pixel-wise epistemic uncertainty maps alongside the segmentation mask.

  • Capability: The system will not only provide a boundary but also a confidence map, alerting clinicians to specific segments of the lesion border where the model is uncertain due to low contrast or image artifacts, thereby facilitating safer human-in-the-loop diagnostic workflows.

5. Hybrid CNN-Transformer Architecture (TransUNet)

  • Improvement: Replace the standard CNN encoder with a hybrid architecture that utilizes a Vision Transformer (ViT) backbone to capture global context, integrated with the existing CNN-based local feature extraction.

  • Capability: This enables the system to model long-range dependencies and the global geometry of the lesion, preventing segmentation errors in cases of extremely large, irregular, or multi-focal lesions where local texture alone is insufficient for context.

Sources

Related papers