Automated Lesion Segmentation of Stroke MRI Using nnU-Net: A Comprehensive External Validation Across Acute and Chronic Lesions

Accurate and generalisable segmentation of stroke lesions from magnetic resonance imaging (MRI) is essential for advancing clinical research, prognostic modelling, and personalised interventions. The gist: Models trained on DWI consistently outperformed those trained on FLAIR in acute stroke lesion segmentation, while lesion volume emerged as a key determinant of accuracy across chronic stroke studies.

Methodology and Framework

The study systematically evaluated the nnU-Net framework across five independent stroke MRI datasets spanning acute and chronic stages. The authors utilized nnU-Net, a fully automated, self-configuring framework that adapts network architecture and training parameters to the dataset at hand, employing its standard workflow from automated preprocessing (nnUNetv2 plan and preprocess) through training and inference. For acute stroke datasets (SOOP and ISLES), models were trained using paired DWI and ADC maps as a two-channel input, while for chronic stroke datasets (ATLAS v2.0, ARC, CCNRP), models were trained using single-channel T1w images. Initial model training involved five-fold cross-validation with a loss function combining the Dice coefficient and cross-entropy loss, followed by identifying the optimal model using nnUNetv2 find best configuration based on mean cross-validation Dice scores.

Evaluation Metrics and Performance

Performance was rigorously evaluated using standard medical image segmentation metrics, specifically the Dice coefficient to quantify spatial overlap, and the 95th percentile Hausdorff Distance (HD95) to measure boundary distance. The results demonstrated that models achieved robust generalisation, with segmentation accuracy approaching reported inter-rater reliability for manual annotations, ranging from Dice ≈.72–.84. For acute stroke, the SOOP-trained DWI+FLAIR model achieved the best segmentation accuracy (median Dice =.777), showing a statistically significant improvement over the SOOP-trained DWI-only model. For chronic stroke, the ATLASv2 model achieved a median Dice of.81 on external test sets.

Key Determinants of Generalisability

The research identified several critical factors governing model generalisability across different conditions. The performance varied systematically based on:

1. Training Data Composition: Increasing training set size led to systematic performance gains for chronic stroke, with the largest improvements observed when moving from small to medium datasets, indicating diminishing returns beyond several hundred training cases.

2. Lesion Characteristics: Lesion volume emerged as a key determinant of accuracy, with larger lesions were segmented more accurately, and models trained on broad volume distributions showed more symmetric cross-dataset performance. Conversely, very small lesions remained challenging to segment, and models trained on restricted volume ranges generalized poorly.

3. Image Quality: Differences in MRI quality contributed to variability; ARC exhibited systematically lower image quality across multiple dimensions (SNR, tissue boundary definition), yet models trained on higher-quality datasets showed robustness at inference.

Modality and Laterality Effects

The study provided specific insights into modality and anatomical factors. For acute stroke, models trained on DWI consistently outperformed those trained on FLAIR, suggesting that DWI provides more reliable features for the acute phase due to the temporal evolution of lesions. For chronic stroke, lesion laterality did not impact segmentation accuracy or generalisability; models trained on left, right, or bilateral lesions achieved comparable performance. Furthermore, volume-matched analysis showed that apparent generalisation asymmetries largely disappeared once lesion distributions were aligned by matching training and test set volumes.

Conclusion and Implications

The findings demonstrate that automated lesion segmentation can achieve performance comparable to human experts when evaluated on independent datasets. The study provides practical guidance for dataset curation and model development, highlighting key factors such as lesion volume, training data quality and dataset diversity as key determinants of accuracy. Future work should focus on capturing temporal dynamics and extending automated segmentation to modalities like CT. All trained models and code are openly shared to promote transparency and community benchmarking. The conclusion is that these findings offer practical guidance for dataset curation and model development toward more robust, generalisable, and clinically relevant lesion segmentation tools.

The gist

Models trained on DWI consistently outperformed those trained on FLAIR in acute stroke lesion segmentation, while lesion volume emerged as a key determinant of accuracy across chronic stroke studies.

How it works

1. The study utilized the nnU-Net framework, a fully automated, self-configuring framework that adapts network architecture and training parameters to the dataset at hand, following its standard workflow from automated preprocessing (nnUNetv2 plan and preprocess) through training and inference.

2. For acute stroke datasets (SOOP and ISLES), models were trained using paired DWI and ADC maps as a two-channel input, while for chronic stroke datasets (ATLAS v2.0, ARC, CCNRP), models were trained using single-channel T1w images.

Here are the specific improvements that can be made to AI systems, derived from the findings of this study, and what those improved systems could achieve:

1. Improve segmentation performance across stroke stages by prioritizing Diffusion Weighted Imaging (DWI) inputs for acute lesions.

2. Develop modality-invariant or multimodal models that effectively integrate DWI and FLAIR information to overcome the limitations of using only one sequence, especially in the subacute phase where lesion appearance evolves differently between modalities.

3. Implement dynamic training strategies for chronic stroke segmentation by incorporating lesion volume distribution as a key feature, specifically training models on broad volume distributions to improve cross-dataset generalizability.

4. Design adaptive training pipelines where model configuration and parameters are automatically optimized based on the specific input dataset's characteristics (e.g., using the nnU-Net framework effectively) rather than relying on fixed settings.

5. Improve robustness to image quality variations by incorporating image quality metrics (like those derived from MRIQC, particularly SNR, CNR, and residual partial volume effect) directly into the model training or inference process to mitigate performance degradation on lower-quality scans.

6. Create automated pre-processing steps that normalize intensity based on dataset-specific characteristics (Zscore normalization) to ensure better feature learning across heterogeneous MRI datasets.

7. Enhance the reliability of models by incorporating uncertainty estimation or leveraging consensus mechanisms in the ground truth annotation process to account for inherent variability in human labeling, particularly for challenging lesion boundaries and small lesions (< 10 cm3).

These improved AI systems can achieve the following:

1. Accurately segment acute stroke lesions with performance approaching human inter-rater reliability (Dice ≈.78), specifically by leveraging DWI data, leading to more reliable prognostic biomarkers for acute stroke severity.

2. Provide a single, robust segmentation tool capable of performing reliably across both acute and chronic stroke stages using the nnU-Net framework, offering consistent performance on independent datasets from diverse clinical sites.

3. Enable personalized stroke intervention by accurately quantifying lesion volume and shape in chronic lesions, allowing clinicians to stratify patients based on tissue damage severity without needing manual tracing expertise.

4. Develop smart models that can perform well even when deployed in real-world clinical settings where image quality is variable (e.g., lower SNR scans), ensuring a consistent level of diagnostic accuracy regardless of scanner quality.

5. Improve the efficiency of research by enabling researchers to quickly train and test models on new stroke datasets, accelerating the development cycle for novel segmentation algorithms.

6. Create more transparent and reproducible AI tools by providing open-source models and detailed documentation that allow other researchers to understand exactly how performance was achieved across different training set sizes, volumes, and modalities.