deSEO: Physics-Aware Dataset Creation for High-Resolution Satellite Image Shadow Removal

summary

Video file (mp4)

The gist

The gist: deSEO introduces a geometry-aware and physics-informed methodology that derives paired supervision for satellite shadow removal from the S-EO shadow detection dataset through a fully

In short

deSEO introduces a novel, geometry-aware pipeline to create paired supervision for satellite shadow removal from existing datasets. It systematically pairs shadowed and shadow-free images using temporal and geometric filtering, ensuring training data respects real-world acquisition variability. The resulting model outperforms direct adaptations of other architectures by consistently reducing visual impact across diverse viewing conditions.

Key concepts

Paired Supervision
This refers to creating training examples where the input image (shadowed) is explicitly matched with a corresponding output (shadow-free). deSEO generates these pairs from a single detection dataset by intelligently selecting temporally and geometrically related images, allowing deep learning models to learn shadow removal more effectively.
Geometry Extraction
This step calculates crucial viewing parameters like off-nadir angle and sun elevation for every image. This geometric information is essential because it helps the system understand how shadows are cast based on the satellite's exact position relative to the terrain and sun, enabling better alignment.
LoFTR-RANSAC Registration
This is a technique used for precise image matching between different views or dates. LoFTR (local feature transformer) finds similar features across images, and RANSAC (Random Sample Consensus) refines these matches by filtering out outliers. This ensures that when pairing images, the geometric correspondence between them is highly accurate.
Validity Masking
A binary mask is created for each image to mark regions where the alignment between paired samples is reliable. The neural network training only focuses on these reliably aligned areas, which helps the model learn to remove shadows even when there are slight geometric differences, such as off-nadir viewing angles.

Terminology used across episodes

This episode discusses

The paper

deSEO: Physics-Aware Dataset Creation for High-Resolution Satellite Image Shadow Removal · Read on arXiv

Lorenzo Beltrame, Jules Salzinger, Filip Svoboda, Phillipp Fanta-Jende, Jasmin Lampert, Radu Timofte, Marco Korner

Austrian Institute of Technology · University of Cambridge · University of Wurzburg

Shadows cast by terrain and tall structures remain a major obstacle for high-resolution satellite image analysis, degrading classification, detection, and 3D reconstruction performance. Public resources offering geometry-consistent paired shadow/shadow-free satellite imagery are essentially missing, and most Earth-observation datasets are designed for shadow detection or 3D modelling rather than removal. Existing deep shadow-removal datasets either target ground-level or aerial scenes or rely on unpaired and weakly supervised formulations rather than explicit satellite pairs. We address this gap with deSEO, a geometry-aware and physics-informed methodology that, to the best of our knowledge, is the first to derive paired supervision for satellite shadow removal from the S-EO shadow detection dataset through a fully replicable pipeline. For each tile, deSEO selects a minimally shadowed acquisition as a weak reference and pairs it with shadowed counterparts using temporal and geometric filtering, Jacobian-based orientation normalisation, and LoFTR-RANSAC registration. A per-pixel validity mask restricts learning to reliably aligned regions, enabling supervision despite residual off-nadir parallax. In addition to this paired dataset, we develop a DSM-aware deshadowing model that combines residual translation, perceptual objectives, and mask-constrained adversarial learning. In contrast, a direct adaptation of a UAV-based SRNet/pix2pix architecture fails to converge under satellite viewpoint variability. Our model consistently reduces the visual impact of cast shadows across diverse illumination and viewing conditions, achieving improved structural and perceptual fidelity on held-out scenes. deSEO therefore provides the first reproducible, geometry-aware paired dataset and baseline for shadow removal in satellite Earth observation.

DOI: 10.5194/isprs-annals-XI-3-2026-11-2026

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "deSEO: Physics-Aware Dataset Creation for High-Resolution Satellite Image Shadow Removal".

Tom: The gist: deSEO introduces a geometry-aware and physics-informed methodology that derives paired supervision for satellite shadow removal from the S-EO shadow detection dataset through a fully replicable pipeline.

Jane: First, who's behind it and why it matters.

Paper summary: Tom: So, to wrap up, this paper introduces deSEO as a physics-aware methodology that creates paired supervision for satellite shadow removal directly from the S-EO shadow detection dataset through a fully replicable pipeline.

Jane: It’s important to remember that this wasn't just about applying an existing model; they built a whole process for generating the training data itself, which is what makes it unique.

Lu: The core contribution is twofold, creating a geometry-aware pipeline for pairing samples and then developing a deshadowing model that incorporates residual translation, perceptual objectives, and mask-constrained adversarial learning.

Meng: The results show that the full model achieves the best balance across metrics like SSIM and LPIPS, meaning it handles shadows visually well while maintaining good structural fidelity on held-out scenes.

Tom: And their ablation study confirmed that the digital surface model input is a dominant factor, showing how much it actually helps when you remove it versus when you don't.

Jane: Ultimately, deSEO provides the first reproducible, geometry-aware paired dataset and baseline for shadow removal in satellite Earth observation.

Lu: It establishes a framework upon which more robust and generalisable deshadowing methods can be developed because of this structured data creation approach.

Meng: For engineers on the ground, it means they have a solid foundation to start improving their models without having to reinvent the wheel on how to handle these acquisition variations.

Tom: This paper is about taking existing detection-focused resources and engineering them into datasets that are explicitly designed for weakly supervised deshadowing in high-resolution satellite imagery.

Conclusion: Tom: So, deSEO is basically about fixing how we train AI models to get rid of shadows in high-resolution satellite pictures by creating a really good training set that respects the physics of light.

Jane: It takes this existing shadow detection dataset and builds a whole new pipeline to create paired samples, which means the AI learns exactly what to remove.

Lu: The authors used something called geometry-aware methods and LoFTR registration to match shadowed images with shadow-free ones using temporal and geometric filters. It’s a very structured way of making training data.

Meng: From an engineering standpoint, the method has to handle all that off-nadir viewing angles, which is tricky because the sensor isn't always looking straight down at the Earth.

Lalam: The underlying vision model combines residual translation with perceptual objectives and mask-constrained adversarial learning, which helps it focus only on those shadow areas it needs to clean up.

Tom: So what does this mean? It means we finally have a reproducible way to create data for shadow removal that accounts for how the sensor is angled, which has been a huge problem before.

Jane: It’s about moving past just using pretty picture matching and making sure the model understands the actual geometry of light hitting the scene.

Lu: The authors say this is the first fully replicable pipeline for this kind of paired supervision from that specific S-EO dataset, which opens up a lot of doors for future research.

Meng: I think what's practical here is that if we can generate training data this reliably, we can build much more robust tools for things like building extraction or terrain modeling.

Tom: Exactly. It sets a new standard for how we approach creating supervision signals in these kinds of complex remote sensing tasks.

Jane: And it shows us that combining physics knowledge, like light transport theory, with deep learning techniques really helps when dealing with real-world imagery challenges.

Lu: This work gives us a solid baseline for future AI research on shadow removal because now we have a better way to test and compare different approaches.

Meng: So the next big question is whether this structured dataset creation approach will translate directly into better performance in real-world applications, or if there are still major hurdles when deploying these models.

More episodes

← Home