Earth observation embeddings are effective sub-grid descriptors for probabilistic weather downscaling

This paper investigates whether Earth observation (EO) foundation model embeddings can serve as effective sub-grid surface descriptors for probabilistic weather downscaling. The authors augment a convolutional conditional neural process (ConvCNP) that downscales coarse ERA5 reanalysis fields at 25 km resolution with a learned local surface descriptor, obtained by compressing a patch of Tessera embeddings at 10 m resolution. The key findings are:

1. Main result: "Although these embeddings summarise surface conditions over annual timescales, they improve downscaling of instantaneous 2 m temperature and 10 m wind speed by encoding persistent surface properties that capture a location’s departure from the coarse-grid atmospheric state."

2. Quantitative improvement: "Across five climatically diverse regions, the embedding improves point and probabilistic skill at stations held out in both space and time, overall improving CRPS skill by 11.5% for 2 m temperature and 6.2% for 10 m wind speed."

3. Variable-dependent contribution: We further analyse how its contribution differs by variable, finding that topography explains more of temperature’s sub-grid structure, while Tessera provides additional surface information for wind speed.

4. Robustness: These improvements persist when the coarse input is changed from ERA5 reanalysis fields to forecasts from the Aurora AI forecasting model, and when predicting at newly deployed stations with no regional history.

5. Novelty claim: "To our knowledge, this is the first evidence that long-timescale Earth-observation embeddings can support short-timescale weather downscaling where sub-grid departures are systematically structured by persistent surface properties."

The paper's contributions are:

• Constructing a learned sub-grid descriptor from EO foundation-model embeddings using a VAE to compress a 640 m neighbourhood of per-pixel embeddings into a 16-dimensional vector.

• Demonstrating consistent gains in off-grid probabilistic downscaling across five regions (Europe, United States, East Asia, Southern Africa, Australia).

• Differentiating qualitatively how the embedding corrects temperature (mostly via elevation) and wind (via additional land-surface information).

• Demonstrating robustness to forecast weather fields from Aurora.

• Showing improved sample efficiency in a simulated Norwegian network ramp-up, where Tessera provides competitive wind-speed downscaling before any local observations are available.

The paper concludes that a frozen EO foundation representation can serve as an effective point-specific surface descriptor for probabilistic downscaling at previously unseen locations, and demonstrates how two independently trained foundation models can be composed: Aurora represents the evolving atmospheric state, while Tessera represents the local surface through which that state is expressed.

Improvements to AI systems:

1. Add a static context encoder module to neural emulators of physical fields. Instead of only ingesting dynamic inputs (e.g., weather states, boundary conditions), the AI system should compress high-resolution static embeddings (e.g., satellite-derived surface features) into a low-dimensional latent vector that is concatenated or cross-attended to the dynamic state at each grid cell. This allows the model to learn location-specific biases without requiring dense local observations.

2. Implement a two-timescale fusion training scheme. Train the AI system to jointly optimize for short-timescale predictive accuracy (e.g., instantaneous fields) while explicitly regularizing the static embedding to preserve long-timescale surface properties (e.g., via a variational autoencoder loss on the embedding). This prevents the model from overfitting to transient dynamics and forces the static descriptor to encode persistent, physically meaningful features.

3. Add a variable-aware gating mechanism that learns to weight the contribution of the static descriptor per output variable. For temperature, the gate should emphasize topographic/elevation channels; for wind, it should emphasize land-cover/roughness channels. This can be implemented as a small per-variable attention head over the static latent vector, improving interpretability and performance.

4. Enable zero-shot deployment at unseen locations by making the static descriptor the primary location identifier, rather than relying on station history or regional climatology. The AI system should be able to take any new coordinate, extract its EO embedding on the fly, and produce calibrated probabilistic predictions immediately, without fine-tuning.

5. Build a cross-foundation-model composition layer that formally couples a dynamic foundation model (e.g., Aurora for atmospheric states) with a static foundation model (e.g., Tessera for surface properties). The composition layer should handle resolution mismatch, temporal alignment, and uncertainty propagation between the two, enabling end-to-end differentiable downscaling from coarse forecasts to fine-grained surface predictions.

What the improved AI system can do:

• Instant, location-agnostic probabilistic downscaling of temperature and wind speed at any point on Earth, using only a coarse atmospheric forecast and a precomputed EO embedding, with no need for local historical observations.

• Variable-specific physical correction: automatically uses elevation for temperature and land-surface roughness/cover for wind, reducing systematic biases that pure dynamic models miss.

• Robust performance across diverse climates and data sources: works equally well with reanalysis or AI-generated forecasts, and maintains skill when deployed to new stations or regions.

• Sample-efficient learning: in sparse observational networks (e.g., newly installed sensors), it can produce competitive wind-speed predictions immediately, then improve as local data arrives.

• Interpretable sub-grid structure: outputs not only predictions but also the relative contribution of static vs. dynamic factors per variable, aiding scientific analysis of how surface properties shape local weather.

Global weather reanalyses and forecasts resolve the evolving atmospheric state on coarse grids, but site-specific applications require predictions at arbitrary locations where near-surface conditions also depend on unresolved terrain and land-surface properties. Existing probabilistic downscalers address this gap using hand-crafted topographic descriptors. We ask instead whether Earth observation foundation models can provide transferable sub-grid surface representations for probabilistic weather downscaling. We augment a convolutional conditional neural process that downscales coarse ERA5 reanalysis fields at 25 km resolution with a learned local surface descriptor, obtained by compressing a patch of TESSERA embeddings at 10 m resolution. Although these embeddings summarise surface conditions over annual timescales, they improve downscaling of instantaneous 2 m temperature and 10 m wind speed by encoding persistent surface properties that capture a location's departure from the coarse-grid atmospheric state. Across five climatically diverse regions, the embedding improves point and probabilistic skill at stations held out in both space and time, overall improving CRPS skill by 11.5% for 2 m temperature and 6.2% for 10 m wind speed. We further analyse how its contribution differs by variable, finding that topography explains more of temperature's sub-grid structure, while TESSERA provides additional surface information for wind speed. These improvements persist when the coarse input is changed from ERA5 to forecasts from the Aurora AI forecasting model, and when predicting at newly deployed stations with no regional history. To our knowledge, this is the first evidence that long-timescale Earth-observation embeddings can support short-timescale weather downscaling where sub-grid departures are systematically structured by persistent surface properties.