ClimateSOM: A Visual Analysis Workflow for Climate Ensemble Datasets

arXiv:2508.06732 · cs.HC, cs.LG · Submitted 2025-08-08 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "ClimateSOM: A Visual Analysis Workflow for Climate Ensemble Datasets".

Jane: The paper was written by Yuya Kawakami, Daniel Cayan, Dongyu Liu and Kwan-Liu Ma from University of California, Davis and University of California, San Diego.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Title: Tom: Welcome back to the show, everybody. Today we're looking at a paper that's got a mouthful of a title: "ClimateSOM: A Visual Analysis Workflow for Climate Ensemble Datasets."

Jane: And honestly, Tom, that title packs in a lot. Let's break it down. ClimateSOM is a tool, and it's built to help climate scientists make sense of these massive collections of climate model runs they call ensembles.

Tom: Right, and I love that the title leads with the acronym. SOM stands for Self-Organizing Map, which sounds technical, but Jane, you want to explain that in plain terms?

Jane: Sure. Imagine you have thousands of weather maps, and you want to arrange them on a big board so that similar maps are next to each other. That's essentially what a Self-Organizing Map does. It takes all that high-dimensional data and lays it out in a 2D grid where neighboring spots are related.

Tom: So it's like organizing a giant photo album by similarity, but for climate data. And the "workflow" part is key here, because the authors, Kawakami and colleagues from UC Davis and Scripps, they're not just giving us a single visualization. They're giving us a whole process.

Jane: Exactly. And the implications are huge. Climate scientists are drowning in data from projects like CMIP6, which is this huge international effort where dozens of models simulate the future under different emission scenarios. Making sense of all that is a real challenge.

Tom: I mean, we're talking about petabytes of information, right? And each model run is a spatiotemporal time series, so it's a movie of the climate, not just a single snapshot.

Jane: Right, and this paper is trying to help scientists answer questions like, "How does one model compare to another?" or "Which models behave similarly?" Those are the kinds of questions that are really hard to answer when you're just looking at spreadsheets of numbers.

Tom: So the title is really about giving scientists a new way to see their data. And the fact that they're using a Self-Organizing Map, which is a type of neural network, is a clever choice because it preserves the local relationships in the data.

Jane: It does. And the authors are very deliberate about that. They could have used other dimensionality reduction techniques, but the SOM keeps things smooth and interpretable, which is exactly what you need when you're trying to build trust in a visual tool.

Tom: So we've got the title, we've got the core idea. But the real meat is in how they actually built this workflow. That's what we're going to dig into next.

Jane: And I can't wait to talk about the LLM integration, because that's where things get really interesting. Stick around.

Summary: Tom: So we've established that "ClimateSOM" is about organizing climate data visually. Now let's get into the summary of the paper itself. Jane, what's the big picture here?

Jane: The big picture is that they've built a three-step workflow. First, you train the Self-Organizing Map on your ensemble data. Then, you get to "anchor" and "annotate" that map, which means you can drag nodes around to make the space more interpretable, and you can label regions of interest.

Tom: And that's where the Large Language Models come in. They're using LLMs to help with that annotation step. So instead of just eyeballing a bunch of maps, you can ask a question like, "Show me the nodes with above-average precipitation over Southern California," and the system will generate a query to find them.

Jane: And it works in the other direction too. You can select a region on the map and ask the LLM to summarize what's going on there. So it's a two-way conversation with your data, which is pretty wild.

Tom: It really is. And the third step is analysis. Once you've got your annotated map, you can take any model run and see it as a distribution over that map. You can compare two runs, either side-by-side or as a vector field showing how one transitions into the other.

Jane: And they also do clustering. They can group model runs that behave similarly, and they can even cluster the models based on how they respond to different emission scenarios. The paper calls these tasks R1, R2, and R3, but essentially it's explore, compare, and cluster.

Tom: Right, and the case studies they did are really compelling. They used downscaled CMIP6 precipitation data for California and the Pacific Northwest. And they found some genuinely interesting things, like how a model called MIROC6 behaves very differently from the others under a medium-high emission scenario.

Jane: That's a great example. In February, under SSP370, MIROC6 produces a lot of "wetter north, drier south" patterns, while the other models are mostly split between "statewide wet" and "statewide dry." That's the kind of insight that would be really hard to spot with traditional statistical summaries.

Tom: Because traditional methods might just look at the average precipitation, and that would completely miss the fact that the spatial pattern is different. ClimateSOM lets you see that at a glance.

Jane: Exactly. And the authors also compared historical runs to future runs under a high-emission scenario, and they found that in April, the future runs shift towards a wetter northeast, but in November, they shift towards a wetter northwest. That's a seasonal difference that matters a lot for agriculture.

Tom: So the summary is that this tool lets scientists see patterns that are hiding in plain sight. And the expert feedback they got was very positive, with one expert saying they'd "never seen anything like it before."

Jane: That's a strong endorsement. But of course, the paper doesn't just stop at the case studies. They also evaluated the LLM integration specifically, and that's what we should talk about next.

Improvements: Tom: So we've talked about what ClimateSOM does, but the paper also has a section on evaluating the LLM integration. Jane, what did they find?

Jane: They ran a short evaluation with eight participants who know California geography well. They asked them to rate how reasonable the LLM's county selections were for a given region, and how reasonable its summaries were. On a five-point scale where one is "strongly agree," they got an average of two point one one for the county selection and two point five for the summaries.

Tom: So not perfect, but generally reasonable. And the paper is honest about the limitations. For example, the LLM sometimes struggles with vaguely defined regions like "Desert Region of California," and it makes occasional syntax errors in the PostGIS queries.

Jane: Right, about ninety percent of the query generation was accurate, but that last ten percent is where you need a human in the loop. The authors suggest a conversational interface where users could correct the LLM when it makes a mistake, which is a really practical improvement.

Tom: And that's not the only improvement they suggest. They also mention that the visual interface has a learning curve. The experts said it's memorable once you learn it, but getting there takes some effort. So better onboarding tutorials would help.

Jane: And there's a bigger issue too. The current system doesn't provide native statistical significance tests. In climate science, you often need to prove that a pattern is anomalous, not just show it visually. Adding that kind of metric would be a significant upgrade.

Tom: That's a good point. And they also note that the clustering view might not scale well if you have hundreds of model runs. The circuit-line diagram they use could get cluttered.

Jane: But here's the thing, Tom. Even with those limitations, the experts were unanimous that the insights ClimateSOM generates are valuable. One expert said that to get the same insights with their current methods, they'd have to generate a huge number of static plots and sift through them.

Tom: So the improvement isn't just about making the tool prettier. It's about making the scientific process faster and more thorough. And the authors are already thinking about how to do that.

Jane: They mention incorporating statistical tests, improving scalability, and even considering temporal ordering, which they currently ignore. That's a deliberate trade-off they made to keep things simple, but it's a direction for future work.

Tom: So the paper is really a foundation. It shows what's possible, and it lays out a roadmap for what comes next. That's a great place to be.

Jane: It is. And I think the most exciting part is that this approach could be applied beyond climate science. Any field that deals with large ensembles of simulations, like fluid dynamics or materials science, could benefit from this kind of visual analysis.

Tom: That's a big deal. But before we wrap up, let's get Lu and Meng in here to talk about the broader implications.

Conclusion: Tom: Alright, we've covered the title, the summary, and the improvements. Now let's bring in Lu and Meng to help us wrap up our discussion of "ClimateSOM: A Visual Analysis Workflow for Climate Ensemble Datasets."

Jane: Lu, you've been quiet. What's your take on the big picture here?

Lu: I think the most exciting implication is that this workflow makes climate models more transparent. We're not just telling people, "Trust the model." We're showing them why the models differ and where the uncertainties are. That's crucial for building public trust in climate science.

Meng: And from an engineering standpoint, I appreciate that they used a Self-Organizing Map instead of a more complex deep learning approach. It's simpler, it's interpretable, and it runs in about ten minutes for their case studies. That's practical.

Jane: That's a good point, Meng. The tool isn't just a proof of concept. It's something that could actually be deployed in a research lab today.

Tom: And the LLM integration, even with its imperfections, is a step towards making these tools accessible to scientists who aren't visualization experts. You don't need to know how to write a PostGIS query to use it.

Lu: Exactly. And I think the vector field comparison is particularly clever. It treats the change from a historical run to a future run as a flow, which is a very natural way to think about climate change.

Meng: The clustering is also well thought out. Using Earth Mover's Distance to compare distributions is a solid choice, and the circuit-line diagram makes it easy to see how clusters evolve over months.

Jane: So we've got a tool that helps scientists explore, compare, and cluster. And the case studies show it can find real insights, like the MIROC6 anomaly and the seasonal differences in the Pacific Northwest.

Tom: And the experts were excited about it. One said it was something they "couldn't do otherwise." That's a strong statement.

Lu: It is. And I think the future work they outline, adding statistical tests and improving scalability, will only make it more valuable. This is a paper that opens a door.

Jane: It really does. So let's say goodbye to "ClimateSOM: A Visual Analysis Workflow for Climate Ensemble Datasets." It's been a fascinating discussion.

Tom: Absolutely. Thanks to Lu and Meng for joining us. And to our listeners, thanks for tuning in. We'll be back with another paper soon.

Jane: Until then, keep asking questions and keep looking at the data. There's always more to see.

Yuya Kawakami, Daniel Cayan, Dongyu Liu, Kwan-Liu Ma

University of California, Davis · University of California, San Diego

cs.HC, cs.LG

Submitted: 2025-08-08

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 58/100

The gist: The paper introduces ClimateSOM, a visual analysis workflow designed to support the interactive exploration and interpretation of climate ensemble datasets.

Key concepts

Self-Organizing Map (SOM)
A SOM is a technique that takes high-dimensional data, like thousands of weather maps, and arranges them onto a 2D grid. This process groups similar maps next to each other, making complex data easier to visualize by showing local relationships.
Workflow
ClimateSOM provides a three-step process for analysis: training the SOM on ensemble data, anchoring and annotating the map with labels, and finally performing analysis like comparing model runs or clustering models based on their behavior.
LLM Integration
Large Language Models are used to assist in the annotation step of ClimateSOM. They allow scientists to ask questions about the data in natural language, such as requesting nodes with above-average precipitation, and receive generated queries or summaries.
Ensemble Datasets
These are massive collections of climate model runs from international efforts like CMIP6. They simulate future climate scenarios under different emission levels, presenting a huge volume of spatiotemporal data that is challenging to interpret manually.

Terminology

Summary

The paper introduces ClimateSOM, a visual analysis workflow designed to support the interactive exploration and interpretation of climate ensemble datasets. The workflow leverages a self-organizing map (SOM) and Large Language Models (LLMs) to abstract spatiotemporal time series—each representing a climate model run—into a distribution over a user-steerable 2D space.

The paper identifies three essential tasks for working with ensemble datasets, based on collaboration with a climate scientist: (1) exploring the general behavior of a single set of model runs, (2) comparing the behavior of two sets of model runs, and (3) clustering the behavior of multiple model runs. These tasks, along with interpretability, serve as the guiding principles for the system.

The workflow consists of one pre-step and three main steps. In the pre-step (S0), a SOM is trained on the flattened ensemble dataset. The paper states that "the SOM must meet two competing criteria: (1) it should retain sufficient information from the input data, and (2) the 2D grid of nodes should be smooth enough to avoid disconnections after projection, which would hinder interpretability." The parameters kR (initial neighborhood update size ratio) and kS (final neighborhood update size ratio) are critical for balancing these criteria.

In the first main step (S1: Anchor), users create an Adjusted SOM Node Space by projecting the SOM nodes into a 2D layout using Minimum-Distortion Embedding (MDE). Users can anchor specific nodes—fixing their positions while allowing others to adjust—to create a more interpretable spatial arrangement than the original SOM grid.

In the second main step (S2: Annotate), users annotate the Adjusted SOM Node Space to produce an Annotated SOM Node Space. An annotation consists of a bounded region within the adjusted space and a text label describing that region. ClimateSOM integrates LLMs to assist with this process in two directions. In the forward direction, users can ask natural language queries (e.g., Show me nodes that have average above 0 over Southern California) and the system generates PostGIS queries to identify the corresponding region. The paper notes that we delegate the LLM not to generate spatial boundaries, but instead to generate a list of counties that correspond to the named region to reduce errors. In the backward direction, users can select a region and the LLM generates a textual summary of the SOM nodes in that region, using per-county summary statistics.

In the third main step (S3: Analyze), each model run is described as a distribution within the Annotated SOM Node Space. For each time step, the best matching unit (BMU) is located in the SOM, and its projected 2D position represents that time step. Aggregating these positions across all time steps produces a 2D distribution. This distributional representation serves as the backbone for the analysis tasks.

For exploring a single set of model runs (R1), ClimateSOM computes a kernel density estimate (KDE) over the distribution, which serves as a fingerprint of the model run. This distribution can also be expressed as percentages over the annotated regions.

For comparing two sets of model runs (R2), ClimateSOM provides two methods: side-by-side comparison and a vector field formulation. The vector field formulation uses a bootstrap approach with 2D optimal transport to represent the spatial transition from one model run to another.

For clustering the behavior of many model runs (R3), ClimateSOM uses Earth Mover's Distance (EMD) to quantify similarity between distributions, followed by UMAP for 2D embedding and HDBSCAN for clustering. A second variant clusters GCMs based on their SSP forcing, using average cosine similarity between vector fields.

The visual interface includes several views: the SOM Node Space Editing View for anchoring and annotating, the Model Run Inspection View showing KDE distributions, the Characteristic View showing breakdowns by annotations, the Model Comparison View for side-by-side or vector field comparisons, and the Clustering View using a circuit-line diagram to show monthly clustering results and their evolution.

The paper demonstrates the utility of ClimateSOM through case studies using LOCA-downscaled CMIP6 precipitation projections over California and the Northwestern United States. In the California case study, the expert identified five annotation regions (Statewide Wet, Wetter North - Drier South, Wetter South - Drier North, Intermediate, and Statewide Dry) and discovered that model runs predicting a wetter California in January are associated with a drier November with particular dryness in the North. In the Northwestern US case study, the expert found that the effect of a higher emissions future on precipitation is drastically different between April and November, with April expecting higher precipitation in the Northeast.

The paper also evaluates the LLM integration with 8 participants, finding average scores of 2.11 (SD=1.32) for region-to-county mapping and 2.5 (SD=1.55) for county-to-summary tasks on a 5-point Likert scale, suggesting the LLM generates reasonable results. The PostGIS query generation was accurate in 90% of test cases.

An expert review with six domain experts (12 to 48 years of experience) confirmed that ClimateSOM can uncover climatologically significant insights, many of which are novel or would otherwise require substantial manual effort. Experts noted the visual complexity of the workflow but positively reviewed its memorability. Limitations identified include the lack of native statistical significance metrics, scalability concerns for the Clustering View with more model runs, the abstraction ignoring temporal ordering, and SOM training time growing linearly with grid size.

Improvements for AI systems

Based on the paper, here are specific improvements I can make to AI systems:

Improvement: Build a spatial reasoning layer on top of LLMs that:

  • Resolves vague geographic regions (e.g., Southern California) to county-level boundaries using a structured county database, rather than relying on the LLM to generate coordinates directly

  • Uses a two-stage pipeline: (1) LLM detects region names and maps them to counties, (2) a deterministic PostGIS query builder constructs the spatial filter

  • Includes a validation step that checks for SQL syntax errors and corrects them automatically

What it can do: Answer natural language queries like Show me SOM nodes with average precipitation above 0 over Southern California with 90% accuracy, reducing the 10% syntax error rate reported in the paper.

Improvement: Create a summarization module that:

  • Takes a region in a 2D latent space (e.g., SOM node space)

  • Samples nodes evenly within that region

  • Converts per-county statistics into a structured text format (high/mod high/neutral/mod low/low categories)

  • Uses LLM to generate a 50-word summary describing regional precipitation patterns

Improvement: Implement a conversational interface that supports:

  • Forward direction: User asks Show me nodes with value X in region Y → system highlights the corresponding region in the visualization

  • Backward direction: User selects a region in the visualization → system generates a textual summary of that region

  • Maintains a chat history so users can iteratively refine queries and see how their understanding evolves

Improvement: Build a module that:

  • Takes any spatiotemporal time series (e.g., climate model runs)

  • Maps each time step to its best-matching unit (BMU) in a trained SOM

  • Projects BMU locations into a 2D space using Minimum-Distortion Embedding (MDE)

  • Produces a kernel density estimate (KDE) as a fingerprint of the model run

Improvement: Create a clustering system that:

  • Uses Earth Mover's Distance (EMD) to compute similarity between model run distributions

  • Applies UMAP + HDBSCAN for clustering

  • Visualizes results as circuit-line diagrams showing cluster evolution over time

  • Supports comparison via optimal transport-based vector fields, showing spatial transitions between two scenarios

Improvement: Implement a user-steerable projection system that:

  • Lets users drag specific nodes to anchor positions in the 2D space

  • Recomputes the embedding using MDE with these anchors as constraints

  • Allows iterative refinement so users can align the projection with their domain knowledge (e.g., placing wet nodes on the left, dry nodes on the right)

Abstract

Ensemble datasets are ever more prevalent in various scientific domains. In climate science, ensemble datasets are used to capture variability in projections under plausible future conditions including greenhouse and aerosol emissions. Each ensemble model run produces projections that are fundamentally similar yet meaningfully distinct. Understanding this variability among ensemble model runs and analyzing its magnitude and patterns is a vital task for climate scientists. In this paper, we present ClimateSOM, a visual analysis workflow that leverages a self-organizing map (SOM) and Large Language Models (LLMs) to support interactive exploration and interpretation of climate ensemble datasets. The workflow abstracts climate ensemble model runs - spatiotemporal time series - into a distribution over a 2D space that captures the variability among the ensemble model runs using a SOM. LLMs are integrated to assist in sensemaking of this SOM-defined 2D space, the basis for the visual analysis tasks. In all, ClimateSOM enables users to explore the variability among ensemble model runs, identify patterns, compare and cluster the ensemble model runs. To demonstrate the utility of ClimateSOM, we apply the workflow to an ensemble dataset of precipitation projections over California and the Northwestern United States. Furthermore, we conduct a short evaluation of our LLM integration, and conduct an expert review of the visual workflow and the insights from the case studies with six domain experts to evaluate our approach and its utility.

Sources

Related papers