MarsRecon: Self-Supervised and Multimodal Surface Representations for Mars

arXiv:2609.22379 · cs.CV, cs.AI · Submitted 2026-09-17 · Read on arXiv

cs.CV, cs.AI

Submitted: 2026-09-17

Updated: 2026-09-17

Comments: 27 pages, 22 figures

License: http://creativecommons.org/licenses/by/4.0/

The gist: High-resolution orbital imagery offers a rich record of the Martian surface, but sparse geological labels limit supervised representation learning.

Terminology

Abstract

High-resolution orbital imagery offers a rich record of the Martian surface, but sparse geological labels limit supervised representation learning. We present MarsRecon, a geospatially aware pipeline for learning visual and multimodal representations from HiRISE observations of Olympus Mons. The pipeline calibrates NASA Planetary Data System products, extracts valid georeferenced patches, and trains a masked autoencoder on unlabeled imagery. Increasing input resolution and filtering invalid tokens reduced held-out reconstruction loss from 0.1751 to 0.1342 in the principal Stage A model series. We then freeze the visual encoder and align its features with observation text, coordinates, and local--global image context. The strongest current local-primary model achieves image-to-text recall@10 of 0.3787, text-to-image recall@10 of 0.9161, and local-to-global recall@10 of 0.4350 on the held-out test split. These results establish a working Mars-specific pretraining and retrieval pipeline; further crop-overlap controls and downstream geological evaluations are needed to assess the broader utility of its embeddings.

Related papers