Zero-Shot Cross-Lingual Recognition of Sign Language Handshapes

arXiv:2609.18772 · cs.CL, cs.CV · Submitted 2026-09-16 · Read on arXiv

cs.CL, cs.CV

Submitted: 2026-09-16

Updated: 2026-09-16

Comments: Accepted at the Workshop on Sign Language Processing (WSLP), EMNLP 2026

Project page: https://marcelgranero.github.io/cross-lingualhandshapes

License: http://creativecommons.org/licenses/by/4.0/

The gist: Sign language processing advances rapidly for high-resource languages such as American Sign Language (ASL), yet most of the world's sign languages lack the phonological annotations new methods

Terminology

Abstract

Sign language processing advances rapidly for high-resource languages such as American Sign Language (ASL), yet most of the world's sign languages lack the phonological annotations new methods require. We present the first zero-shot cross-lingual framework for handshape recognition, transferring from ASL to Catalan Sign Language (LSC). Our approach leverages the decomposition of handshapes into five phonological features -- selected fingers, flexion, spread, thumb position, and thumb contact -- shared across both languages, to decode LSC handshapes from predicted features via a composite phonological distance metric. We evaluate three architectures (MLP, SL-GCN, SHuBERT) trained on two ASL corpora (PopSign, Sem-Lex) against a 37-handshape, single-signer LSC benchmark. Zero-shot transfer proves viable once recording-format disparities are harmonized, reaching 80.0% phonological feature accuracy and 54.5% expected handshape accuracy. Phonological decomposition thus offers a bridge for extending sign language technologies to low-resource languages without any target-language video training labels.

Sources

Related papers