XS-VLA: Teaching Tiny Vision-Language-Action Models with Spatial Supervision and Demonstration Conditioning

arXiv:2607.04171 · cs.RO, cs.LG · Submitted 2026-07-05 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "XS-VLA: Teaching Tiny Vision-Language-Action Models with Spatial Supervision and Demonstration Conditioning".

Dev: Tiny Vision-Language-Action (VLA) models are crucial for real-time robotic control, but scaling them down often compromises essential capabilities like task-conditioned spatial grounding and coherent action generation.

Rosa: First, who's behind it and why it matters.

Title and authors: Rosa: Moving on to the title of this work, "XS-VLA: Teaching Tiny Vision-Language-Action Models with Spatial Supervision and Demonstration Conditioning," it immediately tells us the core idea is about teaching these small VLA models a specific set of skills. It’s not just about making them bigger or faster; it’s about giving them the right kind of knowledge to perform complex physical tasks.

Dev: I think that "Spatial Supervision" points directly toward the localization part, which is what we talked about earlier, and "Demonstration Conditioning" suggests they are focusing on how to handle different input styles for movement. It’s a targeted approach rather than a general scaling effort.

Taro: I'm curious if this means that even with a tiny model, we can achieve the precision needed for tasks requiring fine motor control, because spatial grounding is often where those models fail in the lab setting when things get slightly tricky.

Rosa: That’s exactly what it addresses; it gives them explicit instruction on object location so they don't just guess where to look and interact with. It uses a Qwen3-VL-4B teacher to guide that process without needing tons of human annotation data for bounding boxes.

Dev: From an engineering standpoint, the fact that this spatial information is distilled into a fixed three times three grid vocabulary makes the input deterministic, which simplifies things immensely when we think about deployment and ensuring consistent performance across different runs.

Taro: That determinism in the spatial cues is important because it means the model’s "where to look" behavior becomes predictable, which is a key feature for any autonomous system operating in an unpredictable environment.

Rosa: And on the action side, the demonstration conditioning part using Latent Flow Matching addresses how to make those actions stick together even when human demonstrations have different styles. It focuses on learning coherent motion chunks rather than just memorizing individual steps.

Dev: So, we're not just teaching it what to see; we're teaching it how to translate that visual understanding into a consistent sequence of physical movements that generalize across varied examples. That’s the full picture of XS-VLA.

Taro: It’s interesting because it tackles the organization problem directly, which is something I think is a big hurdle for scaling up these VLA systems effectively.

Rosa: Right, and this whole approach seems tailored to bridge the gap between high-level language understanding and low-level physical execution in a resource-constrained setting.

The paper's summary: Dev: So, to summarize the main point of "XS-VLA: Teaching Tiny Vision-Language-Action Models with Spatial Supervision and Demonstration Conditioning," the paper outlines a system that solves the problem of weak spatial grounding and poor action organization in tiny VLA models.

Rosa: The core summary is that they introduce XS-VLA, which teaches these small models two key competencies: where to look and how to move. They achieve this by separating the training into two stages.

Dev: First, they use Coarse-Grained Spatial Distillation with Qwen3-VL-4B to inject coarse task-relevant spatial cues through a three times three spatial vocabulary into the student model backbone. That’s about teaching it where to look.

Taro: So, that first step builds a foundation of visual awareness that is constrained and task-conditioned, which should be much more reliable than relying on the model's raw visual features alone.

Rosa: Exactly; this stage injects structured knowledge directly into the backbone, bypassing the need for human spatial annotations by using automated pseudo-labeling. It makes sure the model understands object locations related to manipulation without manual work.

Dev: Then, they integrate this spatially enhanced backbone into a policy trained with Latent Flow Matching to organize multimodal demonstrations for continuous action generation. That second stage focuses on teaching the model how to move smoothly and consistently across different examples.

Taro: I see that the latent flow matching then takes those diverse human actions and organizes them into a coherent latent space, which helps prevent the model from just producing averaged or unstable behaviors during training.

Rosa: So, in short, it’s a two-pronged approach: spatial supervision first to teach where to look, followed by demonstration conditioning to teach how to move coherently. This is what XS-VLA aims to achieve on the 0 point 25B scale while maintaining high manipulation performance compared to Vanilla SmolVLA-0 point 25B.

Dev: The summary is that this framework allows tiny VLA models to become effective robot policies when they are explicitly trained on structured spatial priors and organized action learning techniques, leading to substantial improvements across benchmarks like LIBERO and mobile ALOHA tasks.

The paper's improvements: Rosa: Now let’s talk about what the paper suggests as its specific improvements to the existing methods, because they aren't just suggesting a general idea but concrete technical changes.

Dev: They emphasize that their contribution is providing an automated pipeline for spatial supervision, specifically Coarse-Grained Spatial Distillation. Instead of relying on manual annotation of keypoints or bounding boxes, they use Qwen3-VL-4B to generate those labels automatically.

Taro: That automation in generating the spatial cues is a big deal because it removes a major bottleneck—human effort—and makes the system more scalable for use with diverse datasets.

Rosa: It also points out that their method of using symbolic distillation specifically for spatial reasoning, which is different from traditional logit matching, allows them to inject this structured knowledge without needing human annotations or using the teacher model at deployment time.

Dev: Then there’s the Latent Flow Matching component which organizes the action learning by introducing a latent variable that conditions the decoder during training, which solves the problem of unstable behaviors during deployment.

Taro: So, organizing multimodal demonstrations into a coherent latent space means we’re not just getting a collection of random actions; we're getting something structured and reproducible for execution.

Rosa: That leads to the conclusion that these specific mechanisms—CSD and LFM—are what unlock high performance on the 0 point 25B scale when used together, showing that targeted spatial supervision and structured action learning are key ingredients.

Dev: The improvement they highlight is that by using the prior mean for the latent variable during deployment, they prevent those multimodal demonstrations from collapsing into averaged and unstable behaviors, which is a crucial stability point for us.

Conclusion: Rosa: Wrapping up this discussion on "XS-VLA: Teaching Tiny Vision-Language-Action Models with Spatial Supervision and Demonstration Conditioning," the main conclusion is that this framework successfully teaches tiny models both where to look through spatial supervision and how to move coherently via latent flow matching.

Dev: The authors conclude that when these two mechanisms are used together, the resulting system achieves substantial performance gains on benchmarks like LIBERO and in real-world mobile ALOHA tasks, proving that compact VLA models can be effective robot policies.

Taro: I think the biggest implication is that this validates using targeted knowledge injection as a way to boost small architectures beyond their natural limitations.

Rosa: It suggests that instead of simply pushing for bigger models, we should focus on finding these specific ways to inject task-relevant structure into smaller models effectively, which is a very practical direction for field deployment.

Dev: We’re looking at a model that maintains efficiency while delivering tangible performance improvements in real-time control loops, so the stability provided by the latent variable prior mean during deployment is definitely something we need to keep focusing on.

Taro: It confirms that for autonomy, solving the problem of structural organization and localization is often more critical than just raw parameter count when dealing with limited resources.

Rosa: So, in essence, XS-VLA provides a tangible blueprint for getting robust manipulation out of tiny models by focusing precisely on spatial grounding and action coherence.

Department of Computer Science and Technology, Tsinghua University

cs.RO, cs.LG

Submitted: 2026-07-05

Updated: 2026-09-30

Comments: Preprint

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 90/100

The gist: Tiny Vision-Language-Action (VLA) models are crucial for real-time robotic control, but scaling them down often compromises essential capabilities like task-conditioned spatial grounding and coherent

Key concepts

Spatial Supervision
This involves giving tiny VLA models explicit instruction on object locations. It uses a Qwen3-VL-4B teacher to guide the student model using a three times three grid vocabulary, allowing for automated pseudo-labeling of object locations without needing manual human annotation.
Demonstration Conditioning
This technique addresses how to make actions stick together when human demonstrations have different styles. It uses Latent Flow Matching to organize multimodal demonstrations into a coherent latent space, teaching the model how to move smoothly and consistently across varied examples.
Latent Flow Matching
This component organizes diverse human actions into a coherent latent space by introducing a latent variable that conditions the decoder during training. This helps prevent unstable or averaged behaviors during deployment by using the prior mean for the latent variable.
Coarse-Grained Spatial Distillation (CSD)
This is an automated pipeline for spatial supervision. Instead of relying on manual keypoint or bounding box annotation, it uses a teacher model to generate coarse task-relevant spatial cues and inject this structured knowledge directly into the student model's backbone.

Terminology

Summary

Tiny Vision-Language-Action (VLA) models are crucial for real-time robotic control, but scaling them down often compromises essential capabilities like task-conditioned spatial grounding and coherent action generation. This paper introduces XS-VLA, a lightweight framework designed to teach tiny VLA policies precisely where to look and how to move without increasing deployment cost. By explicitly addressing these two bottlenecks—weak spatial grounding and poor organization of multimodal demonstrations—XS-VLA demonstrates that compact models can achieve high manipulation performance when equipped with targeted spatial supervision and structured action learning.

How it works

XS-VLA operates in two distinct stages: first, teaching the model where to look through Coarse-Grained Spatial Distillation (CSD), and second, teaching how to move via Latent Flow Matching (LFM). The framework is designed to be lightweight, operating effectively on a 0.25B parameter scale.

  1. The student backbone is initialized by performing CSD: Coarse-Grained Spatial Distillation uses Qwen3-VL-4B to produce teacher-derived coarse image-plane location labels for taskrelevant objects. These predicted centers are deterministically quantized into a 3x3 spatial vocabulary, producing coarse labels such as top left, center, and bottom right. The student is then trained to predict this label from the original observation and instruction, injecting coarse task-conditioned spatial cues into the model.

  2. The spatially enhanced backbone is then integrated into a VLA policy trained with Latent Flow Matching (LFM): Latent Flow Matching combines a CVAE-style latent variable with flow-based policy learning to organize multimodal demonstrations during training and produce stable action chunks at deployment. During training, a CVAE encoder observes the proprioceptive state and ground-truth action chunk, parameterizing a latent distribution. The final policy decoder conditions on visual language features, robot state, and this latent intent variable, which is set to its prior mean during deployment to yield deterministic and stable action generation.

Key Components of XS-VLA

The framework addresses the dual challenge of tiny VLA models by introducing specific mechanisms for spatial grounding and motion organization:

(1) Coarse-Grained Spatial Distillation (CSD):

This stage provides explicit task-conditioned spatial supervision before action learning. It uses a teacher model, Qwen3-VL-4B, to predict the image-plane center of the target object. The predicted center is quantized into a 3x3 grid, and this coarse spatial label is used as a constrained text generation objective for the student SmolVLM2-0.25B backbone. This procedure separates localization from label generation and ensures a consistent vocabulary across the dataset without requiring human annotations for bounding boxes or keypoints.

(2) Latent Flow Matching (LFM):

This component organizes diverse motion behaviors by introducing a latent variable into the policy learning process. The CVAE encoder observes the proprioceptive state and action chunk to parameterize a latent distribution, which is then used to condition the flow-matching decoder. This allows the model to learn coherent continuous action chunks from diverse multimodal demonstrations. Crucially, during deployment, using the prior mean for this latent variable ensures deterministic and stable action generation, preventing the collapse of multimodal demonstrations into averaged and unstable behaviors.

Performance and Results

XS-VLA demonstrates significant performance gains across various benchmarks. On the LIBERO benchmark, XS-VLA improves average success from 82.8% to 90.3% over Vanilla SmolVLA-0.25B, showing a strong improvement on LIBERO-Long from 63.0% to 89.0%. In real-world Mobile ALOHA experiments involving multi-style human demonstrations, XS-VLA improves average task success from 21.7% to 65.0%. Ablation studies confirm the complementarity of the components: adding LFM alone improves performance to 87.4%, while spatial pretraining without LFM reaches 88.8%, and the full model achieves the best result of 90.3%.

Contributions

The paper makes four primary contributions:

  1. A where-to-look and how-to-move framework for tiny VLA models, identifying spatial grounding and multimodal organization as key bottlenecks within a compact 0.25B model budget.

  2. Coarse-Grained Spatial Distillation, an automated pipeline where Qwen3-VL-4B predicts target object centers, which are deterministically quantized into a 3x3 spatial vocabulary and distilled into SmolVLM2-0.25B to inject coarse task-conditioned spatial cues without human annotations.

Improvements for AI systems

Here are specific, actionable improvements derived from the XS-VLA framework, detailing what the improved AI system can achieve:

The core improvement is transforming tiny Vision-Language-Action (VLA) models from brittle learners into robust manipulators by explicitly teaching them two missing capabilities: precise spatial grounding (where to look) and coherent action generation (how to move).

Here are the specific improvements and resulting system capabilities:

  1. The improved AI system can achieve high-performance robotic manipulation using a highly compressed, ultra-lightweight model (0.25B parameters) that rivals or exceeds larger models (like 7B) on specific benchmarks like LIBERO and real-world tasks.

  2. It can reliably execute complex, multi-stage, and bimanual coordination tasks in the real world with high success rates (up to 90% for single-arm placing).

  3. The system exhibits superior robustness when faced with diverse human demonstration styles (multimodal data), reducing the tendency of compact models to produce averaged, incoherent actions.

Specific Mechanisms and Their Resulting Capabilities:

  1. The system uses a two-stage training process that injects structured knowledge before policy learning:

  2. It first uses a powerful teacher VLM (Qwen3-VL-4B) to generate coarse spatial supervision via Coarse-Grained Spatial Distillation (CSD). This allows the tiny student model to learn where task-relevant objects are located in the image plane using a deterministic 3x3 grid vocabulary, without requiring costly human annotations.

  3. This spatial initialization biases the compact visual backbone toward object regions relevant for manipulation, improving task-conditioned visual grounding independently of action prediction.

  4. The system then uses Latent Flow Matching (LFM) to learn continuous actions from multimodal demonstrations:

  5. It employs a CVAE-style latent variable to organize the latent intent of diverse human motions during training, effectively disentangling conflicting styles into a coherent latent space.

  6. During deployment, the system utilizes this latent variable deterministically (setting it to the prior mean), ensuring stable and reproducible action chunks that do not average incompatible behaviors.

  7. The final model is a highly efficient VLA policy that integrates these learned spatial cues with latent intent, allowing it to perform complex tasks like high-precision bimanual block stacking and long-horizon sequential coordination (e.g., spoon transfer to a bowl).

In summary, the improved AI system can achieve:

"Ultra-lightweight, real-time robotic control that achieves 90%+ success in complex physical manipulation tasks by using teacher-derived spatial priors to define 'where to look' and latent flow matching to generate 'how to move' coherently across diverse human demonstrations."

Sources

Related papers