Attention from Action, for Action: Emergent Visual Bottlenecks for Policy Learning

The gist: Seeker, an action-supervised module that learns where visual evidence is needed for visuomotor control, turns observation–action data into a progression-aware ROI, exposing an explicit spatial bottleneck without relying on semantic boxes, gaze, language grounding, or affordance annotations.

How it works

Seeker is proposed as a task- and state-conditioned readout that learns attention from action by starting from frozen DINO features and iteratively updating a query with gathered visual evidence to produce progression-aware ROIs solely from action supervision. This readout is not a static crop predictor, but rather one whose focus can shift as the task stage and robot state change.

The Seeker architecture involves several key components for iterative refinement:

1. Robot-conditioned Query: The initial query Q(0) is task- and state-conditioned, defined as Q(0) = FiLM(Z, s) = (1 + γ(s)) ⊙ Z + β(s), where Z is the task embedding and s is the proprioceptive state.

2. Iterative Gated Readout: Seeker applies gated cross-attention readout for T refinement steps, where the current query Q∗ = ϕq(Q), MHA(Q∗, K, V) determines the attention map and context.

3. Head Gating: To capture diverse spatial cues, Seeker splits DINO patch features into multiple readout heads and uses query-dependent head gating to update the query based on learned head weights ωh = softmaxh Ch, ϕg(Q¯∗).

Action-supervised ROI Interface

The final context Ce is concatenated with proprioception and a task embedding, then passed to a diffusion action head trained with the standard noise-prediction objective. After multi-task Seeker training, this diffusion head is discarded, and the frozen Seeker readout is used for downstream policies. The resulting ROI attention map Ae is converted into a patch-level ROI by selecting the smallest set of tokens whose cumulative mass exceeds a top p threshold. This explicit ROI is then reused in three downstream forms: RGB cropping, mask-guided background augmentation, and point-cloud filtering.

Performance and Robustness

Seeker improves data efficiency and robustness over no-crop, augmentation, and action-derived crop baselines. In simulation, Seeker raises average success from 42.6% to 62.6%, while in the real world, it raises in-domain success from 48.3% to 76.7% over the best baseline. Furthermore, Seeker ROIs support mask-guided augmentation and improve real-world robustness under both lighting and background shifts, raising average shifted-condition success from 20.0% to 60.0%.

Evaluation Across Modalities

The evaluation tested three claims about action-supervised visual bottlenecks:

1. Action supervision can recover policy-useful ROIs, as Seeker nearly matches the privileged Oracle ROI reference for policy learning without spatial labels.

2. The resulting bottleneck improves policy learning, raising average simulation success from 42.6% to 62.6% and real-world indomain success from 48.3% to 76.7% over the best baseline.

3. The learned ROI is reusable and supports robustness, improving real-world robustness under both lighting and background shifts from 20.0% to 60.0%.

Ablations and Limitations

Ablation studies show that the main benefit comes from exposing the progression-aware ROI as an explicit crop, not merely telling the network where the box is, as low-res cropping nearly matches Seeker on average. Seeker Localizes Rather Than Controls, as direct rollout from its context achieves near-zero average success across tasks. Limitations include that demonstration coverage matters more than raw count, and the training overhead is roughly 1.5–1.8× over training the downstream policy alone.

Conclusion

Seeker turns observation–action data into a progression-aware ROI, exposing an explicit spatial bottleneck without relying on semantic boxes, gaze, language grounding, or affordance annotations. These results point to action-grounded visual bottlenecks as a practical interface between perception and policy learning. This highlights that the visual structure recovered from action supervision depends on the coverage and variation present in the demonstrations.

References

[1] C. Wang, L. Fan, J. Sun, R. Zhang, L. Fei-Fei, D. Xu, Y. Zhu, and A. Anandkumar Mimicplay: Long-horizon imitation learning by watching human play In Conference on Robot Learning (CoRL), 2023

[2] R. Takizawa, I. Karino, K. Nakagawa, Y. Ohmura, and Y. Kuniyoshi Enhancing reusability of learned skills for robot manipulation via gaze information and motion bottlenecks IEEE Robotics and Automation Letters (RA-L), 2025

[3] S. James, K. Wada, T. Laidlow, and A. J Davison Coarse-to-fine q-attention: Efficient learning for visual robotic manipulation via discretisation In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

[4] R. Wang, Z. Zhuang, D. Kragic, and F. T Pokorny PALM: Enhanced generalizability for local visuomotor policies via perception alignment IEEE Robotics and Automation Letters (RA-L), 2026

[5] S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q Jiang, C Li, J Yang, H Su, J Zhu., and L Zhang Grounding DINO: marrying DINO with grounded pre-training for open-set object detection In European Conference on Computer Vision (ECCV), 2024

[6] A. Stone, T. Xiao, Y. Lu, K Gopalakrishnan, K.-H Lee, Q Vuong, P Wohlhart, S Kirmani, B Zitkovich, F Xia., C Finn., and K Hausman Open-world object manipulation using pretrained vision-language models In Conference on Robot Learning (CoRL), 2023

[7] T. Kwon, N Di Palo, and E Johns Language models as zero-shot trajectory generators IEEE Robotics and Automation Letters (RA-L), 2024

[8] J Hu, L Wang, S Li, Y Jiang, X Li, P Weng., and Y Ban Generalizable coarse-to-fine robot manipulation via language-aligned 3d keypoints In International Conference on Learning Representations (ICLR), 2026

[9] V Myers, C Zheng, O Mees, K Fang, and S Levine Policy adaptation via language optimization: Decomposing tasks for few-shot imitation In Conference on Robot Learning (CoRL), 2024

[10] M Shridhar, L Manuelli, and D Fox Perceiver-actor: A multi-task transformer for robotic manipulation In Conference on Robot Learning (CoRL), 2023

[11] A Goyal, V Blukis, J Xu, Y Guo, Y.-W Chao., and D Fox Rvt2: Learning precise manipulation from few demonstrations In Proceedings of Robotics: Science and Systems (RSS), 2024

[12] O Sim’eoni, H V Vo, M Seitzer, F Baldassarre, M Oquab, C Jose, V Khalidov, M Szafraniec., S Yi., M Ramamonjisoa, F Massa., D Haziza., L Wehrstedt., J Wang., T Darcet.

1. Bold ROI interface for policy learning: Seeker turns action supervision into an explicit ROI interface without spatial labels, which allows policies to learn from a progression-aware ROI that shifts with task stage and robot state, improving data efficiency over baselines by raising average in-domain success from the best baseline’s 48.3 to 76.7%.

2. Robust background generalization: The learned ROI supports mask-guided augmentation where Seeker-Guided augmentation preserves predicted control-relevant regions while perturbing task-irrelevant appearance, reducing the burden on the policy to learn under visually distracting textures.

3. Transferable spatial filtering: Seeker's image-plane ROI can serve as a plug-and-play spatial filter for point-cloud policies, improving point cloud success rates by 30.3 absolute points on average over manual workspace cropping, even when trained only from RGB action supervision.