FingerViP: Learning Real-World Dexterous Manipulation with Fingertip Visual Perception

arXiv:2604.21331 · cs.RO · Submitted 2026-04-23 · Read on arXiv

cs.RO

Submitted: 2026-04-23

Updated: 2026-09-24

Code: https://github.com/Improbable-AI/VisionProTeleop

Terminology

• Third-View Camera — 13× in this paper
• Transformer — 10× in this paper · explained in 2 episodes
• Alignment — 7× in this paper · explained in 1 episode
• Diffusion Model — 6× in this paper · explained in 8 episodes
• Diffusion Models — 5× in this paper · explained in 6 episodes
• Wrist Camera — 5× in this paper
• Foundation Model — 4× in this paper · explained in 3 episodes
• Generalization — 4× in this paper · explained in 13 episodes
• International Conference on Robotics and Automation — 3× in this paper
• Reinforcement Learning — 3× in this paper · explained in 4 episodes
• Robustness — 3× in this paper · explained in 5 episodes
• Ablation — 2× in this paper · explained in 1 episode
• Adaptability — 2× in this paper · explained in 1 episode
• Consistency — 2× in this paper · explained in 3 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Action Decoder — 1× in this paper · explained in 1 episode
• Behavioral Cloning — 1× in this paper
• Benchmarking — 1× in this paper · explained in 4 episodes
• Calibration — 1× in this paper · explained in 7 episodes
• Data Augmentation — 1× in this paper · explained in 7 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Human Teleoperation — 1× in this paper
• Imitation Learning (IL) — 1× in this paper · explained in 2 episodes
• Multimodal Dataset — 1× in this paper · explained in 2 episodes
• Regret — 1× in this paper · explained in 3 episodes
• Temporal consistency — 1× in this paper · explained in 2 episodes
• Vision-language models — 1× in this paper · explained in 2 episodes

Sources

• GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
• End to End Learning for Self-Driving Cars
• Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• DEXOP: A Device for Robotic Transfer of Dexterous Human Manipulation
• Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes
• CordViP: Correspondence-based Visuomotor Policy for Dexterous Manipulation in Real-World
• MAPLE: Encoding Dexterous Robotic Manipulation Priors Learned From Egocentric Videos
• EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video
• Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy
• 3D Equivariant Visuomotor Policy Learning via Spherical Projection
• OPEN TEACH: A Versatile Teleoperation System for Robotic Manipulation
• PP-Tac: Paper Picking Using Tactile Feedback in Dexterous Robotic Hands
• Learning Visuotactile Skills with Two Multifingered Hands
• VO-DP: Semantic-Geometric Adaptive Diffusion Policy for Vision-Only Robotic Manipulation
• In-Hand Object Pose Estimation via Visual-Tactile Fusion
• Imitating Human Behaviour with Diffusion Models
• AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System
• REASSEMBLE: A Multimodal Dataset for Contact-rich Robotic Assembly and Disassembly
• RAPID Hand: A Robust, Affordable, Perception-Integrated, Dexterous Manipulation Platform for Generalist Robot Autonomy

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
• Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving