TDC: Sim-to-Real Transferable Directional Compliance for Contact-Rich Manipulation

Sim-to-real transfer for contact-rich manipulation remains challenging due to inherent discrepancies in contact dynamics, and this work proposes a framework that leverages expert-designed controller logic to enable robust policy transfer from simulation to the real world. The core finding is that predicting the contact force direction, which encodes high-level task geometry and is invariant to simulation inaccuracies, allows for training policies exclusively on simulation data, which are then deployed with a lightweight, manually tuned force-aware admittance controller for adaptive compliance in reality.

The Core Insight: Dynamics Invariance

The central motivation stems from the analysis of interaction forces to identify components robust to the dynamics gap. The paper focuses on translational interaction forces and decomposes them into a Tangent Space (Ft) and a Normal Space (Fn). While these subspaces are correlated with dynamics, their directions—the tangent force direction 't' and normal force direction 'n'—are determined by task geometry, which is unrelated to the dynamics. Specifically, the tangent force direction 't' aligns with the feasible motion direction, and the normal force direction 'n' aligns with the surface normal vector at the contact point. The ground-truth force directions in simulation provide robust, dynamics-invariant supervision signals for policy learning.

Policy Formulation and Training

The proposed framework formulates a policy to predict not only reference poses but also the contact state and force direction alongside end-effector poses, acting as a Finite State Machine (FSM) to modulate control modes. The action space consists of three components: the Reference Pose and Gripper Command (Xd), the Normal Force Direction (nt), and the Contact State (ct). To train this policy, a Privileged State-Based Expert FSM is used in simulation to generate diverse demonstrations. In contact interaction phases, the expert FSM follows human-designed kinematic rules tailored to each task. The network architecture utilizes E2VLA, modified by extending the input and output projection layers to accommodate joint prediction of reference poses, normal force directions, and contact states within a unified generative framework. The training loss function balances objectives: L = λ1∥Xd − Xˆ d∥1 + λ2∥n − nˆ∥1 + λ3∥c − cˆ∥1.

Force-Aware Admittance Control

For real-world execution, the policy outputs configure a force-aware admittance controller that combines the predicted force direction with a manually specified target force magnitude. The controller uses translational dynamics: Mx¨r + Dx˙ r + K(xr − xcmd) = Fext − Fcmd. When contact is active (contact state c=1), the commanded force is set along the predicted normal direction 'n' as Fcmd = fn, where f = fH + n · K(xcmd − xr) + n · Dx˙ r. This drives the robot to steadily establish and maintain contact with the target force magnitude fH, which is a task-specific manually tuned scalar. For resistance overcoming tasks, tangent stiffening is enabled by scaling the translational stiffness along the tangent direction 't' by a factor (e.g., 4×).

Stability and Validation

Theoretical analysis confirms the robustness of the controller across different contact conditions. The paper presents three propositions characterizing stability:

1. Under disturbance-free conditions, the closed-loop normal-direction dynamics is asymptotically stable and converges to the equilibrium point xn = xe − fH/ke such that fext,n = fH.

2. When contact is lost due to an external disturbance, the closed-loop normal-direction dynamics are asymptotically stable in velocity, driving the robot toward the environment for re-establishment.

3. When a disturbance occurs during contact, the closed-loop normal-direction dynamics is input-to-state stable (ISS) with respect to the disturbance.

Experiments on four tasks—Microwave Opening, Peg-in-Hole, Whiteboard Wiping, and Door Opening—demonstrate superior performance compared to baselines. The method achieves a 91% overall success rate, significantly outperforming strong baselines in both success rate and robustness. Ablation studies confirm that the synergy of policy and controller is critical; removing either normal force regulation or tangent stiffening leads to substantial drops in success rates, validating the approach's effectiveness in managing task-relevant forces.

Conclusion

The framework successfully identifies force direction as a transferable signal that specifies task-relevant interaction intent and can be reliably learned in simulation, enabling adaptive compliance in the real world with minimal manual tuning. This approach overcomes the limitations of existing paradigms by leveraging pure simulation data for policy training while ensuring stable, task-aligned compliance through a carefully designed force-aware admittance controller. The results validate that predicting dynamics-invariant quantities allows for scalable sim-to-real transfer without relying on unreliable simulated force magnitudes.

Based on the provided scientific paper, here are specific improvements to existing AI systems and what those improved systems could achieve:

) Enhanced Sim-to-Real Transfer for Contact Dynamics:

By training policies exclusively in simulation using dynamics-invariant signals (force direction and contact state) instead of fragile force magnitudes, the system can achieve high performance on real-world contact tasks with minimal or no real-world data collection.

— Improved AI System Capability: Enables deployment of highly generalizable manipulation policies that are robust to the inherent discrepancies in physical contact dynamics between simulation and reality, drastically reducing the cost and safety risks associated with real-world data acquisition.

) Task-Aware Adaptive Compliance Control:

The system generates a force-aware admittance controller where the policy predicts both the necessary contact state and force direction, allowing for selective stiffness tuning (tangent stiffening vs. normal force regulation) based on task intent, rather than using blind, isotropic compliance.

— Improved AI System Capability: Enables robots to exhibit intelligent compliance—e.g., providing high resistance along the direction of a required pull (tangent stiffening during door opening) while maintaining necessary pressure for contact maintenance (normal force regulation during whiteboard wiping). This leads to superior task success rates and execution quality compared to baseline controllers that use fixed, isotropic stiffness settings.

) Robust Contact Maintenance Under Disturbance:

The controller is proven to be Input-to-State Stable (ISS), ensuring that the robot can maintain a target contact force magnitude during interaction even when subjected to external disturbances (e.g., surface displacement or external forces).

— Improved AI System Capability: Enables robots to reliably complete tasks in dynamic, uncertain real-world environments without triggering safety stops due to minor environmental perturbations, significantly enhancing operational robustness and reliability across all four tested contact-rich tasks.

) Fine-Grained Task Execution Quality Enhancement:

The combination of force direction prediction and adaptive compliance (normal force regulation + tangent stiffening) results in measurable improvements in execution quality metrics, such as deeper peg insertion and more thorough ink removal.

— Improved AI System Capability: Enables high-precision manipulation for tasks like peg-in-hole (deeper insertions) and whiteboard wiping (more complete ink removal), moving beyond simple geometric success to achieve higher fidelity interaction outcomes.

) Scalable Policy Training with Low Supervision Overhead:

The framework utilizes privileged supervision via a human-designed Finite State Machine (FSM) in simulation to generate diverse demonstrations, allowing the policy to learn complex state-switching behaviors without requiring explicit force profiles or stiffness mappings in the training data.

— Improved AI System Capability: Allows for rapid skill acquisition and generalization across different contact scenarios by leveraging structured expert knowledge (the FSM), enabling large-scale policy training using only simulation data, which is significantly more scalable and cost-effective than methods relying on expensive real-world force supervision.