Bridging the Sim-to-Real Gap with multipanda ros2: A Real-Time ROS2 Framework for Multimanual Systems

arXiv:2602.02269 · cs.RO, cs.AI, cs.SE, cs.SY, eess.SY · Submitted 2026-02-02 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa, and with me are Dev and Taro, guest researcher.

Dev: Today's paper: "Bridging the Sim-to-Real Gap with multipanda ros2".

Rosa: Multipanda ros2 presents a novel, open-source ROS2 architecture designed for real-time multi-robot control of Franka Robotics robots, addressing critical challenges in torque control, interaction control, and robot-environment modeling.

Dev: First, who's behind it and why it matters.

Title and authors: Rosa: Moving on to the title and authors of "Bridging the Sim-to-Real Gap with multipanda ros2: A Real-Time ROS2 Framework for Multimanual Systems," we see it's written by Jon Skerlj, Seongjin Bien, Abdeldjallil Naceri, and Sami Haddadin.

Dev: Those authors bring a good mix of expertise here; you have the control engineering background from Dev and the autonomy perspective from Taro involved in this work.

Taro: I agree, having researchers who understand both the control loop specifics and how those systems need to interact with an autonomous agent is really valuable for this kind of paper.

Rosa: The title itself immediately tells us the main goal is closing that gap between simulation and reality using a specific ROS2 framework for multi-manipulator setups.

Dev: It points toward a practical application, focusing on a ROS2 architecture because it leverages existing infrastructure instead of trying to build everything from scratch for this control problem.

Taro: That practical focus is what makes me interested; if the implementation is solid and reproducible, it could become a really strong foundation for developing more complex multi-agent behaviors in robotics.

Rosa: The implication here is that we can start deploying sophisticated multi-robot tasks sooner because we have a framework that handles the low-level control requirements reliably across simulation and hardware.

Dev: That capability to deploy faster depends entirely on how well it maintains those 1kHz requirements under real operating conditions, which is something I'll be looking closely at.

Taro: And for autonomy, this means we can train agents in environments that accurately reflect the dynamics of Franka robots without needing perfect real-world hardware for every single step.

Rosa: So, essentially, it’s about providing a structured path to move complex multi-robot control from theoretical models into reliable physical execution.

Dev: That's a fair summary; it provides the necessary tools and structure to handle the complexity of multi-arm systems in a way that respects real-time constraints.

Taro: And I think that structured approach is exactly what we need when we start looking at more advanced social navigation or complex manipulation tasks where coordination is key.

The paper's summary: Rosa: Now, let's talk about the actual summary of "Bridging the Sim-to-Real Gap with multipanda ros2: A Real-Time ROS2 Framework for Multimanual Systems." It explains that they introduce multipanda ros2 as a novel open-source architecture for controlling Franka Robotics robots in a multi-robot setting using a single process.

Dev: The summary emphasizes that the core contributions are tackling key challenges in real-time torque control, focusing heavily on interaction control and robot-environment modeling.

Taro: I see that they are not just looking at basic joint movements; they are explicitly addressing the messy parts of physical contact and how robots need to model their surroundings during those interactions.

Rosa: They also highlight the introduction of a controllet-feature design pattern, which lets them achieve controller switching delays of less than two milliseconds for benchmarking purposes.

Dev: That low switching delay is a major engineering win because it directly addresses the latency issue in the control loop, ensuring that when we switch controllers to handle different robot assignments, the disruption is minimal.

Taro: Minimal disruption is critical; if the switch takes too long, you risk losing synchronization between two robots that are supposed to be working together on a shared task.

Rosa: Furthermore, they explain how integrating MuJoCo simulations with quantitative metrics for kinematic accuracy and dynamic consistency—like torques and forces—helps them validate their system against physical reality.

Dev: Those specific metrics give us the data we need to confirm if the simulation is just kinematically correct or if it’s actually modeling the physics of forces and torques correctly.

Taro: It gives us a concrete way to check for dynamic consistency, which is where most sim2real problems usually manifest themselves when you move beyond simple point tracking.

Rosa: And finally, they show that by identifying real-world inertial parameters can significantly improve force and torque accuracy through iterative physics refinement.

Dev: That feedback loop of using real data to update the model is a powerful mechanism for improving accuracy, moving us closer to a truly reliable simulation environment for complex dynamic tasks.

Taro: That iterative cycle is what we need; it moves us away from just accepting simulation results and towards building models that are physically grounded, which is essential for any serious autonomy work.

The paper's improvements: Rosa: When we look at the specific improvements this paper suggests, they focus on extending soft-robotics approaches to rigid dual-arm, contact-rich tasks to assess force fidelity.

Dev: That extension is interesting because it shows they’re applying concepts from softer robotics into a more rigid, contact-rich scenario, which is a challenging area for control design.

Taro: I'm curious about the implications of that; it suggests that the principles used to manage compliant interaction can be successfully adapted for more rigid manipulation tasks involving dual arms.

Rosa: They also propose incorporating GJK-based self-collision avoidance and manipulability-based singularity avoidance into the final command torque calculation, which adds extra layers of safety.

Dev: Those additions are necessary; incorporating collision and singularity checks directly into the torque calculation means the system is designed to actively avoid states that lead to physical failure or unpredictable motion.

Taro: That active avoidance mechanism is exactly what we want for autonomous systems; having built-in safeguards against self-collision or reaching singular configurations prevents catastrophic failures when things go wrong.

Rosa: And they suggest using real-world inertial parameter identification as a method for iterative physics refinement to improve force and torque accuracy, which we touched on before but it's presented as a key improvement.

Dev: So the paper is not just describing a concept; it’s providing an actionable methodology for improving the fidelity of the control system through practical data-driven updates.

Taro: That methodology is what elevates this from just a theoretical idea to something that can actually be implemented and used in a real-world autonomous system.

Conclusion: Rosa: To wrap up, the conclusion really summarizes that multipanda ros2 provides a robust, reproducible platform for advanced robotics research by offering low-latency control architecture and comprehensive sim2real validation pipeline.

Dev: It essentially confirms that it successfully extends existing approaches to rigid dual-arm tasks and shows that incorporating physical system data into simulation models is an effective strategy for improving force and torque accuracy.

Taro: I think the most significant implication is that this framework offers a way to get high-fidelity, physics-aware simulation environments for AI policy training that are much more reliable than what we had before.

Rosa: It sets a solid foundation for deploying multi-robot systems because it gives us the tools to ensure safety and precision in contact-rich tasks with guaranteed high frequency performance.

Dev: And from an engineering standpoint, the 1kHz target frequency is achievable, with average command torque delays around zero point five milliseconds during experiments.

Taro: I'm just excited to see how this framework evolves as it moves from controlled benchmarks into more unpredictable, real-world situations where true autonomy is tested.

Rosa: It sounds like a really promising piece of work that gives us a solid tool to push the limits on what multi-arm systems can accomplish in practice.

Dev: Indeed, it’s a solid step forward in providing the tools for high-frequency control loops that meet those demanding real-time demands.

Taro: This paper, "Bridging the Sim-to-Real Gap with multipanda ros2: A Real-Time ROS2 Framework for Multimanual Systems," gives us a platform that can help build smarter, more reliable robotic agents for complex tasks.

Munich Institute of Robotics and Machine Intelligence (MIRMI) · Technische Universität München (TUM) · Technische Universität Nürnberg (UTN) · Mohamed bin Zayed University of Artificial Intelligence

cs.RO, cs.AI, cs.SE, cs.SY, eess.SY

Submitted: 2026-02-02

Updated: 2026-10-01

Comments: Published at IEEE ICRA 2026. Source code available at https://github.com/tenfoldpaper/multipanda_ros2

Journal ref: 2026 IEEE International Conference on Robotics and Automation (ICRA), pp. 9679-9686

DOI: 10.1109/ICRA57385.2026.11696149

Code: https://github.com/tenfoldpaper/multipanda_ros2

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 85/100

The gist: Multipanda ros2 presents a novel, open-source ROS2 architecture designed for real-time multi-robot control of Franka Robotics robots, addressing critical challenges in torque control, interaction

Key concepts

Controllet: A controllet is a component within the Multipanda ros2 framework that takes temporary ownership of one or more robots. It is responsible for calculating and executing the specific command torques for those assigned robots during a control cycle. This design allows the system to switch between different robot controllers efficiently without disrupting the overall real-time loop.
1kHz Control Frequency: This refers to running the robot control loop one thousand times per second. Achieving this high frequency is crucial for safety standards and ensuring that the robots react quickly enough to dynamic physical interactions. The framework is engineered specifically to sustain this speed, which is necessary for reliable real-time operation.
Sim-to-Real Bridging: This process involves using a high-fidelity simulation environment, like MuJoCo, and comparing its results against actual physical robot experiments. Multipanda ros2 helps bridge this gap by integrating the simulation with quantitative metrics and allowing researchers to use real-world data to refine the simulated robot's physics parameters.
Iterative Physics Refinement: This is a methodology where real-world measurements of a robot's performance (like actual forces and torques) are used to update the parameters within the simulation model. By continuously updating these virtual parameters based on physical data, researchers can significantly reduce the discrepancies between what happens in simulation and what actually occurs in reality.

Terminology

Accuracy — 19× in this paper · explained in 1 episode
Fidelity — 19× in this paper · explained in 2 episodes
Sim-to-Real Gap — 11× in this paper · explained in 6 episodes
Consistency — 10× in this paper · explained in 3 episodes
Benchmarking — 8× in this paper · explained in 3 episodes
Core — 8× in this paper · explained in 2 episodes
Franka Robotics Robot — 7× in this paper
1kHz Control Frequency — 4× in this paper · explained in 1 episode
Unified Force-Impedance Control — 3× in this paper
Benchmark — 2× in this paper · explained in 2 episodes
International Conference on Robotics and Automation — 2× in this paper
Reinforcement Learning — 2× in this paper · explained in 4 episodes
Robot Operating System — 2× in this paper
Adaptation — 1× in this paper · explained in 1 episode
Admittance Control — 1× in this paper · explained in 1 episode
Computational overhead — 1× in this paper · explained in 2 episodes
Deep Reinforcement Learning — 1× in this paper · explained in 2 episodes
Domain Adaptation — 1× in this paper · explained in 5 episodes
Formal Methods — 1× in this paper · explained in 3 episodes
Interpretability — 1× in this paper · explained in 15 episodes
Robustness — 1× in this paper · explained in 5 episodes
Sim-to-Real Transfer — 1× in this paper · explained in 2 episodes
Transfer Learning — 1× in this paper · explained in 10 episodes
Verification — 1× in this paper · explained in 2 episodes

Summary

Multipanda ros2 presents a novel, open-source ROS2 architecture designed for real-time multi-robot control of Franka Robotics robots, addressing critical challenges in torque control, interaction control, and robot-environment modeling. This framework is significant because it provides native ROS2 interfaces for controlling multiple robots from a single process while sustaining a 1kHz control frequency necessary for safety standards and real-time dynamics.

Core Contributions

The paper outlines several key contributions to the field of multi-robot manipulation:

Reviewing real-time torque control for single- and dual-arm robots, focusing on interaction control and robot–environment modeling challenges.
Introducing an open-source ROS2 multirobot framework for Franka Robotics robots with ≤ 2 ms controlletswitching delays to facilitate reproducible benchmarking.
Integrating MuJoCo simulations with quantitative metrics for both kinematic accuracy and dynamic consistency (torques, forces, and control errors).
Assessing force-based fidelity by extending soft-robotics approaches [14] to rigid dual-arm, contact-rich tasks.
Demonstrating that real-world inertial parameter identification can significantly improve force and torque accuracy, providing a methodology for iterative physics refinement.

Framework Architecture and Control Mechanism

The multipanda ros2 framework is built upon the ROS2 control infrastructure and utilizes a multimode controller (MC) as its central component. The MC does not implement a controller itself but instead uses controllets. Each controllet takes ownership of one or more robots at runtime and implements the actual command torque calculation for the loop. The design emphasizes minimal overhead, using move operators and references to avoid unnecessary copying. The MC manages robot assignment for all controllets, ensuring that only one controllet has ownership of a robot at any given time.

Real-Time Performance Features

The framework is engineered to meet stringent real-time requirements. A central focus is sustaining a 1kHz control frequency, which is necessary for both real-time control and safety standards. To enable reproducible benchmarking and complex interactions, the framework introduces a controllet-feature design pattern that enables controller-switching delays of ≤ 2 ms. This switching function is exposed as a ROS 2 service, allowing the MC to change the active controllet at runtime without conflicting resource requirements.

Simulation and Sim-to-Real Bridging

To bridge the simulation-to-reality (sim2real) gap, the framework integrates a high-fidelity MuJoCo simulation with quantitative metrics for both kinematic accuracy and dynamic consistency. The simulation is implemented as a plugin to wrap the MuJoCo physics engine into a ROS2 package, ensuring consistency between simulated and physical experiments. The evaluation methodology employs several key metrics:

**: q (joint positions), q˙ (joint velocities), τ (joint torques), xEE (end-effector position), FEE (end-effector force), and Cerr (control error). **

Furthermore, the paper demonstrates that real-world inertial parameter identification can significantly improve force and torque accuracy, offering a methodology for iterative physics refinement by updating simulated robot parameters based on real system data. This approach is shown to reduce discrepancies between simulation and reality, particularly in contact-rich tasks.

Controller Implementation and Validation

The framework evaluates performance using the Unified Force-Impedance Control (UFIC) framework and a classical Cartesian impedance controller as a baseline for both simulation and real-world experiments. The UFIC extends standard impedance control with force control, utilizing modulated tank signals to achieve desired force tracking. Dual-arm coordination features include GJK-based self-collision avoidance and manipulability-based singularity avoidance, which are incorporated into the final command torque calculation:

**: τcmd = τtask + τnull + τcor + τca + τma **

Experimental results confirm that the framework sustains the 1kHz target frequency, with average command torque delays around 0.5 ms. The controllet switching experiment yielded an average delay of approximately 2.117ms, indicating about two control cycles before a new controller is activated. Simulation validation shows kinematic fidelity discrepancies up to 0.06 rad for joint positions and 1.3 cm for end-effector position, while dynamic errors are quantified in terms of torques and forces. The integration of real-world identification procedures further reduces these simulation errors, with recorded external force error being reduced by nearly half in the updated simulation parameters.

Conclusion

Multipanda ros2 provides a robust, reproducible platform for advanced robotics research by offering a low-latency control architecture and a comprehensive sim2real validation pipeline. It successfully extends existing approaches to rigid dual-arm tasks and demonstrates that incorporating physical system data into simulation models is an effective strategy for improving force and torque accuracy. Future work plans include exploring more complex validation scenarios and expanding the open-source framework for wider adoption.

Improvements for AI systems

As a fastidious research AI, I have analyzed multipanda ros2 and identified several high-impact areas where this framework can significantly advance AI systems, particularly in robotics, simulation-to-reality (sim2real) transfer, and complex physical interaction tasks.

Here are the specific improvements and the capabilities of the resulting improved AI systems:

The proposed improvements focus on enhancing robustness, fidelity, and adaptability in robotic control loops by leveraging the multipanda ros2 framework's core strengths: high-frequency real-time capability (1kHz), low-latency controller switching (≤2ms), MuJoCo integration, and iterative physics refinement via inertial parameter identification.

Here are the specific improvements that can be made to AI systems:

A robust, reproducible control architecture capable of executing complex, multi-robot manipulation tasks with high precision across simulation and real hardware.
A closed-loop system for adaptive physical interaction, where the robot's dynamic model is continuously refined by real-world sensor data during operation to minimize simulation errors.
A modular framework enabling the rapid deployment and benchmarking of diverse control strategies (e.g., impedance vs. force control) across different robot platforms (single-arm vs. dual-arm).
An AI policy training pipeline that utilizes high-fidelity, physics-aware simulation environments to generate robust policies before deployment on physical hardware, significantly reducing the sim2real gap in contact-rich scenarios.

These improvements enable the improved AI systems to perform the following specific tasks:

An improved system can execute complex, coordinated maneuvers involving two or more Franka robots (or similar multi-arm systems) in a single process, such as synchronized grasping of an object from opposite sides while maintaining precise contact forces (as demonstrated in Task 5).
It can perform high-frequency physical human-robot interaction (pHRI), such as delicate assembly operations or surgical procedures, by maintaining a guaranteed 1kHz control loop frequency and low command latency, ensuring safety standards are met.
The system can adapt its control strategy on the fly: it can seamlessly switch between different modes—from pure position/velocity tracking to compliant force-based interaction (impedance/admittance) or singularity avoidance—with minimal delay (≤2ms), allowing the robot to rapidly transition from a free-space movement task to a contact-rich manipulation task.
It can operate in an iterative refinement cycle: it can run simulation experiments, identify discrepancies between simulated and real torque/force measurements, use the identified real-world inertial parameters to update the MuJoCo model, and then re-run the experiment with a significantly reduced sim2real error (as shown in Fig. 6).
The AI policy training process will be more reliable because it is trained in a simulator that accurately models not just kinematics, but also dynamic consistency (torques and forces), enabling the policy to generalize better when deployed on the real robot, effectively bridging the gap between simulated torque calculations and physical reality.