Magnetic based In-situ Self 3D Pose Estimation for a Modular Soft Tendon-Driven Continuum Robot via IMU-Fusion

arXiv:2609.39950 · cs.RO, cs.SY, eess.SP, eess.SY · Submitted 2026-09-30 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa, and with me are Dev and Taro, guest researcher.

Dev: Today's paper: "Magnetic based In-situ Self 3D Pose Estimation for a Modular Soft Tendon-Driven Continuum Robot via IMU-Fusion".

Rosa: Continuum robots are well suited for gentle manipulation because of their inherent compliance and ability to adapt to complex environments,

Dev: First, who's behind it and why it matters.

Paper summary: Rosa: To get back to the start, this paper presents an embedded pose sensing framework specifically designed for modular soft tendon-driven continuum robots. The main thesis is that because these robots are compliant and can adapt to complex environments, they need a way to know their own configuration without external cameras, which is hard because of their continuously deformable structure.

Dev: The authors claim their solution involves combining inertial measurement units with active magnetic fields to estimate the robot's configuration. They argue that fusing these angular measurements helps improve local orientation estimation and reduces the error that builds up while operating.

Taro: What this means for autonomy is that we’re getting self-contained proprioception; the robot can sense itself without needing an external camera system to keep track of its shape. It addresses the problem where external sensing infrastructure or an unobstructed line of sight isn't available.

Rosa: They achieve a specific update rate of sixteen point seven Hz through this fusion, which enables real-time feedback control for the robot. This is what makes it useful for dynamic tasks where rapid adjustments are necessary.

Dev: From an engineering standpoint, the method relies on a modular architecture where each segment has its own IMU and magnetic source. They use the BNO086 IMU to track attitude during coil activation and then use magnetic measurements to correct the initial heading and subsequent drift.

Taro: The core mechanism involves a sophisticated data fusion scheme where they calculate quaternions and magnetic directions using specific rotation matrix operations. They then employ ambient subtraction and an ellipsoid fit to determine correction factors like the gain and soft-iron correction term Wi.

Rosa: After calculating those initial alignments, they fit the remaining alignment over a forty-five-second initialization window by minimizing the angle between their predicted parent axis and their calculated quaternion heading. This process helps establish a stable starting point for the robot's pose estimation.

Dev: They also track quaternion heading drift by estimating it using a per-segment rotation state bi and then updating it with an integration step involving sigma squared b t I. This is how they keep the orientation accurate between their main update cycles.

Taro: The resulting backbone reconstruction is done at that sixteen point seven Hz rate without integrating acceleration, using the formula p i = p i-one + L i. This method allows for distributed sensing and closed-loop control under external loading without needing to integrate acceleration data.

Rosa: Essentially, the paper proposes a system that uses this fusion to allow the robot to perform complex tasks while relying only on its internal sensors—IMUs and magnetic fields—to know where it is in space. It’s a self-contained sensing solution for gentle manipulation.

Dev: So, the key claims are the update rate of sixteen point seven Hz, the modular architecture integrating IMUs and magnetic fields, and this specific fusion scheme that allows for configuration updates without external cameras. That sets a high bar for embedded sensing on these kinds of robots.

Taro: I’m excited about the experimental validation mentioned, especially testing shape estimation under varied deformation and contact conditions. If that holds up in those messy scenarios, it opens the door for robots that can operate in much more complex physical settings than we currently envision.

Rosa: It really shows how embedding sensing directly into the compliant structure is a viable alternative to relying on external visual tracking for these kinds of tasks. This moves the capability from being an external dependency to an inherent property of the robot itself.

Dev: I'm still focused on the practicalities; how long can we expect this system to run reliably outside of a highly controlled lab environment before those magnetic field references or IMU readings degrade significantly?

Conclusion: Rosa: The paper, "Magnetic based In-situ Self three dee Pose Estimation for a Modular Soft Tendon-Driven Continuum Robot via IMU-Fusion," by Zheng Cao, Guo Ning (Andrew) Sue, Xiangyun Bu, David Quinn, Junzhe Hu, and Carmel Majidi, really highlights a path toward making continuum robots inherently aware of their own configuration through embedded sensors.

Dev: The implication is that we can design manipulation systems where the robot doesn't need a dedicated external vision system to know its pose, which is a huge step for deployment in obstructed or dynamic settings. It shifts the burden from external hardware to internal sensor fusion.

Taro: For autonomy researchers, this means we can build reactive systems that rely on accurate self-estimation in environments where visual tracking is unreliable or impossible, giving robots a more robust form of spatial awareness. Imagine navigating a complex industrial area without needing constant external cameras.

Rosa: It’s about achieving reliable, real-time feedback control by fusing inertial and magnetic data at a steady rate of sixteen point seven Hz, which is crucial for maintaining stability during manipulation tasks. This capability could be used in applications requiring gentle interaction with delicate objects.

Dev: The system’s success hinges on the robustness of that magnetic-inertial fusion scheme, especially how well it handles those varying deformation states and external disturbances during operation. That's where the long-term reliability question really comes into focus for control engineers.

Taro: I think the wider implication is that this research provides a foundation for truly autonomous manipulation, where self-knowledge is an intrinsic part of the robot's operation, not something tacked on with external sensors. It pushes us toward systems that can operate effectively in environments we currently deem too complex or dynamic for reliable visual tracking.

Rosa: We're seeing a strong move toward self-contained sensing architectures for soft robotics, where the robot learns and knows its own shape through its integrated components. This is a very practical direction for field robotics.

Dev: Ultimately, the paper demonstrates a sensing and communication scheme that provides configuration updates at sixteen point seven Hz using only internal sensors, which is a major win for real-time feedback loops.

Taro: This work lays groundwork for next-generation robots that can operate reliably in unstructured physical spaces by providing them with high-frequency, self-generated pose information.

Rosa: It’s a testament to how specialized sensor fusion can enable complex tasks on flexible platforms when external sensing is impractical or impossible.

Zheng Cao, Guo Ning (Andrew) Sue, Xiangyun Bu, David Quinn, Junzhe Hu, Carmel Majidi

Carnegie Mellon University

cs.RO, cs.SY, eess.SP, eess.SY

Submitted: 2026-09-30

Updated: 2026-09-30

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

Importance score: 66/100

The gist: Continuum robots are well suited for gentle manipulation because of their inherent compliance and ability to adapt to complex environments, but their continuously deformable structure makes accurate

Key concepts

Continuum Robots: These are flexible robots made of soft materials, like tendons, that can bend and deform easily. They are good for gentle tasks but difficult to track precisely because their structure constantly changes shape.
IMU-Fusion: This technique combines data from an Inertial Measurement Unit (IMU) and magnetic sensors. The IMU tracks movement using accelerometers and gyroscopes, while the magnetic sensors correct for orientation drift, resulting in a more stable and accurate pose estimate.
Magnetic Field References: The robot uses embedded coils that generate magnetic fields. These fields are used as external references to help determine the robot's orientation. This allows the system to function even when external vision systems are blocked or unavailable.
Backbone Reconstruction: This is the process of building a 3D model of the robot's structure over time. By using sequential pose updates from multiple segments, the system reconstructs how each part of the robot is positioned in space without needing acceleration data.

Terminology

Alignment — 12× in this paper · explained in 1 episode
Closed-Loop Control — 10× in this paper · explained in 2 episodes
Accuracy — 7× in this paper · explained in 1 episode
Calibration — 5× in this paper · explained in 7 episodes
Hysteresis — 5× in this paper · explained in 3 episodes
Trace — 4× in this paper · explained in 2 episodes
Angular Error — 3× in this paper · explained in 1 episode
Modular Architecture — 2× in this paper · explained in 4 episodes
Benchmark — 1× in this paper · explained in 2 episodes
Consistency — 1× in this paper · explained in 3 episodes
Core — 1× in this paper · explained in 2 episodes
Data Fusion — 1× in this paper · explained in 3 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 5 episodes
Modularity — 1× in this paper · explained in 9 episodes
Robustness — 1× in this paper · explained in 5 episodes

Summary

Continuum robots are well suited for gentle manipulation because of their inherent compliance and ability to adapt to complex environments, but their continuously deformable structure makes accurate configuration estimation challenging, particularly when external vision systems are unavailable or obstructed. This work presents an embedded pose sensing framework that combines inertial measurement units (IMUs) and active magnetic fields to estimate the robot configuration without relying on external cameras.

The gist

The proposed system achieves an update rate of 16.7 Hz by fusing angular measurements from the IMU and magnetic-field references to improve local orientation estimation and reduce accumulated orientation error during operation, enabling real-time feedback control.

Design and Sensing Architecture

The robot comprises nine serial TPU 90A modules, each equipped with a PCB integrating a BNO086 IMU, an ATtiny3226 microcontroller, a planar coil (magnetic source), and a switching MOSFET. Each module reads its IMU over SPI and exposes registers on a shared I2C bus. The BNO086 Game Rotation Vector fuses accelerometer and gyroscope measurements without magnetic input to track attitude during coil activation, while magnetic measurements correct its arbitrary initial heading and subsequent drift. The wound coil is replaced by additively wound traces on an FR4 PCB, designed with an asymmetric outline to provide more features in the magnetic field for processing downstream.

Data Acquisition and Pose Reconstruction

The system operates through interleaved continuous attitude polling and intermittent magnetic bursts, achieving an attitude update rate of 16.7 Hz. A ten-segment sweep takes approximately 52 ms, resulting in the achieved update rate. At each magnetic update, a fusion scheme is employed:

Quaternion and magnetic directions are calculated as:

**/u q i = R i e (2ee T - I)f(A i W I ∆B i), where Me converts the predicted parent axis in the child frame into the child direction in the parent frame. 2) Ambient subtraction removes constant offsets, and an ellipsoid fit determines gain and soft-iron correction Wi. 3) The remaining alignment Ai is fitted over a 45 s initialization window by minimizing the angle between Me f(AiWi∆Bi) and R T i u q i. 4) Quaternion heading drift is estimated using a per-segment rotation state bi: uˆi = exp([bi]×)u q i, Pi ← Pi + σ squared b ∆t I. 5) The backbone is reconstructed at the attitude update rate without integrating acceleration: p0 = 0, pi = pi−1 + Luˆi, i = 1,...,N. This process allows for distributed sensing and closed-loop control under external loading without external visual tracking during deployment. The total cycle time for a ten-segment sweep is about 52 ms. Every Tpair = 300 ms, one coil fires for 40 ms while the master polls only the other segments’ magnetometer blocks, taking approximately 3 ms per round. Samples are accepted only after settling and a bitwise change indicating a new conversion. Each segment receives a parentcoil correction every NTpair ≈ 3 s. The resulting attitude update rate is achieved at 16.7 Hz. The backbone is reconstructed at the attitude update rate without integrating acceleration: p0 = 0, pi = pi−1 + Luˆi, i = 1,...,N (Equation 5). This reconstruction avoids integrating acceleration and supports distributed sensing and closed-loop control under external loading without external visual tracking during deployment. The architecture supports distributed sensing and closed-loop control under external loading without external visual tracking during deployment. The main contributions are: 1) A modular magnetic–inertial sensing architecture for embedded three-dimensional shape reconstruction. 2) A sensing and communication scheme providing configuration updates at 16.7 Hz. 3) Experimental validation of shape estimation under varied deformation and contact conditions, and closedloop tip-height control under an external disturbance. The architecture supports distributed sensing and closed-loop control under external loading without external visual tracking during deployment. The main contributions are: 1) A modular magnetic–inertial sensing architecture for embedded three-dimensional shape reconstruction. 2) A sensing and communication scheme providing configuration updates at 16.7 Hz. 3) Experimental validation of shape estimation under varied deformation and contact conditions, and closedloop tip-height control under an external disturbance. The architecture supports distributed sensing and closed-loop control under external loading without external visual tracking during deployment. The main contributions are: 1) A modular magnetic–inertial sensing architecture for embedded three-dimensional shape reconstruction. 2) A sensing and communication scheme providing configuration updates at 16.7 Hz. 3) Experimental validation of shape estimation under varied deformation and contact conditions, and closedloop tip-height control under an external disturbance.

Improvements for AI systems

Here are specific improvements for AI systems derived from this scientific paper:

Enhance real-time, robust pose estimation for continuum robots in visually obstructed environments by integrating a modular magnetic-inertial sensing architecture. This system will enable autonomous operation and closed-loop control of soft robots without reliance on external cameras or precise kinematic models, achieving an update rate of 16.7 Hz.
Develop a novel fusion algorithm (like the one described in Section III-C) that combines local orientation estimates from multiple robot segments with magnetic field measurements to reconstruct the full three-dimensional configuration of a modular soft continuum robot in real-time. This will allow the AI to maintain precise end-effector positioning during interaction with objects, even under external loading or occlusion.
Implement a learning-based mapping (MLP) that maps raw magnetic field readings to unit vectors, incorporating input normalization to mitigate amplitude variations. This model will be trained on diverse poses sampled across the reachable workspace (as described in Section III-B) to provide robust directional information for the pose estimation pipeline.
Create a predictive drift correction module that utilizes per-segment rotation states and magnetic updates (as shown in Equation 3 and 4) to estimate and compensate for gyroscope heading drift, reducing orientation error during long operation or static holds.
Design a closed-loop control system capable of maintaining precise tip-height control based on the estimated backbone shape, as derived from the fused pose estimation. This system will allow the robot to accurately track a desired vertical position while simultaneously managing dynamic disturbances (like applying a 500g load) by adjusting tendon spool velocity in real-time.
Improve long-term state estimation capabilities by developing methods to differentiate between sensor drift, creep, and actual motion repetition during cyclic actuation or static holds. The AI system will use the fused estimate to track these subtle deviations more effectively over extended periods without ground truth, providing more reliable proprioceptive feedback for long-duration tasks.
Develop a generalized model-based approach that can handle material nonlinearities and hysteresis in continuum robots by incorporating learned corrections derived from the magnetic/inertial fusion, moving beyond purely kinematic models to improve accuracy under complex deformation conditions (as suggested by the limitations discussed in Section II).

Abstract

Continuum robots are well suited for gentle manipulation because of their inherent compliance and ability to adapt to complex environments. However, their continuously deformable structure makes accurate configuration estimation challenging, particularly when external vision systems are unavailable or obstructed. In this work, we present an embedded pose sensing framework that combines inertial measurement units (IMUs) and active magnetic fields to estimate the robot configuration without relying on external cameras. The angular measurements from the IMU and magnetic-field references are fused to improve local orientation estimation and reduce accumulated orientation error during operation. This pose sensing scheme achieves an update rate of 16.7 Hz, allowing real-time feedback. The proposed system is experimentally validated through closed-loop control, where the estimated robot configuration is used to maintain the end-effector at a desired position while interacting with an object. These results demonstrate the potential of distributed magnetic--inertial sensing for real-time pose estimation and closed-loop control of continuum robots.