Deformable In-Hand Slip-Aware Tactile Sensor with Integrated Velocity Sensing, Force/Torque and Pressure Map Estimation

arXiv:2606.11952 · cs.RO · Submitted 2026-06-10 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "Deformable In-Hand Slip-Aware Tactile Sensor with Integrated Velocity Sensing, Force/Torque and Pressure Map Estimation".

Dev: This paper introduces a novel tactile sensor that integrates velocity, force/torque,

Rosa: First, who's behind it and why it matters.

Title and authors: Rosa: Well, Dev, Taro, this paper introduces something really interesting called "Deformable In-Hand Slip-Aware Tactile Sensor with Integrated Velocity Sensing, Force/Torque and Pressure Map Estimation." It seems like the main idea here is putting velocity sensing, force/torque measurement, and pressure mapping all into one single device using a flexible contact pad.

Dev: That sounds ambitious for a single unit, Rosa; integrating those three modalities into one compliant structure is certainly a significant engineering challenge that needs careful handling of latency and loop rates. What catches my eye immediately is the claim that this is the first sensor to combine these sensing modalities within a single compliant structure, which suggests they’ve overcome some major integration hurdles.

Taro: I'm excited because combining those different data streams into one physical platform opens up new possibilities for autonomy; if we can get robust, real-time information on how an object is slipping while simultaneously knowing the forces and pressure distribution, that changes how we think about in-hand manipulation entirely.

Rosa: Exactly! So, what's the core of this sensor? The paper explains that it uses a deformable contact pad made of TPU 85A, and it leverages Hall-effect sensors to reconstruct the pressure map while optical mouse sensors embedded in the pad measure planar sliding velocity.

Dev: The methodology for mapping those raw measurements is where I need to focus; they use a neural network architecture with a shared encoder and three task-specific output heads. That setup means the system has to learn how to decode those different physical quantities from the input signals, which introduces its own set of potential failure modes we need to monitor closely.

Taro: I'm curious about what happens when things get messy in the real world; if the object is curved or made of a material we haven't seen before, how does this sensor handle that deviation from the ideal flat surface tracking? That’s where I want to push on what happens when things misbehave.

Rosa: The paper suggests that this integrated design allows it to robustly track both flat and curved surfaces across a wide range of diffuse material properties, which is a big deal for practical robotics outside of a perfectly controlled lab setting.

Dev: Tracking those surfaces is one thing, but let's talk about the performance metrics; the results show an NRMSE of approximately three to four percent while in contact for objects included in their training set, which gives us a concrete number to work with regarding accuracy.

Taro: A low NRMSE during contact is good, but I’m more interested in how this sensor behaves when the object starts moving unpredictably; what does it tell the system about the grasp state if things go wrong?

Rosa: The paper also points out that this setup helps in estimating relative sliding velocities, full six-DoF force/torque estimates, and spatial pressure information all at once, which directly supports a robust estimation of grasp state and external contacts.

Title and authors: Dev: From an engineering standpoint, the calibration process described—which involves performing a predetermined linear motion to estimate "Counts per mm" CPMM and then rotating the sensors to find their poses—that whole sequence has implications for system setup time and how quickly we can get operational.

Taro: If we think about future applications, this sensor could be huge for tasks involving delicate object placement or conforming grasping because it gives us localized pressure details that rigid sensors just can't provide.

Rosa: Right, so the authors suggest that the deformable pad itself enhances contact dynamics and increases contact torque authority compared to rigid designs, which is a key reason they built it this way over older methods.

Dev: I see why they focused on enhancing those dynamics; if the pad can handle the deformation, it means we might be able to achieve better control authority in complex interactions where a rigid sensor would just report a hard stop.

Taro: Thinking about the implications for the wider field, if this technique proves reliable across diverse materials, it could mean that robotic systems can operate much more reliably in environments with unknown or highly variable object properties.

Rosa: That’s what I was thinking; it moves us closer to scenarios where we aren't just interacting with known geometry but truly understanding the physical state of whatever we are holding in real time.

Dev: But we have to consider the limitations they acknowledge; specifically, they mention that the viscoelastic behavior of the contact pad can lead to drift during prolonged contact because of slow deformation, which is a failure mode we need to account for in long-duration tasks.

Taro: That's a fair caveat; if drift is significant over time, an autonomous system would need a way to recalibrate or fuse that tactile data with other sensory inputs to maintain accuracy over extended manipulation sessions.

Rosa: So, while the sensor is powerful for real-time feedback, we still have to design control loops that can manage that slow deformation effect when the task demands sustained contact.

Dev: And regarding external influences, they address magnetic disturbances by including a third output head in their neural network architecture specifically to predict the three-axis magnetic signal of the 13th Hall-effect sensor, which is smart because it accounts for environmental noise.

Taro: That compensation mechanism is crucial for real deployment; if we deploy this on a robot near strong magnetic fields, that feature ensures we don't lose our force readings due to external interference.

Rosa: So, the overall picture of the Deformable In-Hand Slip-Aware Tactile Sensor with Integrated Velocity Sensing, Force/Torque and Pressure Map Estimation is a system that successfully fuses multiple physical measurements into one structure for slip-aware control.

Dev: Indeed; it combines optical measurement for velocity, Hall-effect mapping for F/T and pressure, all within a compliant pad design to improve contact dynamics.

Taro: It suggests a path toward more intuitive human-like manipulation because the robot isn't just reacting to forces but is also sensing the subtle surface geometry through that pressure map reconstruction.

Title and authors: Rosa: It definitely gives us a much richer picture of what's happening at the contact interface than we get from just force readings or just simple slip detection.

Dev: Before we move on to how this translates into practical deployment, I want to quickly touch on the limitations they flagged; they noted that the resolution of the pressure map is limited by those embedded magnets, which means reconstructing sharp edges or fine details isn't achievable with that specific setup.

Taro: That makes sense; it's a trade-off between capturing broad contact states and achieving high spatial resolution, so we have to decide what level of detail is actually necessary for the task at hand.

Rosa: So, we’ve covered the core concept, the mapping methodology, and those specific limitations regarding resolution and drift in the Deformable In-Hand Slip-Aware Tactile Sensor with Integrated Velocity Sensing, Force/Torque and Pressure Map Estimation.

Dev: That brings us to where this research sits right now; it’s a highly capable sensor for complex manipulation tasks where slip awareness is paramount, even though we have to manage the inherent drift in the pad material.

Taro: And looking ahead, I think the real impact will come when we start fusing this type of tactile data with vision and kinematic information to build truly robust perception systems for unknown environments.

Rosa: That’s a great direction for future work; leveraging these modalities together could lead to a system that can adapt its contact strategy based on everything it senses simultaneously.

Dev: I agree; the potential for slip-aware control systems that dynamically adjust grip force based on integrated velocity sensing is where the most immediate practical gains will show up in manipulation performance.

Taro: If we can build those slip-aware controllers effectively, we could see robots performing much more delicate tasks than what's currently possible with standard rigid sensors.

Rosa: So, to wrap up our discussion on this paper, the Deformable In-Hand Slip-Aware Tactile Sensor with Integrated Velocity Sensing, Force/Torque and Pressure Map Estimation is a significant step in integrating multiple sensing modalities into a compliant structure for grasping.

Dev: It’s an important piece of hardware because it provides relative sliding velocities alongside full six-DoF F/T and pressure distribution estimates within one unit, which supports grasp state estimation.

Taro: The implications are that we gain a much more comprehensive understanding of contact dynamics, moving beyond simple force thresholds to understanding the actual interaction with the object's surface geometry.

Rosa: It really shows how combining different sensing modalities can create a sensor capable of handling both flat and curved surfaces reliably for in-hand manipulation.

Dev: We have to keep an eye on that viscoelastic drift during long operations, though; that's the main operational hurdle we need to solve in the next iteration of this design.

Taro: And I think the future lies in using this data not just for sensing, but for advanced planning that can anticipate slip before it even happens by predicting contact states based on all these integrated measurements.

The paper's summary: Rosa: So, to recap, this paper introduces a novel tactile sensor that manages to put velocity tracking, force/torque sensing, and pressure mapping all inside one physical device using a deformable contact pad for slip-aware control during in-hand manipulation.

Dev: That's the big headline; it’s about combining those three distinct sensing types into a single compliant structure, which is a tough engineering feat we need to really look at from a loop rate and latency standpoint.

Taro: And what I find particularly compelling is that this single integration allows the sensor to robustly track both flat and curved surfaces while still providing rich, dynamic information about the contact states.

Rosa: Exactly; it's not just measuring one thing anymore; it’s giving us a holistic view of the grasp—how hard we’re pushing, how fast things are sliding, and what the pressure distribution actually looks like on the object.

Dev: That holistic view is great for stability assessment, but Rosa, I have to ask about its practical deployment. Can we expect this sensor to hold up outside of a highly controlled lab setting? How long do you think we can rely on it before that viscoelastic behavior in the pad causes significant drift that messes up the readings?

Taro: That's a valid concern for real-world autonomy; if the pad deforms slowly and drifts over time, an autonomous system needs a way to either self-calibrate or fuse that data with other inputs to maintain accuracy during extended manipulation sessions.

Rosa: I agree, it's definitely a hurdle we have to overcome for true field deployment; but the results show that for objects they tested in their training set, they hit about a three to four percent error rate while actively in contact, which is pretty respectable for dynamic tasks.

Dev: Three to four percent NRMSE while in contact is solid, but I need more than just accuracy; I need to know how this system performs under stress. For example, what happens if we introduce unexpected external forces or magnetic interference?

Taro: That's where the authors did some clever work by including a mechanism specifically designed to compensate for external magnetic disturbances on the Hall-effect sensors, which should help keep the force and pressure estimates consistent regardless of where the robot is oriented.

Rosa: It sounds like this sensor is really positioned to handle complex, dynamic in-hand tasks where knowing exactly what's happening at the contact point—the velocity, the torque, and the shape of that pressure map—is essential for things like delicate object placement or navigating curved surfaces without dropping it.

Dev: I see how that rich data set could feed into slip-aware control; if we can reliably distinguish between rotational and linear slip modes using those integrated velocity measurements, we could dynamically adjust grip force in real time, which is a huge step up from traditional methods.

Taro: If this level of integrated perception becomes standard, it opens the door for autonomous systems to perform much more nuanced manipulation—not just gripping an object, but truly understanding its physical state during that interaction.

Rosa: That's what excites me most; imagine robots handling anything from soft materials to oddly shaped items with a level of dexterity that was previously impossible because they couldn't accurately "feel" the surface geometry in real-time.

Dev: I’m still focused on the implementation side, though; achieving this level of data fusion reliably within the required loop rate is going to be a significant engineering challenge for the control software we design around it.

Taro: And that challenge is exactly why we need researchers like us to push on how the system handles those unpredictable real-world misbehaviors, because that's where true autonomy lives.

The paper's improvements: Rosa: So, we've seen how this sensor works and its impressive ability to map velocity, force, and pressure together using that TPU pad for in-hand manipulation feedback.

Dev: Right; but the authors didn't just stop there; they laid out a roadmap for what needs to be improved to move this from a lab curiosity into something truly robust for the field.

Taro: I'm looking at their suggestions about leveraging these combined modalities for in-hand perception and contact estimation, which points toward building a system that can reason about object properties based on what the sensor is reporting.

Rosa: That makes sense; they want to use this rich data to build an AI system capable of inferring things like friction coefficients or even recognizing subtle surface details that a rigid sensor would completely miss.

Dev: From an engineering standpoint, I'm interested in their call for multimodal sensor fusion and fallback strategies, which suggests we shouldn't rely on just one sensing modality; if the pressure map reconstruction gets fuzzy due to noise, the system should have a backup plan based on the force or velocity data.

Taro: That aligns with my thoughts about handling world misbehavior; by planning for these fallbacks now, we make the autonomous system much more resilient when it encounters unexpected contact conditions in a cluttered or uncertain environment.

Rosa: It seems like their long-term vision is to create a compliant manipulation planner that can adapt its contact strategy on the fly based on real-time surface geometry sensed through that pressure map reconstruction capability.

Dev: That means the system isn't just reacting; it’s proactively adjusting how it touches the object to minimize slippage, which is exactly what we need for smoother, more dexterous motions in complex scenarios.

Taro: If we can get a planner that uses this information to anticipate slip before it happens by modeling contact states across different materials, that moves us closer to true intelligent interaction where the robot anticipates physics.

Rosa: That sounds like the kind of capability that could impact everything from delicate medical device handling to complex assembly tasks where precise force control is paramount.

Dev: While they’re working on those advanced manipulation plans, I’m still concerned about the inference time; they noted it runs at one millisecond on a single CPU core, so scaling this up for real-time robotic applications will require serious optimization of that neural network architecture.

Taro: That’s a fair point; computational efficiency is the next big hurdle for deploying these advanced perception systems in fast-paced robotic tasks.

Rosa: So, we have this incredibly powerful sensor design, clear pathways to improve its robustness through fusion and planning, but we still have to solve those practical problems of drift and computational load before widespread field use is feasible.

Conclusion: Rosa: So, to wrap up our discussion on the Deformable In-Hand Slip-Aware Tactile Sensor with Integrated Velocity Sensing, Force/Torque and Pressure Map Estimation, we've seen how this system successfully integrates velocity tracking with force and pressure mapping into one compliant unit for slip-aware control.

Dev: It really is a significant piece of hardware because it provides relative sliding velocities alongside full six-DoF F/T and pressure distribution estimates within a single unit, which directly supports grasp state estimation.

Taro: I think the main implication here is that we’re moving toward perception where robots don't just react to force thresholds but actually understand the physical state of the contact interface during manipulation.

Rosa: Exactly; this sensor opens up possibilities for more delicate and dexterous in-hand tasks, especially when dealing with objects whose shapes or materials are hard to model precisely.

Dev: We have to keep an eye on that viscoelastic drift during long operations, though; that's the main operational hurdle we need to solve in the next iteration of this design before we can trust it for long-duration missions.

Rosa: That’s a great direction for future work; leveraging these modalities together could lead to a system that adapts its contact strategy based on everything it senses simultaneously.

Dev: I agree, the computational efficiency will be key to making those advanced planning strategies practical for real-time control loops.

Taro: We need robust methods to handle that trade-off between high-fidelity perception and low inference time if we’re going to get this into a production robot quickly.

Department of Electrical Engineering, Chalmers University of Technology · Department of Automatic Control, Lund University

cs.RO

Submitted: 2026-06-10

Updated: 2026-10-01

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 78/100

The gist: This paper introduces a novel tactile sensor that integrates velocity, force/torque, and pressure map sensing into a single device with a deformable contact pad to enable slip-aware control for

Key concepts

Deformable Contact Pad: The physical interface of the sensor is a 3D-printed TPU 85A pad with a gyroid minimal-surface structure. This material deforms under contact, allowing it to measure sliding velocity and map pressure distribution across the surface. Its compliance helps in tracking both flat and curved surfaces.
Hall-effect Sensors (MLX90393): These sensors are used to measure magnetic fields generated by the sensor's internal structure. They are mapped via a neural network to estimate the 6-DoF Force/Torque values and help compensate for external magnetic disturbances, providing crucial force information.
Optical Mouse Sensor Units (PAW3335DB): These sensors are embedded in the pad to measure x- and y-displacement. By tracking their movement, the system can compute planar sliding velocities, which is essential for slip-aware control during in-hand manipulation tasks.
Multimodal Sensor Fusion: This technique integrates data from different sensing methods—velocity from optical sensors, F/T and pressure from Hall-effect sensors—into a single model. This fusion allows the sensor to robustly estimate complex contact states like grasp stability and object properties.

Terminology

Calibration — 10× in this paper · explained in 7 episodes
Hysteresis — 7× in this paper · explained in 3 episodes
Accuracy — 6× in this paper · explained in 1 episode
Core — 6× in this paper · explained in 2 episodes
Decode — 3× in this paper · explained in 2 episodes
Latent Representation — 3× in this paper · explained in 2 episodes
International Conference on Robotics and Automation — 2× in this paper
Adam — 1× in this paper · explained in 2 episodes
Affordance — 1× in this paper · explained in 1 episode
Data Fusion — 1× in this paper · explained in 3 episodes
Failure Modes — 1× in this paper · explained in 3 episodes
Fidelity — 1× in this paper · explained in 2 episodes
Robustness — 1× in this paper · explained in 5 episodes

Summary

This paper introduces a novel tactile sensor that integrates velocity, force/torque, and pressure map sensing into a single device with a deformable contact pad to enable slip-aware control for in-hand manipulation. The sensor's primary contribution is combining these multiple sensing modalities within one compliant structure, which allows it to robustly track both flat and curved surfaces while providing rich, dynamic information about contact states.

Key Contributions

To the best of our knowledge, this is the first sensor to combine these sensing modalities within a single compliant structure.

"The main contributions are: A tactile sensor featuring a deformable contact pad that enables optical measurement of planar sliding velocity, estimation of full 6-DoF F/T, and reconstruction of contact pressure distribution through pad deformation using Hall-effect sensors."

The resulting sensor provides relative sliding velocities, full 6-DoF F/T estimates, and spatial pressure information within a single integrated design, which supports robust estimation of grasp state, in-hand motion, object properties, and external contacts.

Sensor Design and Construction

The tactile sensor is designed to combine multiple sensing modalities into a single unit with a microcontroller and USB-C interface. It interacts with objects through a deformable contact pad, which is 3D-printed using TPU 85A. The pad features a gyroid minimal-surface structure for uniform deformation properties and a hexagonal top pattern to improve wear resistance while allowing local deformation to propagate. The construction involves two stacked PCBs, where the bottom PCB houses the USB-C interface and ESP32-S3 microcontroller, and the top PCB contains sensing elements: 13 Hall-effect sensors (MLX90393) and two optical mouse sensor units (PAW3335DB). The optical sensors are embedded in the contact pad, measuring x- and y-displacement to compute linear sliding velocities.

Sensing Modalities and Mapping

The sensor measures three primary quantities: planar sliding velocities, 6-DoF Forces/Torques (F/T), and the contact pressure distribution. The Hall-effect measurements are mapped to these quantities using a neural network architecture composed of a shared encoder and three task-specific output heads. This architecture is designed to decode task-specific quantities from this latent representation. Specifically, one head produces a deconvolutional decoder producing a normalized 32 × 32 pressure map, another predicts the 6 F/T values, and the third predicts the 3-axis magnetic signal of the 13th Hall-effect sensor to compensate for external magnetic disturbances.

Velocity Estimation and Calibration

Planar sliding velocity is inferred using optical sensors embedded in the deformable pad. The relationship between planar sliding velocity, denoted as vector v s = [v sx, v sy, ωs], and the individual optical sensor velocities (v m1, v m2) is defined by Equation (5). The calibration procedure involves two steps: first, performing a predetermined linear motion on a flat surface to estimate the Counts per mm CPMM and both sensor orientations using Equation (7). Second, performing a rotation to estimate the 2D pose of the optical sensors relative to the tactile sensor frame. This allows for calculating radial direction ϕi and estimating distance li traveled along that radial direction to find the final poses (xi, yi) for each optical sensor.

Performance Evaluation

The performance is evaluated through experiments examining F/T estimation, hysteresis and sensor drift over time, magnetic field compensation, pressure map estimation, planar velocity tracking over multiple surfaces, and in-hand slip-aware control. The results show that for objects included in the training set, the sensor achieves an NRMSE of approximately 3–4% while in contact. Furthermore, the deformable pad minimizes stick–slip effects compared to rigid designs, resulting in smoother motion, and it successfully performs rotational sliding on all tested objects. The paper demonstrates that the proposed tactile sensor is capable of accurately measuring sliding displacement while simultaneously estimating 6-DoF F/T and contact pressure distributions.

Limitations and Future Work

The study highlights limitations, such as the viscoelastic behavior of the contact pad leading to drift during prolonged contact, which is attributed to slow deformation. Additionally, the resolution of the pressure map is limited by the embedded magnets, preventing reconstruction of sharp edges or fine details, unlike high-resolution optical sensors. Future work will focus on leveraging these modalities for in-hand perception, contact estimation, more advanced sliding manipulation with planning, alternative contact pads, and multimodal sensor fusion and fallback strategies. The inference time is noted as 1.0 ms on a single CPU core.

Improvements for AI systems

Based on the provided scientific paper, here are specific improvements that can be made to AI systems by integrating its sensing capabilities, and what those improved systems could achieve:

A multimodal perception system capable of simultaneous estimation of contact state, object geometry, and motion during in-hand manipulation.
Improved grasp stability assessment for parallel grippers by incorporating real-time estimates of 6-DoF Force/Torque (F/T) and contact pressure maps derived from the deformable tactile sensor.
Slip-aware control systems that can distinguish between various modes of slip (e.g., rotational vs. linear) based on integrated velocity sensing, enabling the selection of appropriate control strategies for different manipulation tasks.
A perception system capable of robust object pose estimation and property reasoning (such as friction identification) under uncertain contact conditions by fusing tactile data with visual/kinematic information, leveraging the pressure map reconstruction capability.
A compliant manipulation planner that utilizes the sensor's ability to track surface geometry across diverse materials (flat, curved) to adapt contact strategies in real-time, minimizing slippage and maximizing dexterity for tasks involving complex object geometries or varying material properties.
An AI-driven system capable of compensating for external magnetic disturbances on tactile sensors during robotic operation, ensuring the accuracy of F/T and pressure readings regardless of the robot's orientation.

This improved AI system can perform:

Perform dexterous in-hand manipulation with parallel grippers by accurately estimating object pose, grasp stability, and contact forces in real-time.
Execute slip-aware control that dynamically adjusts grip force and motion based on the sensed velocity and contact state (stick vs. slip), resulting in smoother movements on curved or complex objects where traditional rigid sensors fail due to geometry mismatch.
Estimate the friction coefficient of unknown surfaces during interaction by correlating measured forces, pressure distributions, and slip dynamics with pre-trained models derived from the sensor's rich data set.
Perform soft manipulation tasks where precise contact shape and localized pressure distribution are critical for tasks like delicate object placement or surface conforming grasping.
Enable robust robotic interaction with objects exhibiting high material variability (e.g., plastic, wood) by leveraging the sensor's performance across different surfaces, leading to more reliable force control in dynamic environments.
Operate reliably in electromagnetically noisy environments (like near motors), as the AI system can use the integrated magnetic compensation mechanism to maintain accurate force sensing even when the sensor orientation changes.