Visible Touch: Rendering Contact for Visuomotor Policies

arXiv:2609.14156 · cs.RO · Submitted 2026-09-12 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "Visible Touch: Rendering Contact for Visuomotor Policies".

Dev: Integrating contact information into visuomotor policies remains an open problem because most modern policies operate from vision and proprioception alone, yet touch is essential for robust manipulation.

Rosa: First, who's behind it and why it matters.

Title and authors: Rosa: So, we're talking about the paper 'Visible Touch: Rendering Contact for Visuomotor Policies', which really tackles that long-standing issue of how to get touch information into these vision-based policies without having to rebuild the entire architecture. It sounds like they found a way to make contact data directly usable by image-conditioned models.

Dev: That's right, Rosa, and the authors are basically saying that most modern policies rely only on vision and proprioception, which means touch is currently an add-on that requires extra steps or different hardware. They propose integrating contact signals into the visual input stream instead.

Taro: I'm curious about what this means for autonomy when things go wrong; if the policy isn't explicitly looking at a failure in its visual stream, how does it react to a physical slip?

Rosa: Well, according to their summary of 'Visible Touch: Rendering Contact for Visuomotor Policies', the paper introduces an image-space contact-overlay method. Basically, they project the raw sensor readings onto the policy's RGB observations so that the policy sees both what it sees and where things are touching at the same time.

Dev: That sounds like a significant reduction in complexity because it removes the need for external force calibration or adding separate encoders to handle those contact signals. It shifts how we think about sensor integration entirely.

Taro: If it's just an overlay, does that mean the policy still has to learn how to interpret that visual representation of touch? What happens when the world misbehaves and the overlay is misleading?

Rosa: The authors suggest that because the contact signals are in a spatial frame with the scene, the policy already uses its spatial reasoning for free when it processes this augmented image input. They claim this allows any image-conditioned policy to use it directly without needing architectural changes.

Dev: I see the core idea is leveraging what's already in the visual tokens, which avoids creating a separate processing pathway or a state vector concatenation which we often see with other approaches like those mentioned in their related work section.

Taro: But they also characterize when this overlay helps and when it doesn't; they found that the benefit scales with task headroom and grasp structure, peaking on multi-grasp long-horizon tasks. That suggests it's not a universal fix for every single manipulation problem.

Rosa: Exactly, Taro, and their finding on optimal overlay granularity is interesting because they found that the best visual detail depends entirely on how spatially consistent the underlying contact signal is; too much detail just creates visual clutter that can distract the policy.

Title and authors: Dev: From an engineering standpoint, that granularity issue tells us we need a way to dynamically decide how much information to render based on what the task actually requires, rather than just rendering everything at maximum detail.

Taro: I wonder if this dynamic adjustment capability is something we can build in or if it's purely a characteristic of the specific task structure they tested; I want to know how robust this system is when the underlying physical interaction changes unexpectedly.

Rosa: The paper shows that for real-world contact-rich manipulation tasks, they see gains of about thirty point three percentage points on success rates, compared to just fine-tuning pretrained VLAs where they see a gain of about twenty-five point three percentage points on LIBERO fine-tuning suites and the results from the paper 'Visible Touch: Rendering Contact for Visuomotor Policies' show +25 point 3pp on LIBERO fine-tuning suites and +30 point 3pp on real-world contact-rich manipulation tasks (<ref:2609.14156#pg1>).

Dev: Those percentage gains are substantial, especially when you compare them to the separate stream alternatives they analyzed; it shows that this image-space delivery is indeed more effective than just feeding a separate tactile encoder into the existing loop.

Taro: If we look at what they did in terms of hardware, they used a custom low-cost magnetic contact sensor fabricated from off-the-shelf parts, and they characterized the signal as the "l2 norm of the offset-corrected flux vector r = ∥s∥two ∈ R," which captures the total contact-induced deflection summed across all nine magnetometers (<ref:2609.14156#pg0>).

Rosa: That low-cost aspect is a huge practical advantage for field robotics, because having expensive, high-fidelity tactile sensors isn't always feasible when you need to deploy something quickly. The hardware is designed to be cost efficient with the total assembled cost under approximately forty dollars (<ref:2609.14156#pg0>).

Dev: And the characterization confirms that even this raw magnetic flux provides a consistent and unambiguous contact signal without needing any force calibration, which simplifies the entire setup considerably for deployment.

Taro: It’s interesting to consider the implications for future autonomy; if we can reliably inject these kinds of physical interaction details into models, it opens up possibilities for much more nuanced object manipulation than what vision alone allows.

Rosa: Absolutely, and their conclusion is that this image-space delivery outperforms both a compact state-vector baseline and information-matched separate-stream alternatives (<ref:2609.14156#pg1>). This suggests we can achieve better performance by sticking to image-conditioned policies while enriching the visual input spatially.

Dev: I think the main implication is that for fine-tuning models like miniVLA or BC-Transformer, this method provides a significant performance uplift without requiring any modifications to their existing architectures, which is very attractive for deployment timelines.

Title and authors: Taro: For me, the future work they point toward—investigating optimal overlay granularity with finer-grained sensors and additional simulators—that’s where we need to focus if we want to move this from lab success to reliable field performance in unstructured settings.

Rosa: So, to wrap up on 'Visible Touch: Rendering Contact for Visuomotor Policies', the main contribution is delivering image-space contact awareness drop-in into any image-conditioned policy without architectural changes or force calibration. It's a method and hardware package that generates three dee-printable molds from user geometry (<ref:2609.14156#pg0>).

Dev: We saw how this technique improves average task success by twenty-five point three percentage points on LIBERO fine-tuning suites and by thirty point three percentage points on real-world contact-rich manipulation tasks, showing tangible performance gains (<ref:2609.14156#pg1>).

Taro: I’d add that the fact that the overlay benefit scales with task headroom is important because it tells us this isn't a universal fix; it’s highly dependent on the complexity of what the robot is trying to do (<ref:2609.14156#pg2>).

Rosa: Indeed, and their analysis shows that image-space delivery outperforms other methods by matching task success gains with grasp structure and task complexity. This suggests we’re getting closer to policies that can robustly handle complex physical interactions.

Dev: So, the practical implication for control engineering is that we might be able to skip the complex calibration and fusion steps and just feed this augmented image directly into our standard vision pipelines, which is a huge win for loop rate stability.

Taro: Ultimately, if we can make this work reliably outside of a controlled lab environment—if it generalizes well to different morphologies—then we are talking about a massive step toward truly robust autonomous manipulation systems.

Rosa: We've discussed the title, the mechanism, and the performance figures for 'Visible Touch: Rendering Contact for Visuomotor Policies'. This paper shows how to integrate touch information directly into image-conditioned policies using a spatial overlay method that requires no architectural changes or force calibration.

Dev: The results show gains of over twenty-five percentage points on fine-tuning tasks and thirty percentage points on real-world contact tasks, which is significant performance improvement for these types of models <ref:2609.14156#pg1>.

Taro: I think the main implication for autonomy is that we can finally give robots a way to "see" the physical forces and geometry of contact in the same visual context as they see their environment.

Rosa: We’ve covered how they achieve this using a low-cost magnetic sensor and a specific characterization of when this overlay is most effective, which peaks on multi-grasp long-horizon tasks (<ref:2609.14156#pg2>).

Dev: I think we should keep an eye on the future work they mention regarding finer sensor resolution and simulators, as that’s where we’ll likely see the next iteration of this approach maturing for deployment.

The paper's summary: Rosa: So, to recap what we've just discussed, the core of 'Visible Touch' is taking raw tactile contact measurements and projecting them onto an image so that any existing vision-based policy can use them without having to rewrite its code or calibrate for force.

Dev: That’s right, Rosa; the magic is in that image-space overlay which makes it a drop-in input for models like miniVLA. I'm focusing on the loop rate here—if this augmentation adds significant computational overhead, we might see latency issues that could affect real-time control.

Taro: From an autonomy standpoint, I'm thinking about how this handles unexpected contact events; when the world misbehaves and things slip unexpectedly, does this visual cue help the policy recover or just get stuck?

Rosa: Well, the authors found that this method gives models a substantial performance boost across different tasks—we're looking at gains of over thirty percentage points on real-world contact tasks compared to baselines.

Dev: Those gains are impressive, but I need to know about the failure modes; what happens if the contact signal itself is noisy or if the projection mapping introduces artifacts that confuse the policy?

Taro: The paper suggests that when it works best, it's on multi-grasp long-horizon tasks and it depends heavily on how spatially consistent the physical contact signal actually is. If we can get that spatial consistency right, the policy seems to handle those complex interactions much better.

Rosa: That’s a key finding; the benefit scales with task headroom, meaning it’s not just a small bump in performance but something that really helps when the robot is trying to do complicated sequences of actions.

Dev: I'm still concerned about deployment outside of controlled labs; how long do you think this system can maintain its accuracy and reliability when exposed to unstructured environments over an extended period?

Taro: The authors explicitly state that generalization to other morphologies, like deformable objects or highly unstructured settings, is what they plan to investigate next; right now, the focus is on confirming the mechanism works well within their tested contact-rich tasks.

Rosa: So we're looking at a powerful visual enhancement that bypasses architectural changes and calibration needs, but we still have to confirm its long-term robustness in messy real environments.

Dev: Exactly; from a control engineering side, if the latency introduced by rendering those arrows becomes too high or inconsistent, it could destabilize the policy's fine-tuning process significantly.

Taro: It seems like this work provides a really solid foundation for integrating physical interaction data directly into perception streams, which is a big step toward richer autonomy.

The paper's improvements: Rosa: So, we've talked about how 'Visible Touch' integrates contact data into vision inputs, and now we need to look at what they suggest to make this system even better with these proposed improvements.

Dev: I’m interested in the specifics of those suggested changes; if they propose more detailed visual cues than just the basic overlay, that directly impacts how stable our control loop stays when things are moving fast.

Taro: When you talk about improvements, are we talking about increasing the granularity of that visual feedback—like going from a general area to seeing specific contact points—and how that affects failure detection?

Rosa: They suggest a dynamic approach where the level of detail on the overlay adjusts based on what the task demands; they can switch between coarse vertical bars and detailed per-taxel arrows depending on whether the grasp requires high or low precision.

Dev: A dynamic adjustment mechanism sounds promising for latency management, but I need to know how that switching happens in real time without introducing jitter into the perception pipeline.

Taro: That granularity idea ties back to my question about misbehavior; if we can dynamically tailor the visual input to match the complexity of the physical contact, it should give the AI a much clearer signal when things go wrong.

Rosa: It seems like they're pushing toward a system that can intelligently decide how much tactile detail is useful in real-time, maximizing information gain while keeping visual clutter at bay.

Dev: That intelligent decision-making has implications for robustness; if the AI can suppress unnecessary visual noise when it doesn't matter, it should help keep the processing load manageable for a consistent loop rate.

Taro: If we can achieve that level of task-specific detail tailoring, I think we could see much better emergent retry behavior when a physical interaction fails because the policy gets an unambiguous reading of exactly what went wrong.

Rosa: That’s exciting because it moves us away from a one-size-fits-all input and toward a system that adapts its sensory focus to the current physical state of manipulation.

Dev: But I still need assurance on how these improvements hold up under extended field use; if we introduce more complex rendering logic, we're increasing the potential for unpredictable behavior when things are running for many hours straight.

Taro: The authors are looking ahead at testing these finer-grained sensor ideas with more simulators to see if that dynamic granularity translates well from the lab setup to real-world physics.

Rosa: So they’re confirming that while the mechanism is powerful, the next hurdle is proving that this adaptive visual enhancement remains reliable and effective across diverse physical scenarios.

Conclusion: Rosa: So, to wrap up 'Visible Touch: Rendering Contact for Visuomotor Policies', we've seen how this image-space contact-overlay method successfully drops into any image-conditioned policy without needing architectural changes or force calibration.

Dev: It really shows how we can enhance existing vision models with physical interaction data by simply augmenting the input stream, which is a big win for our latency concerns because it avoids adding complex, separate processing streams.

Taro: I think the real impact here is giving autonomy researchers a way to handle situations where the world misbehaves by providing that explicit physical context directly to the policy's perception.

Rosa: Exactly; this capability means policies can actually "see" the forces and geometry of contact in a way vision alone just can't, which opens up possibilities for much more nuanced object manipulation in real-world settings.

Dev: I still have that lingering question about field reliability; we need to see how long this augmentation stays stable when deployed outside of our controlled lab environment over many hours of operation.

Taro: The authors flag that generalizing this dynamic visual tailoring to truly unstructured environments remains a challenge, but the mechanism itself is solid for handling complex contact states during manipulation.

Rosa: So, while we have a very strong method for getting touch information into image-conditioned policies, our next big step is proving its long-term stability and generalization across different physical morphologies.

Dev: I think we need to focus our next engineering efforts on building the low-latency rendering pipeline that can handle those dynamic visual cues without introducing any significant control loop jitter.

Taro: I’m looking forward to seeing how this foundational work evolves when they start testing it against more complex, non-rigid objects and messy physical interactions in simulation.

University of California, Los Angeles

cs.RO

Submitted: 2026-09-12

Updated: 2026-10-07

Comments: Accepted as a Spotlight at the 10th Conference on Robot Learning (CoRL 2026). Project website: https://visibletouch.github.io/

Code: https://github.com/Stanford-ILIAD/openvla-mini

Project page: https://visibletouch.github.io

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: Integrating contact information into visuomotor policies remains an open problem because most modern policies operate from vision and proprioception alone, yet touch is essential for robust

Key concepts

Image-space contact-overlay method
This is the core technique where raw contact measurements from a physical sensor are transformed into visual information overlaid onto the robot's RGB image. Instead of feeding raw numbers into the policy, it creates a 'visual cue' that shows where and how objects are touching in the camera view, making it directly usable by vision-based policies.
Rendering pipeline
This component is responsible for projecting contact data onto the image. It takes local contact measurements and uses the robot's known position and camera parameters to calculate where each contact point appears on the screen. This process creates an augmented image that serves as a direct input for fine-tuning vision models.
Magnetic tactile sensor
This is a low-cost hardware component used to capture physical contact without needing expensive, bulky force sensors. It works by detecting changes in magnetic flux caused by external forces deforming magnets embedded in a soft elastomer surface. This provides an unambiguous signal about contact presence and location.
Spatial consistency of the underlying contact
This refers to how the physical arrangement of contacts relates to each other in space. The paper finds that the effectiveness of the visual overlay is highest when it matches this spatial structure, meaning the visual cues are most helpful when they accurately reflect how objects are physically arranged and touching.

Terminology

Summary

Integrating contact information into visuomotor policies remains an open problem because most modern policies operate from vision and proprioception alone, yet touch is essential for robust manipulation. The key insight presented in this work is that when contact signals are exposed in the same spatial frame as the scene, they become directly usable by any image-conditioned policy without architectural changes.

The gist: Visible Touch introduces an image-space contact-overlay method that drops into any image-conditioned policy with no architectural changes and no force calibration, improving performance across three distinct policy families.

Motivation and Challenges

Touch is fundamental to human manipulation, providing crucial information about secure grasping, contact states, and grip firmness. However, current visuomotor robot policies overwhelmingly operate from vision and proprioception alone. This gap stems from three primary challenges: sensing (high-fidelity tactile sensors are often expensive or bulky), signal interpretation (raw readings are difficult to interpret in isolation), and architectural integration (existing approaches either concatenate contact to the proprioceptive state vector or add a separate tactile encoder, both requiring modifications).

Visible Touch Method

Visible Touch is a method and hardware package designed for image-space contact-aware policy learning. It consists of two coupled components: (i) a rendering pipeline that projects per-element contact measurements onto the policy’s RGB observations as a spatial overlay, and (ii) a low-cost magnetic tactile sensor that drives the pipeline on real hardware. The rendering pipeline is sensor-agnostic, operating on raw per-element measurements without calibration to physical force units.

The projection process involves:

  1. Transforming each sensing element's local position through the robot’s live forward kinematics and calibrated camera parameters to determine its projected image location in the camera frame.

  2. Calculating the screen-space arrow length, which reflects both measurement intensity and perspective foreshortening at the element’s depth, providing a depth-consistent visual cue.

  3. Alpha-blending these arrows onto the base RGB image with per-layer opacity to create an augmented image that serves as a drop-in input for fine-tuning pretrained VLAs, requiring no architectural changes.

Hardware and Sensor Details

The method utilizes a custom low-cost magnetic contact sensor fabricated from off-the-shelf parts via a parametric CAD-to-mold pipeline. This sensor captures contact through 3D magnetic flux: external forces deform a soft elastomer surface embedded with permanent magnets, and the resulting field changes are recorded by an underlying magnetometer array. The hardware is designed for cost efficiency, with the total assembled cost under approximately underly [50].

The sensor characterization confirms that raw magnetic flux provides a consistent and unambiguous contact signal without requiring force calibration. The aggregate sensor response is computed as the l2 norm of the offset-corrected flux vector r = ∥s∥2 ∈ R, which captures the total contact-induced deflection summed across all nine magnetometers.

Performance and Findings

Visible Touch demonstrates significant performance improvements across different policy architectures and task types. Across the LIBERO benchmark, imagespace overlays improve BC-Transformer success by 15.7 percentage points on average in the 2-view setting. When fine-tuning pretrained VLAs, miniVLA gains 25 percentage points on average on LIBERO. Furthermore, across four real-world contact-rich tasks with π0.5, the method shows gains of 30 percentage points.

The analysis reveals several key patterns:

(Q1)

(Q2)

(Q3a)

The overlay benefit scales with task headroom and grasp structure, peaking on multi-grasp long-horizon tasks. The optimal overlay granularity matches the spatial consistency of the underlying contact signal, suggesting that excess spatial detail can introduce visual clutter that displaces the encoder’s attention from task-relevant features.

Contributions

The main contributions are:

  1. Visible Touch: an image-space contact-overlay method that drops into any image-conditioned policy with no architectural changes and no force calibration.

  2. A low-cost, open-source magnetic contact sensor and parametric CAD-to-mold pipeline that generates 3D-printable molds from user-specified geometry.

  3. A characterization of when and how contact overlays help: gains scale with task headroom and grasp structure, peaking on multi-grasp long-horizon tasks; image-space delivery outperforms a compact state-vector baseline and information-matched separate-stream alternatives; and optimal overlay granularity matches the spatial consistency of the underlying contact.

Limitations

The work notes that the comparison against a compact state-vector baseline does not establish superiority over learned tactile encoders or other feature-level fusion mechanisms. Generalization to other morphologies, deformable objects, and unstructured settings remains future work. The conjecture that optimal overlay granularity tracks the spatial structure of the contact signal merits further investigation with finer-grained sensors and additional simulators.

Improvements for AI systems

Based on the provided scientific paper, here are the specific improvements that can be made to AI systems and what those improved systems can achieve:

The core improvement is integrating contact information directly into image-conditioned policy inputs via a novel Visible Touch method, bypassing architectural modifications or separate state vectors.

Here are the specific improvements and capabilities:

  1. A new class of image-conditioned policies (including pretrained Vision-Language-Action models like miniVLA and π0.5) will be enhanced by rendering raw contact sensor data as a spatial overlay directly onto their RGB observations.

  2. This augmentation is drop-in, meaning it requires no architectural changes to the underlying policy backbone, making it immediately applicable to existing large models (like BC-Transformer, miniVLA, and π0.5).

  3. The system can leverage low-cost magnetic contact sensors fabricated via a parametric CAD-to-mold pipeline that generates 3D-printable molds from user geometry, drastically reducing the cost barrier for tactile sensing hardware (under the cost of a few dollars).

  4. The policy will gain superior performance on complex, contact-rich manipulation tasks, specifically showing significant gains (+25 to +43 percentage points) on benchmarks like LIBERO and real-world tasks.

  5. The improved AI system will exhibit enhanced robustness and stability during fine-tuning across different random seeds (reducing seed-to-seed variability by up to 70% compared to plain baselines).

  6. The policy will demonstrate superior performance in scenarios where vision alone is ambiguous (e.g., grasping slippery, small, or occluded objects), as the contact overlay provides crucial missing information that pixels alone cannot convey.

  7. The system can dynamically determine the optimal level of tactile detail to use—ranging from coarse vertical bars (Binbars) to detailed per-taxel arrows (Multi-arrows)—depending on the specific task's structural requirements, maximizing information gain while minimizing visual clutter.

  8. The improved system will exhibit emergent retry behavior during real-world manipulation; when a grasp fails, the tactile signal after the miss unambiguously identifies the failure and prompts a recovery attempt—a qualitative benefit not present in vision-only or simple state-vector baselines.

In summary, these improvements allow AI systems to transition from relying solely on visual perception to achieving high-fidelity contact awareness. The resulting system can perform complex, multi-stage manipulation tasks (like transferring objects or plugging in chargers) with significantly higher success rates by effectively seeing the physical forces and geometry of contact in the same spatial context as it sees the scene.

Abstract

Integrating contact information into visuomotor policies remains an open problem. Touch is essential to robust manipulation, yet most modern policies, including pretrained vision-language-action (VLA) models, operate from vision and proprioception alone. Existing approaches to closing this gap require specialized tactile hardware, add separate tactile encoders, or commit to non-image policy backbones, all incompatible with the modern paradigm of image-conditioned policies built on pretrained 2D visual representations. Our key insight is that the bottleneck is not the contact information itself, but how it is delivered: when contact signals are exposed in the same spatial frame as the scene the policy already attends to, they become directly usable by any image-conditioned policy without architectural changes. We operationalize this insight in Visible Touch, paired with a custom low-cost magnetic contact sensor that is open-sourced and fabricated from off-the-shelf parts via a parametric CAD-to-mold pipeline. Across the LIBERO benchmark, Visible Touch improves BC-Transformer success by 15.7 percentage points on average in the 2-view setting, with similar gains in the 1-view setting; controlled comparisons show that the contact-integration strategy strongly affects how effectively tactile information is used. The pattern holds when fine-tuning pretrained VLAs: miniVLA on LIBERO gains 25 percentage points on average, and π 0.5 on four real-world contact-rich tasks gains 30 percentage points with our custom sensor.

Sources

Related papers