Toward Humanoid Robots in Construction: A Teleoperation Feasibility Study

arXiv:2610.00718 · cs.RO, cs.HC · Submitted 2026-09-30 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "Toward Humanoid Robots in Construction".

Dev: We present a teleoperation system that enables a single operator to perform construction tasks on a Unitree G1 humanoid,

Rosa: First, who's behind it and why it matters.

Paper summary: Rosa: Welcome back everyone. We're talking about this paper today, "Toward Humanoid Robots in Construction: A Teleoperation Feasibility Study." It looks like they've put forward a system where one person can guide a Unitree G1 humanoid to do construction jobs, combining head and hand tracking with foot pedals for movement.

Dev: I’m really interested in seeing how practical this setup is, Rosa. I want to know if we're talking about something that could actually run outside of a controlled lab environment for extended periods on a real job site.

Taro: From an autonomy standpoint, I’m curious if this approach provides any useful framework when the environment throws unexpected challenges at the robot during operation.

Rosa: Exactly, Taro, that's what I want to dig into—does this system handle things that aren't perfectly planned?

Dev: Right, and from an engineering viewpoint, my main concern is the latency and how reliable those control loops are when you’re dealing with real-world movement. The paper mentions processing inputs through a controller PC before relaying commands to the G1, which means we have to keep that loop rate tight or we're looking at some serious instability.

Rosa: That’s a big question for field deployment, Dev. If the latency is too high, you can't really feel the robot respond in real time while you're trying to manage a complex physical task on site.

Taro: And if things go wrong, say the operator makes an error, what happens next? Does the system have a safety fallback when dealing with misbehaving robots or unpredictable human interaction?

Rosa: I think we need to focus on the real-world deployment aspect here. The paper claims they deployed this on an active construction site and tested it against specific tasks from the O*NET database.

Dev: So, what were those specific tests like, Rosa? Did they just do a simple pick-and-place thing, or was it more complex in terms of coordination?

Taro: That’s where I want to know if the system handles things that require dynamic decision-making while executing the locomotion and manipulation simultaneously.

Rosa: They tested two specific tasks: tool transport and painting. Tool transport involved grasping a misplaced hand tool, walking about four meters to a bin, and placing it inside.

Paper summary: Dev: So that first test focused on coordinating both the movement via those GLYDR pedals and the hand tracking for the manipulation part at the same time?

Taro: That’s interesting because carrying an object between two distinct points requires a lot of dynamic balance adjustments, which should really stress the locomotion control system.

Dev: Indeed, and they reported a success rate of one hundred percent for that tool transport task with the teleoperation setup <ref:2610.00718#pg0>. However, they also pointed out that the average teleoperation time was about seventy-two seconds compared to just four seconds when done manually.

Rosa: Wow, an eighteen-fold slowdown there is significant; it really highlights how much extra effort just for moving things adds up.

Taro: I see that, and I wonder if that slowdown is mostly due to the locomotion part, or if the hand tracking manipulation itself was the bottleneck in terms of precision.

Rosa: The paper attributes most of that time penalty to needing additional locomotion and repositioning just to get the object from one location to another.

Dev: That makes sense from a control perspective; you're not just controlling the arm, you're controlling the whole body's path while trying to maintain grasp stability during that movement.

Taro: When we think about real construction scenarios, what happens if that misplaced tool isn't exactly where the operator expects it to be? Does this system have enough adaptability for that kind of uncertainty?

Rosa: The painting task involved using the robot to grasp a roller brush and apply paint across a piece of paper. They achieved an eighty percent success rate there <ref:2610.00718#pg0>.

Dev: So, while tool transport was perfect, the continuous contact required for painting seems to introduce more fragility into the hand tracking system?

Taro: I think that eighty percent success rate suggests that maintaining a stable grasp through extended motion is a real hurdle for pure hand tracking methods when dealing with tools like roller brushes <ref:2610.00718#pg0>.

Rosa: That’s what the authors flagged, suggesting that reliable grasp acquisition and retention are important limitations of these pure hand tracking systems.

Dev: And they also mentioned something else concerning sustained operation—they observed motor overheating during extended teleoperation sessions.

Taro: Motor overheating is a serious physical limitation; if the hardware gets too hot, performance degrades quickly, which directly impacts the reliability we need for field work.

Paper summary: Rosa: So, while this study shows feasibility with high success rates on specific tasks like tool transport and painting, it also clearly shows significant time penalties and stability issues with continuous manipulation.

Dev: That points toward the fact that while we can get the basic motion working, scaling this up to complex construction jobs will require major improvements in how we manage power and control feedback.

Taro: Thinking about the bigger picture, if these limitations aren't addressed—the slow speed and unstable grasping—how does this impact the broader goal of getting humanoids into messy, real-world industrial settings autonomously?

Rosa: It shows that teleoperation is a valid near-term approach for generating valuable demonstration data even with these current limitations.

Dev: The collected data in LeRobot format, which includes things like the Zed Mini camera stream and tactile sensing, is super important for training future imitation learning policies.

Taro: So, the real implication here isn't just about making it work today, but about how this data collection pipeline helps us build smarter autonomy later.

Rosa: I agree; generating that synchronized demonstration data is a huge asset for future autonomous execution on these platforms.

Dev: We need to keep pushing on reducing that time penalty and the motor thermal issues because those are tangible engineering problems we have to solve before you can trust this system for anything more than simple, short tasks.

Taro: And from an autonomy perspective, as long as we can get the data pipeline working reliably, we have a path forward for training policies that understand how to handle those kinds of physical uncertainties when they occur in construction.

Rosa: So, to wrap up on this paper by Parastoo Ali Pour et al., this study confirms that a single operator can indeed perform construction tasks on a Unitree G1 using XR and foot pedals.

Dev: But it also clearly lays out the trade-offs: you gain remote operation, but you pay for it with significant time slowdowns and grasp stability challenges during manipulation.

Taro: The key lesson seems to be that for real-world application, we need to tackle the locomotion speed penalty and ensure better grasp retention before we can move toward more complex autonomy in construction.

Conclusion: Rosa: So, we've been looking at how this system lets one person run a Unitree G1 on construction sites, and now we need to talk about what that title itself says about the whole endeavor.

Dev: Exactly, Rosa; that paper is called "Toward Humanoid Robots in Construction: A Teleoperation Feasibility Study," which tells us it’s really focused on testing if this setup can actually work in a real job environment.

Taro: I think the title signals that they're not just looking at a neat lab demonstration, but they're trying to figure out if this teleoperation method has any real promise for actual construction work.

Rosa: Right, and when you break it down simply, this paper is basically checking if we can use a seated operator to guide a humanoid robot through the messy reality of building sites using XR and foot pedals.

Dev: It's about testing the practical limits of that combination—specifically how reliable the control loops are when you’re dealing with the physical demands of construction tasks, which is what that feasibility study really means.

Taro: And for autonomy, it suggests that if we can get this kind of remote guidance working, we have a starting point for training policies on robots to handle those complex construction scenarios where things aren't perfectly planned.

Rosa: So, the core implication is that this isn't just a proof-of-concept; it’s an attempt to bridge the gap between controlled testing and the actual demands of industrial labor.

Dev: That brings us to the real question, Rosa, how long can we expect this setup to stay functional outside of a perfectly controlled lab environment before we hit serious issues with latency or thermal management?

Taro: And if those challenges are overcome, what's the bigger picture for autonomous construction; does this method pave the way for robots doing more than just simple pick-and-place?

Parastoo Ali Pour, David R. Martin, Chang Min Hur, Bo Zhang, Tommy Zhou, Brandon Thomas Lichter, Shane Stanfield, Pramod Khargonekar, Mohammad Abdullah Al Faruque

University of California at Irvine

cs.RO, cs.HC

Submitted: 2026-09-30

Updated: 2026-09-30

Comments: Accepted at IROS 2026 Workshop on Future of Construction

Code: https://github.com/unitreerobotics/xr

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 76/100

The gist: We present a teleoperation system that enables a single operator to perform construction tasks on a Unitree G1 humanoid, combining extended reality (XR) based upper body control with pedal-based

Key concepts

XR Teleoperation: This involves using extended reality technology, specifically a Meta Quest 3 headset, to allow an operator to see the robot's view and track their hands. This setup lets the operator control the robot's upper body movements as if they were physically touching it from a distance.
Pedal-Based Locomotion: This refers to controlling the robot's movement using foot pedals, like those from GLYDR. This enables a seated operator to manage both walking and manipulating tools simultaneously, allowing for complex tasks on a job site.
Imitation Learning Data: The system records every teleoperation session in a specific format containing video streams, motor states, and actions. This collected data is intended to be used as training material so that future autonomous robot policies can learn how to perform construction tasks effectively.

Terminology

Core — 5× in this paper · explained in 2 episodes
Adaptability — 1× in this paper · explained in 1 episode
Foundation Model — 1× in this paper · explained in 3 episodes

Summary

We present a teleoperation system that enables a single operator to perform construction tasks on a Unitree G1 humanoid, combining extended reality (XR) based upper body control with pedal-based locomotion to enable simultaneous manipulation and locomotion.

How it works

The core of the system is designed for remote, seated operation of a humanoid robot on an active job site. The operator utilizes a Meta Quest 3 headset for egocentric video streaming and hand tracking, while using GLYDR foot pedals for lower-body locomotion control. These inputs are processed through a controller PC, which relays commands to the Unitree G1 humanoid equipped with Inspire five-finger hands with tactile sensing. This setup achieves three primary objectives: (1) an extended reality perception system that lets the operator actively look around from the robot’s point of view, (2) hand tracking based manipulation, and (3) hands-free locomotion control that enables simultaneous locomotion and manipulation control by a seated operator.

The active visual perception is achieved by equipping the robot with a StereoLabs ZED Mini stereo camera mounted on the 2-degrees of freedom (DoF) neck developed in [12]. The stereo images are streamed to the Quest headset via WebRTC, and the operator’s head pitch and yaw recorded from the Quest headset are converted to pulse width modulation (PWM) values that drive the camera to match the operator’s head orientation. Upper body and hand control are managed through Unitree’s XR teleoperate framework, which maps operator and finger poses to robot joint positions.

Data Collection Pipeline

A dedicated data collection pipeline was implemented to record each teleoperation session in the LeRobot dataset format. This format captures critical information, including the Zed Mini Stereo camera stream, tactile sensing, motor states, motor actions, and velocity commands. The resulting synchronized demonstrations are intended to be used as training data for imitation-learning, allowing each teleoperated task to both accomplish useful work and contribute data toward future autonomous execution.

Evaluation of Construction Tasks

The system was deployed on an active construction site and evaluated on two representative high-importance construction tasks drawn from the O'NET occupational database [13] for Construction Laborers. These tasks were:

Tool Transport: This task involved the operator controlling the robot to grasp a misplaced hand tool, walk approximately 4 meters to a designated tool bin, and place the tool inside. This task evaluated simultaneous locomotion and manipulation through the GLYDR pedals and Quest hand tracking.
Painting: This task required the operator to use the robot to grasp a roller brush and apply a paint coating across a piece of paper, evaluating sustained, continuous-contact manipulation, requiring the operator to maintain contact and consistent coverage during an extended arm motion.

Performance Results

The preliminary evaluation showed that both operators were able to complete Tool Transport with a 100% success rate and Painting with an 80% success rate, demonstrating the feasibility of using a teleoperated humanoid for construction tasks. However, the paper reported significant performance penalties compared to manual execution:

(Tool Transport)

The average teleoperation time was approximately 72 seconds versus 4 seconds manually (a ∼18× slowdown). The larger penalty is attributed to additional locomotion and repositioning required to carry an object between locations.

(Painting)

The painting task took 149 seconds via teleoperation compared to 47 seconds manually (a ∼3× slowdown). The lower success rate for painting was primarily associated with unsuccessful or unstable grasp of the roller brush, suggesting that reliable grasp acquisition and retention are important limitations of pure hand tracking systems. Furthermore, the study observed motor overheating during extended teleoperation sessions, highlighting the need for improved control strategies.

Conclusion and Future Work

The work demonstrated the potential of teleoperated humanoids for construction applications by achieving high success rates while generating valuable training data. Key limitations identified include the substantial time penalty compared to manual execution due to locomotion requirements, grasp stability issues in manipulation tasks, and motor overheating during sustained operation. Future work will focus on expanding the task suite, improving system effectiveness through haptic feedback methods for the operator, and developing control approaches that reduce motor overheating. The collected demonstration dataset is planned for publication to assist in training autonomous policies.

The gist: A teleoperation system combining XR-based upper body control with pedal-based locomotion enables a single operator to perform construction tasks on a Unitree G1 humanoid, demonstrating feasibility but revealing significant time penalties and grasp stability challenges.

Improvements for AI systems

Here are specific improvements to AI systems based on the findings in this research, categorized by the area of improvement:

Acknowledge and Integrate Teleoperation Data for Imitation Learning (IL) Training:
Develop Robust Grasp Acquisition and Retention Modules for Manipulation Tasks:
Implement Adaptive Control Strategies to Mitigate Motor Overheating During Extended Teleoperation:
Enhance Locomotion Control Policies for Simultaneous Manipulation/Locomotion Scenarios:
The improved AI system will be capable of performing construction tasks by leveraging recorded teleoperation data (from the LeRobot dataset) to train vision-language-action (VLA) models for autonomous execution in real-world, unstructured environments.
The system can reliably grasp and transport tools across varied terrain while maintaining a 100% success rate, as demonstrated in the study's Tool Transport evaluation.
The AI will exhibit enhanced manipulation skills during painting tasks by incorporating learned strategies to ensure stable contact and consistent coverage, aiming to raise the current 80% success rate for sustained manipulation.
The control system can dynamically adjust its gait and locomotion commands (using GLYDR input) in real-time based on the object being manipulated, allowing for seamless transitions between walking, carrying tools, and performing precise manipulation simultaneously.
The system will incorporate predictive modeling of motor thermal states to proactively throttle or adjust control frequencies during prolonged teleoperation sessions, extending operational endurance and safety beyond current limits.