Visuomotor Robotic Pruning in Planar Orchards Using Hybrid Reinforcement Learning
cs.RO, cs.AI
Submitted: 2026-09-21
Updated: 2026-09-21
Comments: for associated video file, see https://www.youtube.com/watch?v=AjlBe6A0xdo&t
Code: https://github.com/OSUrobotics/lpy
License: http://creativecommons.org/licenses/by/4.0/
The gist: Dormant tree pruning is labor-intensive yet essential for maintaining modern high-productivity fruit orchards.
Terminology
Abstract
Dormant tree pruning is labor-intensive yet essential for maintaining modern high-productivity fruit orchards. In this work, we focus on pruning of modern planar tree training systems - V-Trellis apples and UFO cherries - where trunks and primary branches are trained into approximately planar walls. We introduce an end-to-end pipeline to learn a closed-loop visuomotor controller for robotic pruning. This controller is trained entirely using simulation and synthetically generated data and deployed in real orchards in a zero-shot manner. The pipeline comprises synthetic generation of planar orchard tree meshes, construction of a physics-based orchard simulator, automated collection of successful pruning trajectories via motion planning, and policy learning with a novel hybrid reinforcement-learning algorithm that combines offline demonstrations with online simulated rollouts. The controller uses optical-flow inputs from a wrist-mounted camera - avoiding the need for full 3D-reconstruction - and continuously guides the cutter through cluttered branch environments to a specified cutpoint with correct tool orientation. In exhaustive simulated task-space evaluations over 3,000 pruning points, the policy attains 49.9% success on V-Trellis apples and 46.0% on UFO cherries. We validate the learned controller across 38 physical trials - comprising 28 outdoor field trials in commercial and experimental orchards and 10 indoor laboratory tests - demonstrating zero-shot sim-to-real transfer. The learned policy also outperforms a classical RRT-Connect baseline on physical hardware in laboratory trials.
Sources
- Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
- AWAC: Accelerating Online Reinforcement Learning with Offline Datasets
- Asymmetric Actor Critic for Image-Based Robot Learning
- High-Dimensional Continuous Control Using Generalized Advantage Estimation
- Open3D: A Modern Library for 3D Data Processing
- Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matching
- Depth Anything 3: Recovering the Visual Space from Any Views
- Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
- Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving