pipette
ESEspañol

E2DRL: End-to-End Differentiable Reinforcement Learning for 3D Obstacle Avoidance in Quadruped Robots

Cheng Fan, Bingyang Zhou, Gang Yang, Zixuan Zhuang, Aoqian Zhang, Pengzhi Yang, Xin Liu, Chunzheng Wang, Yan Rui Tan, Fan Shi

PreprintClaims a big stepReal-world use

In the authors' words

Differentiable reinforcement learning shows strong promise for robot policy learning, but its application to legged locomotion has remained limited to blind walking or simulated 2D tasks. Extending it to end-to-end visual obstacle avoidance is challenging because hard collision constraints and early termination upon contact corrupt first-order gradients, trapping optimizers in local minima. We present E2DRL, which addresses this with a soft differentiable obstacle-interaction formulation: penetration penalties across multiple body keypoints with velocity-conditioned stop-gradient shaping to provide the first-order training signals for 3D obstacle avoidance. To realize a deployable perception-control system, a privileged teacher trained via Soft Analytic Policy Optimization (SAPO) in a differentiable simulator is distilled into a recurrent depth-based student via DAgger, then blended with a base locomotion policy using a terrain-aware gating module for deployment. In simulation, E2DRL achieves higher sample efficiency than PPO and demonstrates superior velocity-tracking performance and 3D obstacle-avoidance capability compared with both PPO and Duality-based DCBF NMPC. We present the first sim-to-real validation of end-to-end differentiable RL for 3D obstacle avoidance on a Unitree Go2 in static and dynamic environments.

Main resultThe abstract does not state a limitation.

In plain words AI summary

The authors created E2DRL, a method that uses soft penalties when robot body points touch obstacles and adjusts training signals based on speed. They trained an expert system in a differentiable simulator, then transferred its knowledge to a simpler system that uses depth cameras for real-world use. In tests, E2DRL learned faster than PPO and outperformed both PPO and a nonlinear model predictive controller at following speeds and avoiding obstacles. The team validated their method on a real Unitree Go2 robot in both static and moving obstacle environments, a first for this type of learning.

Written by Kimi K2.5 using only the abstract above. Jev checked that every sentence is supported by the abstract sentences it cites, and removed any that were not. Point at a sentence to highlight its source. 1 sentence was removed for not being supported by the abstract.

Appeared: Thursday, October 15. arXiv. Preprint, not yet peer-reviewed.