Robot Learning

CoMo: Learning Continuous Latent Motion from Internet Videos for Scalable Robot Learning

Abstract: Learning latent motion from Internet videos is crucial for building generalist robots. However, existing discrete latent action methods suffer from information loss and struggle with complex and fine-grained dynamics. We propose CoMo, which aims to learn more informative continuous motion representations from diverse, internet-scale videos.

Jiange Yang, Yansong Shi, Haoyi Zhu 朱皓怡, Mingyu Liu, Kaijing Ma, Yating Wang, Gangshan Wu, Tong He, Limin Wang

Tra-MoE: Learning Trajectory Prediction Model from Multiple Domains for Adaptive Policy Conditioning

Abstract: Learning from multiple domains is a primary factor that influences the generalization of a single unified robot system. In this paper, we aim to learn the trajectory prediction model by using broad out-of-domain data to improve its performance and generalization ability.

Jiange Yang, Haoyi Zhu 朱皓怡, Yating Wang, Gangshan Wu, Tong He, Limin Wang

Tra-MoE: Learning Trajectory Prediction Model from Multiple Domains for Adaptive Policy Conditioning

Point Cloud Matters: Rethinking the Impact of Different Observation Spaces on Robot Learning

Extensive experiments prove that point cloud observations are beneficial for robot learning.

Haoyi Zhu 朱皓怡, Yating Wang, Di Huang, Weicai Ye, Wanli Ouyang, Tong He

RH20T: A Robotic Dataset for Learning Diverse Skills in One-Shot

Abstract: A key challenge in robotic manipulation in open domains is how to acquire diverse and generalizable skills for robots. Recent research in one-shot imitation learning has shown promise in transferring trained policies to new tasks based on demonstrations.

Hao-Shu Fang, Hongjie Fang, Zhenyu Tang, Jirong Liu, Junbo Wang, Haoyi Zhu 朱皓怡, Cewu Lu