Haoyi Zhu 朱皓怡

Haoyi Zhu 朱皓怡

Ph.D student in Computer Science

USTC

Shanghai AI Lab

Hi, there!

I am a third-year Ph.D. student in Computer Science at University of Science and Technology of China (USTC) and Shanghai AI Laboratory, advised by Prof. Tong He, Prof. Wanli Ouyang and Prof. Xiaogang Wang. I am currently working with Dr. Enze Xie at NVIDIA. I earned my B.S. degree in Artificial Intelligence Honor Class at Shanghai Jiao Tong University (SJTU), advised by Prof. Cewu Lu. I also have had the privilege of working with Dr. Hao-Shu Fang and Dr. Jim Fan.

Research for fun and truth. My current research interests focus on World Model, Embodied AI and Spatial Intelligence. The ultimate goal of my life is to discover myself, find the truth, and change the world! Feel free to follow me on and for latest research announcements and updates!

In my personal life, I am passionate (but amateur) about football, music, literature, philosophy, traditional Chinese painting, and modern Chinese poems!

“The philosophers have only interpreted the world, in various ways. The point, however, is to change it.”

✨ News ✨

  • Jan. 2026: $\pi^3$, WinT3R, and OmniWorld accepted by ICLR 2026!
  • Oct. 2025: Won the National Scholarship and the First Price Academic Scholarship at USTC!
  • Oct. 2025: Aether won Outstanding Paper Award and Oral Presentation at ICCV 2025 RIWM workshop!
  • Sep. 2025: OmniWorld released — a multi-domain and multi-modal dataset for 4D world modeling.
  • Sep. 2025: WinT3R released — feed-forward reconstruction with online camera pose and point map prediction.
  • Jul. 2025: $\pi^3$ announced — reference-free visual geometry reconstruction. Paper, code, demo.
  • Jun. 2025: Aether and VQ-VLA accepted by ICCV 2025!
  • Jun. 2025: DeepVerse released — an auto-regressive 4D world model.
  • Feb. 2025: SPA accepted by ICLR 2025; Tra-MoE accepted by CVPR 2025.
  • Oct. 2024: SPA announced — 3D spatial awareness for embodied AI. Paper, code, models.
  • Sep. 2024: PointCloudMatters accepted by NeurIPS D&B 2024. Code.
  • Oct. 2023: PonderV2 and UniPAD announced — universal pre-training for 3D vision.
  • Jul. 2023: RH20T announced — a large-scale real-world robotic dataset.
  • Nov. 2022: MineDojo won Outstanding Paper Award at NeurIPS 2022!
  • Nov. 2022: AlphaPose accepted by TPAMI — 8.3K+ stars on GitHub.
  • Jun. 2022: MineDojo announced — open-ended agents with internet-scale knowledge in Minecraft.
Interests
  • World Model
  • Embodied AI
  • Spatial Intelligence
Education

  • B.S. in Artificial Intelligence Honor Class, 2019 - 2023

    Shanghai Jiao Tong University

  • Ph.D. in Computer Science, 2023 - Present

    University of Science and Technology of China

Publications

Visit my Google Scholar page for a comprehensive listing!

.js-id-world-model
All World Model Computer Vision Embodied AI Robot Learning Others
OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling
arXiv preprint, 2025.
... (other authors), Haoyi Zhu 朱皓怡, ... (19 authors)
OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling
WinT3R: Window-Based Streaming Reconstruction With Camera Token Pool
arXiv preprint, 2025.
Zizun Li, Jianjun Zhou, Yifan Wang, Haoyi Guo, Wenzheng Chang, Yang Zhou, Haoyi Zhu 朱皓怡, Junyi Chen, Chunhua Shen, Tong He
WinT3R: Window-Based Streaming Reconstruction With Camera Token Pool
$\pi^3$: Scalable Permutation-Equivariant Visual Geometry Learning
arXiv preprint, 2025.
Yifan Wang, Jianjun Zhou, Haoyi Zhu 朱皓怡, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang, Chunhua Shen, Tong He
$\pi^3$: Scalable Permutation-Equivariant Visual Geometry Learning
VQ-VLA: Improving Vision-Language-Action Models via Scaling Vector-Quantized Action Tokenizers
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025.
Yating Wang, Haoyi Zhu 朱皓怡, Mingyu Liu, Jiange Yang, Hao-Shu Fang, Tong He
VQ-VLA: Improving Vision-Language-Action Models via Scaling Vector-Quantized Action Tokenizers
DeepVerse: 4D Autoregressive Video Generation as a World Model
arXiv preprint, 2025.
Junyi Chen, Haoyi Zhu 朱皓怡, Xianglong He, Yifan Wang, Jianjun Zhou, Wenzheng Chang, Yang Zhou, Zizun Li, Zhoujie Fu, Jiangmiao Pang, Tong He
DeepVerse: 4D Autoregressive Video Generation as a World Model
CoMo: Learning Continuous Latent Motion from Internet Videos for Scalable Robot Learning
arXiv preprint, 2025.
Jiange Yang, Yansong Shi, Haoyi Zhu 朱皓怡, Mingyu Liu, Kaijing Ma, Yating Wang, Gangshan Wu, Tong He, Limin Wang
CoMo: Learning Continuous Latent Motion from Internet Videos for Scalable Robot Learning
Aether: Geometric-Aware Unified World Modeling
💫✨ Outstanding Paper Award ✨💫 at 1st ICCV workshop on Reliable and Interactive World Models (RIWM), 2025 & International Conference on Computer Vision (ICCV), 2025.
Haoyi Zhu 朱皓怡, Yifan Wang, Jianjun Zhou, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Chunhua Shen, Jiangmiao Pang, Tong He
Aether: Geometric-Aware Unified World Modeling
Tra-MoE: Learning Trajectory Prediction Model from Multiple Domains for Adaptive Policy Conditioning
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
Jiange Yang, Haoyi Zhu 朱皓怡, Yating Wang, Gangshan Wu, Tong He, Limin Wang
Tra-MoE: Learning Trajectory Prediction Model from Multiple Domains for Adaptive Policy Conditioning
SPA: 3D Spatial-Awareness Enables Effective Embodied Representation
The Thirteenth International Conference on Learning Representations (ICLR), 2025.
Haoyi Zhu 朱皓怡, Honghui Yang, Yating Wang, Jiange Yang, Limin Wang, Tong He
SPA: 3D Spatial-Awareness Enables Effective Embodied Representation
Point Cloud Matters: Rethinking the Impact of Different Observation Spaces on Robot Learning
Neural Information Processing Systems (NeurIPS) Dataset & Benchmark, 2024.
Haoyi Zhu 朱皓怡, Yating Wang, Di Huang, Weicai Ye, Wanli Ouyang, Tong He
Point Cloud Matters: Rethinking the Impact of Different Observation Spaces on Robot Learning
PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm
IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2025..
Haoyi Zhu 朱皓怡, Honghui Yang, Xiaoyang Wu, Di Huang, Sha Zhang, Xianglong He, Tong He, Hengshuang Zhao, Chunhua Shen, Yu Qiao, Wanli Ouyang
PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm
UniPAD: A Universal Pre-training Paradigm for Autonomous Driving
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Honghui Yang, Sha Zhang, Di Huang, Xiaoyang Wu, Haoyi Zhu 朱皓怡, Tong He, Shixiang Tang, Hengshuang Zhao, Qibo Qiu, Binbin Lin, Xiaofei He, Wanli Ouyang
UniPAD: A Universal Pre-training Paradigm for Autonomous Driving
RH20T: A Robotic Dataset for Learning Diverse Skills in One-Shot
IEEE International Conference on Robotics and Automation (ICRA), 2024.
Hao-Shu Fang, Hongjie Fang, Zhenyu Tang, Jirong Liu, Junbo Wang, Haoyi Zhu 朱皓怡, Cewu Lu
RH20T: A Robotic Dataset for Learning Diverse Skills in One-Shot
AlphaTracker: a multi-animal tracking and behavioral analysis tool
Frontiers in Behavioral Neuroscience, 2023.
Zexin Chen, Ruihan Zhang, Hao-Shu Fang, Yu E. Zhang, Aneesh Bal, Haowen Zhou, Rachel R. Rock, Nancy Padilla-Coreano, Laurel R. Keyes, Haoyi Zhu 朱皓怡, Yong-Lu Li, Takaki Komiyama, Kay M. Tye, Cewu Lu
AlphaTracker: a multi-animal tracking and behavioral analysis tool
AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI).
Hao-Shu Fang, Jiefeng Li, Hongyang Tang, Chao Xu, Haoyi Zhu 朱皓怡, Yuliang Xiu, Yong-Lu Li, Cewu Lu
AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time
X-NeRF: Explicit Neural Radiance Field for Multi-Scene 360 Insufficient RGB-D Views
IEEE Winter Conference on Applications of Computer Vision (WACV), 2023.
Haoyi Zhu 朱皓怡, Hao-Shu Fang, Cewu Lu
X-NeRF: Explicit Neural Radiance Field for Multi-Scene 360 Insufficient RGB-D Views
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
💫✨ Outstanding Paper Award ✨💫. Neural Information Processing Systems (NeurIPS) Dataset & Benchmark, 2022.
Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu 朱皓怡, Andrew Tang, De-An Huang, Yuke Zhu, Anima Anandkumar
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
Unsupervised Multi-Task Learning for 3D Subtomogram Image Alignment, Clustering and Segmentation
IEEE International Conference on Image Processing (ICIP), 2022.
Haoyi Zhu 朱皓怡, Chuting Wang, Yuanxin Wang, Zhaoxin Fan, Mostofa Rafid Uddin, Xin Gao, Jing Zhang, Xiangrui Zeng, Min Xu
Unsupervised Multi-Task Learning for 3D Subtomogram Image Alignment, Clustering and Segmentation

Experience

 
 
 
 
 
USTC
Ph.D in Computer Science
USTC
Sep 2023 – Present Shanghai, China
  • Ph.D. at USTC, majoring in Computer Science..
  • Doctoral advisor: Prof. Tong He, Prof. Wanli Ouyang and Prof. Xiaogang Wang.
  • Won the National Scholarship and the First Price Academic Scholarship at USTC.
  • Conducting AI research on 3D vision, world model, and Embodied AI. Published several first-author papers at T-PAMI, NeurIPS, ICLR, ICCV, etc.
  • Aether won the Outstanding Paper Award and presented an Oral Presentation at ICCV 2025 RIWM workshop.
 
 
 
 
 
SANA team, Efficient AI Lab, NVIDIA
Research Intern
SANA team, Efficient AI Lab, NVIDIA
Dec 2025 – Present Shanghai, China
 
 
 
 
 
Shanghai AI Lab
Research Intern
Shanghai AI Lab
Sep 2023 – Nov 2025 Shanghai, China
  • Conducting AI research on 3D vision, world model and Embodied AI.
  • Published several first-author papers at T-PAMI, NeurIPS, ICLR, ICCV, etc.
  • Aether won the Outstanding Paper Award and presented an Oral Presentation at ICCV 2025 RIWM workshop.
 
 
 
 
 
MVIG, SJTU
B.S. in Artificial Intelligence
MVIG, SJTU
Sep 2019 – Jun 2023 Shanghai, China
 
 
 
 
 
Jim Fan's Team (current GEAR Lab), NVIDIA AI Lab
Remote Research Intern
Jim Fan’s Team (current GEAR Lab), NVIDIA AI Lab
Feb 2022 – Feb 2023 Shanghai, China
 
 
 
 
 
Xu Lab, CMU
Remote Research Intern
Xu Lab, CMU
Apr 2021 – Feb 2022 Shanghai, China

Poems

Some of my modern Chinese poems

审判
Aug 28, 2025
温哥华随笔
Dec 11, 2024
石头
Dec 5, 2021
姐姐，我在一无所有的冬天里想你
Dec 1, 2021
孤独:永恒
Nov 20, 2021
See all

Contact