I am Kaizhi Zheng (郑恺之), a fourth-year CSE Ph.D. candidate at the University of California, Santa Cruz, working with Prof. Xin (Eric) Wang. Previously I was an M.S’ student at the University of Michigan, Ann Arbor, working with Prof. Chad Jenkins.

Research Interests

My research interests focus on multimodal understanding, generation, and embodied AI. My research goal is to establish intelligent agents who can understand and interact with the environment. Feel free to email me if you have common interests in this area. I will be happy to talk with you!

News

• [2024.06] I will join Adobe Research as a full-time research intern in 2024 summer.
• [2024.02] I’m joining Microsoft as a part-time research intern now!
• [2023.10] Our paper R2H has been accepted by EMNLP 2023!
• [2023.06] I will join Samsung Research America (SRA) as a research intern in 2023 summer.
• [2023.06] Our SlugJARVIS team wins the top three in the first Alexa Prize SimBot Challenge! [Media Coverage]
• [2023.04] Our paper ESC has been accepted by ICML 2023!
• [2023.03] Our Sage team has been selected to work on Alexa Prize Taskbot Challenge 2!
• [2022.09] Our paper VLMbench has been accepted by NeurIPS 2022!
• [2022.05] Our SlugJARVIS team won the Alexa Prize SimBot Public Benchmark Challenge! [Media Coverage]

Publication & Preprints

• EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing
  Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, Kevin Lin, Jianfeng Wang, Lijuan Wang, Xin Eric Wang
  Preprint
  

• MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos
  Xuehai He, Weixi Feng, Kaizhi Zheng, Yujie Lu, Wanrong Zhu, Jiachen Li, Yue Fan, Jianfeng Wang, Linjie Li, Zhengyuan Yang, Kevin Lin, William Yang Wang, Lijuan Wang, Xin Eric Wang
  Preprint
  

• Toffee: Efficient Million-Scale Dataset Construction for Subject-Driven Text-to-Image Generation
  Yufan Zhou, Ruiyi Zhang, Kaizhi Zheng, Nanxuan Zhao, Jiuxiang Gu, Zichao Wang, Xin Eric Wang, Tong Sun
  Preprint
  

• MiniGPT-5: Interleaved Vision-and-Language Generation via Generative Vokens
  Kaizhi Zheng*, Xuehai He*, Xin Eric Wang
  Preprint
  

• R2H: Building Multimodal Navigation Helpers that Respond to Help
  Yue Fan, Jing Gu, Kaizhi Zheng, Xin Eric Wang
  EMNLP 2023
  

• ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigation
  Kaiwen Zhou, Kaizhi Zheng, Connor Pryor, Yilin Shen, Hongxia Jin, Lise Getoor, Xin Eric Wang
  ICML 2023
  

• JARVIS: A Neuro-Symbolic Commonsense Reasoning Framework for Conversational Embodied Agents
  Kaizhi Zheng*, Kaiwen Zhou*, Jing Gu*, Yue Fan*, Jialu Wang*, Zonglin Di, Xuehai He, Xin Eric Wang
  Preprint
  

• VLMbench: A Compositional Benchmark for Vision-and-Language Manipulation
  Kaizhi Zheng, Xiaotong Chen, Odest Chadwicke Jenkins, Xin Eric Wang
  NeurIPS 2022 (Track Datasets and Benchmarks)
  

• Manipulation-Oriented Object Perception in Clutter through Affordance Coordinate Frames
  Xiaotong Chen, Kaizhi Zheng, Zhen Zeng, Shreshtha Basu, James Cooney, Jana Pavlasek, Odest Chadwicke Jenkins
  Humanoids 2022
  

• Composable Causality in Semantic Robot Programming
  Emily Sheetz, Xiaotong Chen, Zhen Zeng, Kaizhi Zheng, Qiuyu Shi, Odest Chadwicke Jenkins
  ICRA 2022
  

Education

• B.S. in EE, Huazhong University of Science and Technology, 2015-2019
• M.S. in ECE, University of Michigan, Ann Arbor, 2019-2021
• Ph.D. in CSE, University of California, Santa Cruz, 2021-Present