Hang Zhou

Ph.D Department of Computer Vision Technology (VIS) Baidu Inc. zhouhang09@baidu dot com zhouhang@link.cuhk.edu.hk Github    Scholar

About me

I am currently a senior researcher at Department of Computer Vision Technology ( VIS), Baidu Inc, leading the R&D of virtual human creation. I obtained my Ph.D. degree from Multimedia Lab (MMLab), The Chinese University of Hong Kong (CUHK) in 2021, supervised by Prof. Xiaogang Wang and Prof. Ziwei Liu. I received my bachelor's degree in Acoustics from the School of Physics at Nanjing University (NJU) in 2017.

My research interests include deep learning and its applications on audio-visual learning, image/video generation, and virtual human creations. Please drop me an email if you are interested in internship or full-time opportunities.

News

• [2024-09] One papers accepted to SIGGRAPH Asia 2024.

• [2024-07] Two papers accepted to ECCV 2024.

• [2024-01] One paper accepted to ICLR 2024.

• [2023-07] One paper accepted to ICCV 2023.

• [2023-04] One paper accepted to SIGGRAPH 2023.

• [2023-03] One paper accepted to CVPR 2023.

• [2022-12] One paper accepted to AAAI 2023.

• [2022-10] Three papers accepted to WACV 2023.

• [2022-09] Three papers accepted, one to SIGGRAPH Asia 2022 and two to NeurIPS 2022.

• [2022-07] Four papers accepted to ECCV 2022.

• [2022-05] One paper accepted to SIGGRAPH 2022.

• [2022-03] Three papers accepted to CVPR 2022.

• [2021-12] Two papers accepted to AAAI 2022.

• [2021-04] One paper accepted to IJCAI 2021.

• [2021-03] Three papers accepted to CVPR 2021!

• [2021-01] One paper accepted to CHI 2021.

• [2020-10] I am among the Top 10% High Score Reviewers of NeurIPS 2020.

• [2020-07] Two papers accepted to ECCV 2020.

• [2020-06] I am awarded Outstanding Reviewer of CVPR 2020.

• [2020-02] One paper accepted to CVPR 2020.

• [2019-07] Two papers accepted to ICCV 2019.

• [2018-11] One paper accepted to AAAI 2019 as Oral.

Publications

*Equal contribition, †Corresponding author

TALK-Act: Enhance Textural-Awareness for 2D Speaking Avatar Reenactment with Diffusion Model Jiazhi Guan, Quanwei Yang, Kaisiyuan Wang, Hang Zhou†, Shengyi He, Zhiliang Xu, Haocheng Feng, Errui Ding, Jingdong Wang, Hongtao Xie, Youjian Zhao†, Ziwei Liu SIGGRAPH Asia 2024 Conference Proceedings. PDF   Project

ReSyncer: Rewiring Style-based Generator for Unified Audio-Visually Synced Facial Performer Jiazhi Guan*, Zhiliang Xu*, Hang Zhou†, Kaisiyuan Wang, Shengyi He, Zhanwang Zhang, Borong Liang, Haocheng Feng, Errui Ding, Jingtuo Liu, Jingdong Wang, Youjian Zhao†, Ziwei Liu European Conference on Computer Vision (ECCV) 2024 (Oral Presentation). PDF   Project

3D-Aware Text-driven Talking Avatar Generation Xiuzhe Wu, Yang-Tian Sun, Handi Chen, Hang Zhou, Jingdong Wang, Zhengzhe Liu, Xiaojuan Qi European Conference on Computer Vision (ECCV) 2024. PDF

DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, Gang Zeng International Conference on Learning Representations (ICLR) 2024 (Oral Presentation). PDF   Project   Code

ReliTalk: Relightable Talking Portrait Generation from a Single Video Haonan Qiu, Zhaoxi Chen, Yuming Jiang, Hang Zhou, Xiangyu Fan, Lei Yang, Wayne Wu, Ziwei Liu International Journal of Computer Vision (IJCV) 2024. PDF   Project   Code

Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement Jiaxiang Tang, Hang Zhou, Xiaokang Chen, Tianshu Hu, Errui Ding, Jingdong Wang, Gang Zeng International Conference on Computer Vision (ICCV) 2023. PDF   Project   Code

Efficient Video Portrait Reenactment via Grid-based Codebook Kaisiyuan Wang, Hang Zhou†, Qianyi Wu, Jiaxiang Tang, Zhiliang Xu, Borong Liang, Tianshu Hu, Errui Ding, Jingtuo Liu, Ziwei Liu, Jingdong Wang SIGGRAPH 2023 Conference Proceedings. PDF   Project

StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-based Generator Jiazhi Guan*, Zhanwang Zhang*, Hang Zhou†, Tianshu Hu†, Kaisiyuan Wang, Dongliang He, Haocheng Feng, Jingtuo Liu, Errui Ding, Ziwei Liu, Jingdong Wang IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023. PDF   Project

Robust Video Portrait Reenactment via Personalized Representation Quantization Kaisiyuan Wang, Changcheng Liang, Hang Zhou†, Jiaxiang Tang, Qianyi Wu, Dongliang He, Zhibin Hong, Jingtuo Liu, Errui Ding, Ziwei Liu, Jingdong Wang AAAI Conference on Artificial Intelligence (AAAI) 2023. PDF   Project

ReEnFP: Detail-Preserving Face Reconstruction by Encoding Facial Priors Yasheng Sun, Jiangke Lin, Hang Zhou, Zhiliang Xu, Dongliang He, Hideki Koike IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2023. PDF   Project

SeCo: Separating Unknown Musical Visual Sounds with Consistency Guidance Xinchi Zhou, Dongzhan Zhou, Wanli Ouyang, Hang Zhou, Di Hu IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2023. PDF

Exploiting Visual Context Semantics for Sound Source Localization Xinchi Zhou, Dongzhan Zhou, Di Hu, Hang Zhou, Wanli Ouyang IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2023. PDF

Masked Lip-Sync Prediction by Audio-Visual Contextual Exploitation in Transformers Yasheng Sun*, Hang Zhou*, Kaisiyuan Wang, Qianyi Wu, Zhibin Hong, Jingtuo Liu, Errui Ding, Jingdong Wang, Ziwei Liu, Hideki Koike SIGGRAPH Asia 2022 Conference. PDF   Project

Make Your Brief Stroke Real and Stereoscopic: 3D-Aware Simplified Sketch to Portrait Generation Yasheng Sun*, Qianyi Wu*, Hang Zhou*, Kaisiyuan Wang, Tianshu Hu, Chen-Chieh Liao, Dongliang He, Jingtuo Liu, Errui Ding, Jingdong Wang, Shio Miyafuji, Ziwei Liu, Hideki Koike International Conference on Multimodal Interaction (ICMI) 2023 Oral. PDF   Project

Audio-Driven Co-Speech Gesture Video Generation Xian Liu, Qianyi Wu, Hang Zhou, Yuanqi Du, Wayne Wu, Dahua Lin, Ziwei Liu NeurIPS 2022 (Spotlight Presentation). PDF   Project   Code

Delving into sequential patches for DeepFake detection Jiazhi Guan, Hang Zhou, Zhibin Hong, Errui Ding, Jingdong Wang, Chengbin Quan, Youjian Zhao NeurIPS 2022. PDF

StyleSwap: Style-Based Generator Empowers Robust Face Swapping Zhiliang Xu*, Hang Zhou*†, Zhibin Hong, Ziwei Liu, Jiaming Liu, Zhizhi Guo, Junyu Han, Jingtuo Liu, Errui Ding, Jingdong Wang European Conference on Computer Vision (ECCV) 2022. PDF   Project   Code

Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation Xian Liu, Yinghao Xu, Qianyi Wu, Hang Zhou, Wayne Wu, Bolei Zhou European Conference on Computer Vision (ECCV) 2022 (Oral Presentation). PDF   Project   Code

Joint-Modal Label Denoising for Weakly-Supervised Audio-Visual Video Parsing Haoyue Cheng, Zhaoyang Liu, Hang Zhou, Chen Qian, Wayne Wu, Limin Wang European Conference on Computer Vision (ECCV) 2022. PDF   Code

TokenMix: Rethinking Image Mixing for Data Augmentation in Vision Transformers Jihao Liu, Boxiao Liu, Hang Zhou, Hongsheng Li, Yu Liu European Conference on Computer Vision (ECCV) 2022. PDF   Code

EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model Xinya Ji, Hang Zhou, Kaisiyuan Wang, Qianyi Wu, Wayne Wu, Feng Xu, Xun Cao SIGGRAPH 2022 Conference Proceedings. PDF   Project  Code

Few-Shot Head Swapping in the Wild Changyong Shu, Hemao Wu, Hang Zhou†, Jiaming Liu†, Zhibin Hong, Changxing Ding, Junyu Han, Jingtuo Liu, Errui Ding, Jingdong Wang IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022 (Oral Presentation). PDF   Project  Code

Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation Xian Liu, Qianyi Wu, Hang Zhou, Yinghao Xu, Rui Qian, Xinyi Lin, Xiaowei Zhou, Wayne Wu, Bo Dai, and Bolei Zhou IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022. PDF   Project  Code

Expressive Talking Head Generation with Granular Audio-Visual Control Borong Liang*, Yan Pan*, Zhizhi Guo†, Hang Zhou†, Zhibin Hong, Xiaoguang Han, Junyu Han, Jingtuo Liu, Errui Ding, Jingdong Wang IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022. PDF

Visual Sound Localization in the Wild by Cross-Modal Interference Erasing Xian Liu*, Rui Qian*, Hang Zhou*, Di Hu, Weiyao Lin, Ziwei Liu, Bolei Zhou, Xiaowei Zhou AAAI Conference on Artificial Intelligence (AAAI) 2022. PDF   Code

SepFusion: Finding Optimal Fusion Structures for Visual Sound Separation Dongzhan Zhou, Xinchi Zhou, Di Hu, Hang Zhou, Lei Bai, Ziwei Liu, Wanli Ouyang. AAAI Conference on Artificial Intelligence (AAAI) 2022. PDF

Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation Hang Zhou, Yasheng Sun, Wayne Wu, Chen Change Loy, Xiaogang Wang, Ziwei Liu IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021. PDF   Project   Code

Visually Informed Binaural Audio Generation without Binaural Audios Xudong Xu*, Hang Zhou*, Ziwei Liu, Bo Dai, Xiaogang Wang, Dahua Lin IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021. PDF   Project   Code

Audio-Driven Emotional Video Portraits Xinya Ji, Hang Zhou, Kaisiyuan Wang, Wayne Wu, Chen Change Loy, Xun Cao, Feng Xu IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021. PDF   Project

Speech2Talking-Face: Inferring and Driving a Face with Synchronized Audio-Visual Representation Yasheng Sun*, Hang Zhou*, Ziwei Liu, Hideki Koike International Joint Conference on Artificial Intelligence (IJCAI) 2021. PDF   Project   Code

PTeacher: a Computer-Aided Personalized Pronunciation Training System with Exaggerated Audio-Visual Corrective Feedback Yaohua Bu*, Tianyi Ma*, Weijun Li, Hang Zhou, Jia Jia, Shengqi Chen, Kaiyuan Xu, Dachuan Shi, Haozhe Wu, Zhihan Yang, Kun Li, Zhiyong Wu, Yuanchun Shi, Xiaobo Lu, Ziwei Liu ACM CHI Conference on Human Factors in Computing Systems (CHI) 2021. PDF   Demo

Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation Hang Zhou*, Xudong Xu*, Dahua Lin, Xiaogang Wang, Ziwei Liu European Conference on Computer Vision (ECCV) 2020. PDF   Project   Code

Discriminability Distillation in Group Representation Learning Manyuan Zhang, Guanglu Song, Hang Zhou, Yu Liu European Conference on Computer Vision (ECCV) 2020. PDF

Rotate-and-Render: Unsupervised Photorealistic Face Rotation from Single-View Images Hang Zhou*, Jihao Liu*, Ziwei Liu, Yu Liu, Xiaogang Wang IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020. PDF   Code

Vision-Infused Deep Audio Inpainting Hang Zhou, Ziwei Liu, Xudong Xu, Ping Luo, Xiaogang Wang International Conference on Computer Vision (ICCV) 2019. PDF   Project   Code

A Graph-based Framework to Bridge Movies and Synopses Yu Xiong, Qingqiu Huang, Lingfeng Guo, Hang Zhou, Bolei Zhou, Dahua Lin International Conference on Computer Vision (ICCV) 2019 (Oral Presentation). PDF   Project   Code

Talking Face Generation by Adversarially Disentangled Audio-Visual Representation Hang Zhou, Yu Liu, Ziwei Liu, Ping Luo, Xiaogang Wang AAAI Conference on Artificial Intelligence (AAAI) 2019 (Oral Presentation). PDF   Project   Code

Professional Activities

• Organizer of ECCV 2020 SenseHuman Workshop
• Journal Reviewer of IJCV, Transactions on Audio Speech and Language Processing, Neurocomputing, TPAMI, etc.
• Conference Reviewer of ICCV, CVPR, ECCV, ICML, AAAI, NeurIPS, ICLR, Siggraph, etc.

Collaborators

• I am fortunate to have closely collaborated with Xudong Xu, Qianyi Wu, Dr. Wayne Wu, Dr. Yu Liu and Prof. Ping Luo.
• I am also fortunate to have provided help to talented mentees including Xian Liu, Kaisiyuan Wang, Yasheng Sun, Jiaxiang Tang, Xinya Ji, and Jihao Liu.

Teaching Experience

Teaching Assistant of the following courses in The Chinese University of Hong Kong:

• ELEG5491, Introduction to Deep Learning , Spring 2019.

• ELEG5760, Machine Learning for Signal Processing Applications, Fall 2018.

• ENGG2420B, Complex Analysis and Differential Equations for Engineers, Fall 2018.

• ENGG1110, Problem Solving By Programming, Spring 2018.

• ENGG2420B, Complex Analysis and Differential Equations for Engineers, Fall 2017.