Huaizu Jiang

Assistant Professor
Khoury College of Computer Sciences
Northeastern University
Email: h.jiang AT northeastern.edu

CVGoogle ScholarGitHub

About Me

I am an Assistant Professor in the Khoury College of Computer Sciences at Northeastern University. I have broad research interests in computer vision, computational photography, natural language processing, and machine learning. My long-term research aims to teach machines to develop visual intelligence in a manner analogous to humans. In the short term, my research goal is to create smart visual perception tools to improve people’s life experiences of using cameras.

Prior to joining Northeastern University, I was a Postdoc Researcher at Caltech and a Visiting Researcher at NVIDIA working with Prof. Anima Anandkumar. I obtained my Ph.D. from UMass Amherst, where I was fortunately advised by Prof. Erik Learned-Miller. When pursuing my Ph.D., I did joyful internships at NVIDIA Research and Facebook AI Research (FAIR). I got my M.E. and B.E. degrees from Xi'an Jiaotong University in 2009 and 2012, respectively.

Hiring

I am looking for highly motivated students to join my lab. Check out opportunities for more details.

Group

PhD students Master's students, interns, and visitors Undergrads

Teaching

CS 5330: Pattern Recognition and Computer Vision [Fall 2021][Fall 2022]

CS 7150: Deep Learning [Spring 2022][Spring 2023]


Preprints

(*: Equal contribution, †: Equal advising)
SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency
Yiming Xie*, Chun-Han Yao*, Vikram Voleti, Huaizu Jiang†, Varun Jampani†
in arXiv 2024.
[PDF][Code][Project]
HouseCrafter: Lifting Floorplans to 3D Scenes with 2D Diffusion Model
Hieu T. Nguyen*, Yiwen Chen*, Vikram Voleti, Varun Jampani, Huaizu Jiang
in arXiv 2024.
[PDF][Code (coming soon)][Project]
HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Models
Xiaogang Peng*, Yiming Xie*, Zizhao Wu, Varun Jampani, Deqing Sun, Huaizu Jiang
in arXiv 2023.
[PDF][Code][Project]
Diagnosing Human-object Interaction Detectors
Fangrui Zhu, Yiming Xie, Weidi Xie, Huaizu Jiang
in arXiv 2023.
[PDF][Code]
A Strong Baseline for Point Cloud Registration via Direct Superpoints Matching
Aniket Gupta, Yiming Xie, Huaizu Jiang, Hanumant Singh
in arXiv 2023.
[PDF][Code]

Publications

(*: Equal contribution, †: Equal advising)
Towards Flexible Visual Relationship Segmentation
Fangrui Zhu, Jianwei Yang, Huaizu Jiang
in NeurIPS 2024.
[PDF][Code (coming soon)]
SMooDi: Stylized Motion Diffusion Model
Lei Zhong, Yiming Xie, Varun Jampani, Deqing Sun, Huaizu Jiang
in ECCV 2024.
[PDF][Code]
ODTFormer: Efficient Obstacle Detection and Tracking with Stereo Cameras Based on Transformer
Tianye Ding*, Hongyu Li*, Huaizu Jiang
in IROS 2024.
[PDF][Code][Project]
StereoNavNet: Learning to Navigate using Stereo Cameras with Auxiliary Occupancy Voxels
Hongyu Li, Taskin Padir, Huaizu Jiang
in IROS 2024.
[PDF][Code (coming soon)]
NeuFlow: Real-time, High-accuracy Optical Flow Estimation on Robots Using Edge Devices
Zhiyong Zhang, Huaizu Jiang, Hanumant Singh
in IROS 2024.
[PDF][Code]
Vessel-aware Aneurysm Detection Using Multi-scale Deformable 3D Attention
Alberto M. Ceballos-Arroyo, Hieu T. Nguyen, Fangrui Zhu, Shrikanth M. Yadav, Jisoo Kim, Lei Qin, Geoffrey Young, Huaizu Jiang
in MICCAI 2024.
[PDF (coming soon)][Code (coming soon)]
Zero-shot Referring Expression Comprehension via Structural Similarity Between Images and Captions
Zeyu Han, Fangrui Zhu, Qianru Lao, Huaizu Jiang
in CVPR 2024.
[PDF][Code]
SportsSloMo: A New Benchmark and Baselines for Human-centric Video Frame Interpolation
Jiaben Chen, Huaizu Jiang
in CVPR 2024.
[PDF][Code][Data]
OmniControl: Control Any Joint at Any Time for Human Motion Generation
Yiming Xie, Varun Jampani, Lei Zhong, Deqing Sun, Huaizu Jiang
in ICLR 2024.
[PDF][Code][Project]
E(2)-Equivariant Graph Planning for Navigation
Linfeng Zhao*, Hongyu Li*, Taskin Padir, Huaizu Jiang†, Lawson Wong
in IEEE Robotics and Automation Letters (RA-L) 2024 (Oral in IROS 2024).
[PDF][Code (coming soon)][Project]
Pixel-Aligned Recurrent Queries for Multi-View 3D Object Detection
Yiming Xie, Huaizu Jiang, Georgia Gkioxari†, Julian Straub
in ICCV 2023.
[PDF][Code][Project Page]
StereoVoxelNet: Real-Time Obstacle Detection Based on Occupancy Voxels from a Stereo Camera Using Deep Neural Networks
Hongyu Li, Zhengang Li, Neset Unver Akmandor, Huaizu Jiang, Yanzhi Wang, Taskin Padir
in ICRA 2023.
[PDF][Code][Project Page]
DCVNet: Dilated Cost Volume Networks for Fast Optical Flow
Huaizu Jiang, Erik Learned-Miller
in WACV 2023.
[PDF][Code]
PlanarRecon: Real-time 3D Plane Detection and Reconstruction from Posed Monocular Videos
Yiming Xie, Matheus Gadelha, Fengting Yang, Xiaowei Zhou, Huaizu Jiang
in CVPR 2022.
[PDF][Code][Overview Video][Project Page]
Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactions
Huaizu Jiang*, Xiaojian Ma*, Weili Nie, Zhiding Yu, Yuke Zhu, Anima Anandkumar
in CVPR 2022 (Oral).
[PDF][Code & Data]
RelViT: Concept-guided Vision Transformer for Visual Relational Reasoning
Xiaojian Ma, Weili Nie, Zhiding Yu, Huaizu Jiang, Chaowei Xiao, Yuke Zhu, Song-Chun Zhu, Anima Anandkumar
in ICLR 2022.
[PDF][Code][Slides]
In Defense of Grid Features for Visual Question Answering
Huaizu Jiang, Ishan Misra, Marcus Rohrbach, Erik Learned-Miller, Xinlei Chen
in CVPR 2020.
[PDF][Code]
Winner of the VQA Challenge 2020.
SENSE: a Shared Encoder for Scene flow Estimation
Huaizu Jiang, Deqing Sun, Varun Jampani, Zhaoyang Lv, Erik Learned-Miller, Jan Kautz
in ICCV 2019 (Oral).
[PDF][Supp Mat][Code]
Half&Half: New Tasks and Benchmarks for Studying Visual Common Sense
Asish Singh*, Hang Su*, SouYoung Jin, Huaizu Jiang, Chetan Manjesh, Geng Luo, Ziwei He, Li Hong, Erik Learned-Miller, Rosemary Cowell
in CVPR 2019, Workshop on Vision Meets Cognition.
[PDF]
Automatic Adaptation of Object Detectors to New Domains Using Self-training
Aruni Roy Chowdhury, Prithvijit Chakrabarty, Ashish Singh, SouYoung Jin, Huaizu Jiang, Liangliang Cao, Erik Learned-Miller
in CVPR 2019.
[PDF][Supp Mat][Project][Code]
Self-Supervised Relative Depth Learning for Urban Scene Understanding
Huaizu Jiang, Gustav Larsson, Michael Maire, Greg Shakhnarovich, Erik Learned-Miller
in ECCV 2018.
[PDF][Supp Mat][Project] A short version appeared in CVPR 2017 SUN workshop.
Unsupervised Hard Example Mining from Videos for Improved Object Detection
SouYoung Jin*, Aruni Roy Chowdhury*, Huaizu Jiang, Ashish Singh, Aditya Prasad, Deep Chakraborty, Erik Learned-Miller
in ECCV 2018.
[PDF][Code][Project]
Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation
Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, Jan Kautz
in CVPR 2018 (Spotlight).
[PDF][Video1][Video2][CVPR Spotlight Video][Project][UCF101 Results]
Reasoning about Fine-grained Attribute Phrases using Reference Games
Jong-Chyi Su*, Chenyun Wu*, Huaizu Jiang, Subhransu Maji
in ICCV 2017.
[PDF][Code and Data][Project]
Face Detection with the Faster R-CNN
Huaizu Jiang, Erik Learned-Miller
in FG (IEEE Conference on Automatic Face and Gesture Recognition) 2017.
[PDF][Code]
Salient Object Detection: A Benchmark
Ali Borji, Ming-Ming Cheng, Huaizu Jiang, Jia Li
in TIP 2015.
[PDF][Project]


Salient Object Detection: A Discriminative Regional Feature Integration Approach
Huaizu Jiang, Jingdong Wang, Zejian Yuan, Yang Wu, Nanning Zheng, Shipeng Li
in CVPR, 2013 and IJCV 2017.
[CVPR version][IJCV version] [MATLAB Code] [C++ Code] [Groundtruth of MSRA-B] [Project]


Automatic Salient Object Segmentation Based on Context and Shape Prior
Huaizu Jiang, Jingdong Wang, Zejian Yuan, Tie Liu, Nanning Zheng, Shipeng Li
in BMVC 2011.
[PDF] [Code]