Hanbyul (Han) Joo

Associate Professor
Department of Computer Science and Engineering
Seoul National University (SNU)
Email: hbjoo-at-snu-ac-kr
Office: Rm 324 Bldg 302

Google Scholar CV

I am an associate professor at Seoul National University (SNU) in the Department of Computer Science and Engineering. Before joining SNU, I was a Research Scientist at Facebook AI Research (FAIR), Menlo Park. I finished my Ph.D. in the Robotics Institute at Carnegie Mellon University, where I worked with Yaser Sheikh. During my PhD, I interned at Facebook Reality Labs, Pittsburgh (Summer and Fall, 2017) and Disney Research Zurich (Summer, 2015). I received my M.S. in EE and B.S. in CS at KAIST, Korea. I am a recipient of the Samsung Scholarship and the CVPR Best Student Paper Award in 2018.

The goal of my research is to endow machines and robots with the ability to perceive and understand human behaviors in 3D. Ultimately, I dream to build an AI system that can behave like humans in new environments and can interact with humans using a broad channel of nonverbal signals (kinesic signals or body languages). I pursue this direction by creating new tools in computer vision, machine learning, and computer graphics.

• Apr 2026The code of our Dexterous World Models paper is available: Link
• Apr 2026My student Hyunsoo Cha will be starting an internship at NVIDIA Research. Congrats!
• Mar 2026I will be giving talks in CVPR 2026 workshops: Agents in Interaction, from Humans to Robots, 4D Digital Twins
• Feb 2026Two papers got accepted in CVPR 2026
• Jan 2026Two papers got accepted in ICLR 2026
• Oct 2025I was awarded the Okawa Research Grant
• Aug 2025I will serve as an area chair for ICLR 2026, 3DV2026, and as a lead area chair for CVPR 2026 and ECCV 2026
• Aug 2025I gave talks in Arirang TV: The Evolution of AI and AI: Reading Humans
• Jul 2025I will be giving a talk in CORL 2025 workshop: Human to Robot (H2R)
• Jun 2025I am co-organizing a RSS workshop on Dexterous Manipulation: Learning and Control with Diverse Data
• May 2025I will be giving talks in CVPR 2025 workshops: Global 3D Human Poses Workshop, 3D Humans Workshop, and Agents in Interactions, from Humans to Robots
• Mar 2025I will be giving a talk in an ICRA workshop: From Computer Vision-based Motion Tracking to Wearable Assistive Robot Control
• Jan 2025I will serve as an area chair for CVPR 2025, ICCV 2025, and NeurIPS 2025

	HRDexDB
	SNU ParaHome
	FrankMocap
	CMU Panoptic Studio Dataset
	OpenPose Library

	SNU ParaDex: Capturing Dexterous Robot Hand Manipulation
	SNU ParaDex2
	SNU RobotHome: Capturing Human-Robot Interactions in A Home Setting
	SNU ParaHome

* indicates equal contributions

• HRDexDB: A Large-Scale Dataset of Dexterous Human and Robotic Hand Grasps

  Jongbin Lim*, Taeyun Ha*, Mingi Choi, Jisoo Kim, Byungjun Kim, Subin Jeon, Hanbyul Joo

  Preprint 2026

  Paper Project Page
• 
  Text-Guided 6D Object Pose Rearrangement via Closed-Loop VLM Agents

  Sangwon Baik, Gunhee Kim, Mingi Choi, Hanbyul Joo

  Preprint 2026

  Paper Project Page Code
• Dexterous World Models

  Byungjun Kim*, Taeksoo Kim*, Junyoung Lee, Hanbyul Joo

  CVPR 2026

  Paper Project Page Code
• Vanast: Virtual Try-On with Human Image Animation via Synthetic Triplet Supervision

  Hyunsoo Cha, Wonjung Woo, Byungjun Kim, Hanbyul Joo

  CVPR 2026  Highlight

  Paper Project Page
• Durian: Dual Reference Image-Guided Portrait Animation with Attribute Transfer

  Hyunsoo Cha, Byungjun Kim, Hanbyul Joo

  ICLR 2026

  Paper Project Page
• Target-Aware Video Diffusion Models

  Taeksoo Kim, Hanbyul Joo

  ICLR 2026

  Paper Project Page
• 
  GraspDiffusion: Synthesizing Realistic Whole-body Hand-Object Interaction

  Patrick Kwon, Chen Chen, Hanbyul Joo

  WACV 2026

  Paper Project Page
• OmniEgoCap: Device-Agnostic Sequence-Level Egocentric Motion Reconstruction

  Kyungwon Cho, Hanbyul Joo

  Preprint 2025

  Paper Project Page
• 
  Audio Driven Real-Time Facial Animation for Social Telepresence

  Jiye Lee, Chenghui Li, Linh Tran, Shih-En Wei, Jason Saragih, Alexander Richard, Hanbyul Joo†, Shaojie Bai†

  SIGGRAPH Asia 2025

  Paper Project Page
• 
  Learning to Generate Human-Human-Object Interactions from Textual Descriptions

  Jeonghyeon Na*, Sangwon Baik*, Inhee Lee, Junyoung Lee, Hanbyul Joo

  NeurIPS 2025

  Paper Project Page
• 
  DAViD: Modeling Dynamic Affordance of 3D Objects using Pre-trained Video Diffusion Models

  Hyeonwoo Kim, Sangwon Baik, Hanbyul Joo

  ICCV 2025

  Paper Project Page
• 
  Learning 3D Object Spatial Relationships from Pre-trained 2D Diffusion Models

  Sangwon Baik, Hyeonwoo Kim, Hanbyul Joo

  ICCV 2025

  Paper Project Page
• HairCUP: Hair Compositional Universal Prior for 3D Gaussian Avatars

  Byungjun Kim, Shunsuke Saito, Giljoo Nam, Tomas Simon, Jason Saragih, Hanbyul Joo†, Junxuan Li†

  ICCV 2025  Oral presentation

  Paper Project Page
• 
  ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactions

  Jeonghwan Kim*, Jisoo Kim*, Jeonghyeon Na, Hanbyul Joo

  CVPR 2025

  Paper Project Page Code/Dataset
• PERSE: Personalized 3D Generative Avatars from A Single Portrait

  Hyunsoo Cha, Inhee Lee, Hanbyul Joo

  CVPR 2025

  Paper Project Page Code
• 
  Learning to Transfer Human Hand Skills for Robot Manipulations

  Sungjae Park*, Seungho Lee*, Mingi Choi*, Jiye Lee, Jeonghwan Kim, Jisoo Kim, Hanbyul Joo

  CORL X-Embodiment Workshop 2024

  Paper Project Page
• 
  Beyond the Contact: Discovering Comprehensive Affordance for 3D Objects from Pre-trained 2D Diffusion Models

  Hyeonwoo Kim*, Sookwan Han*, Patrick Kwon, Hanbyul Joo

  ECCV 2024  Oral presentation — 200/8585 = 2.3%

  Paper Project Page
• 
  Guess The Unseen: Dynamic 3D Scene Reconstruction from Partial 2D Glimpses

  Inhee Lee, Byungjun Kim, Hanbyul Joo

  CVPR 2024

  Paper Project Page Code
• 
  PEGASUS: Personalized Generative 3D Avatars with Composable Attributes

  Hyunsoo Cha, Byungjun Kim, Hanbyul Joo

  CVPR 2024

  Paper Project Page Code
• 
  GALA: Generating Animatable Layered Assets from a Single Scan

  Taeksoo Kim*, Byungjun Kim*, Shunsuke Saito, Hanbyul Joo

  CVPR 2024

  Paper Project Page Code
• 
  Mocap Everyone Everywhere: Lightweight Motion Capture With Smartwatches and a Head-Mounted Camera

  Jiye Lee, Hanbyul Joo

  CVPR 2024

  Paper Project Page Code
• 
  CHORUS: Learning Canonicalized 3D Human-Object Spatial Relations from Unbounded Synthesized Images

  Sookwan Han, Hanbyul Joo

  ICCV 2023  Oral presentation — 152/8260 = 1.8%

  Paper Project Page Code
• 
  Chupa: Carving 3D Clothed Humans from Skinned Shape Priors using 2D Diffusion Probabilistic Models

  Byungjun Kim*, Patrick Kwon*, Kwangho Lee, Myunggi Lee, Sookwan Han, Daesik Kim, Hanbyul Joo

  ICCV 2023  Oral presentation — 152/8260 = 1.8%

  arXiv Project Page Code
• 
  NCHO: Unsupervised Learning for Neural 3D Composition of Humans and Objects

  Taeksoo Kim, Shunsuke Saito, Hanbyul Joo

  ICCV 2023

  arXiv Project Page Code
• 
  Locomotion-Action-Manipulation: Synthesizing Human-Scene Interactions in Complex 3D Environments

  Jiye Lee, Hanbyul Joo

  ICCV 2023

  arXiv Project Page Code
• 
  BANMo: Building Animatable 3D Neural Models from Many Casual Videos

  Gengshan Yang, Minh Vo, Natalia Neverova, Deva Ramanan, Andrea Vedaldi, Hanbyul Joo

  CVPR 2022  Oral presentation — 344/8161 = 4.2%

  arXiv Project Page
• 
  Ego4D: Around the World in 3,000 Hours of Egocentric Video

  Grauman et al.

  CVPR 2022  Oral presentation — 344/8161 = 4.2%

  Best Paper Award Finalist

  arXiv Project Page
• 
  Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motion

  Evonne Ng, Hanbyul Joo, Liwen Hu, Hao Li, Trevor Darrell, Angjoo Kanazawa, Shiry Ginosar

  CVPR 2022

  arXiv Project Page Code+Data
• 
  Body2Hands: Learning to Infer 3D Hands from Conversational Gesture Body Dynamics

  Evonne Ng, Shiry Ginosar, Trevor Darrell, Hanbyul Joo

  CVPR 2021

  arXiv Project Page
• 
  3D Multi-bodies: Fitting Sets of Plausible 3D Human Models to Ambiguous Image Data

  Benjamin Biggs, Seb Ehrhadt, Hanbyul Joo, Ben Graham, Andrea Vedaldi, David Novotny

  NeurIPS 2020  Spotlight

  arXiv Project Page
• 
  Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild

  Jason Y. Zhang*, Sam Pepose*, Hanbyul Joo, Deva Ramanan, Jitendra Malik, Angjoo Kanazawa

  ECCV 2020

  arXiv Project Page
• 
  PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization

  Shunsuke Saito, Tomas Simon, Jason Saragih, Hanbyul Joo

  CVPR 2020  Oral presentation — 335/6424 = 5.2%

  arXiv Project Page Code Colab Demo
• 
  You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactions

  Evonne Ng, Donglai Xiang, Hanbyul Joo, Kristen Grauman

  CVPR 2020  Oral presentation — 335/6424 = 5.2%

  arXiv Project Page Code/Data
• 
  Single-Network Whole-Body Pose Estimation

  Gines Hidalgo, Yaadhav Raaj, Haroon Idrees, Donglai Xiang, Hanbyul Joo, Tomas Simon, Yaser Sheikh

  ICCV 2019

  arXiv OpenPose Training OpenPose
• 
  Towards Social Artificial Intelligence: Nonverbal Social Signal Prediction in A Triadic Interaction

  Hanbyul Joo, Tomas Simon, Mina Cikara, Yaser Sheikh

  CVPR 2019  Oral presentation — 288/5165 = 5.5%

  arXiv Code/Dataset Video
• 
  Monocular Total Capture: Posing Face, Body and Hands in the Wild

  Donglai Xiang, Hanbyul Joo, Yaser Sheikh

  CVPR 2019  Oral presentation — 288/5165 = 5.5%

  arXiv Dataset and Code Video
• 
  Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies

  Hanbyul Joo, Tomas Simon, and Yaser Sheikh

  CVPR 2018  Oral presentation — 70/3359 = 2.0%

  CVPR Best Student Paper Award

  PDF Supplementary Video Project Page
• 
  Structure from Recurrent Motion: From Rigidity to Recurrency

  Xiu Li, Hongdong Li, Hanbyul Joo, Yebin Liu, Yaser Sheikh

  CVPR 2018

  PDF
• 
  Hand Keypoint Detection in Single Images using Multiview Bootstrapping

  Tomas Simon, Hanbyul Joo, Iain Matthews, and Yaser Sheikh

  CVPR 2017

  arXiv Project Page Code Dataset
• 
  Panoptic Studio: A Massively Multiview System for Social Interaction Capture

  Hanbyul Joo, Tomas Simon, Xulong Li, Hao Liu, Lei Tan, Lin Gui, Sean Banerjee, Timothy Godisart, Bart Nabbe, Iain Matthews, Takeo Kanade, Shohei Nobuhara, and Yaser Sheikh

  TPAMI 2017 (Extended version of ICCV 2015)

  Published Version arXiv Dataset
• 
  Panoptic Studio: A Massively Multiview System for Social Motion Capture

  Hanbyul Joo, Hao Liu, Lei Tan, Lin Gui, Bart Nabbe, Iain Matthews, Takeo Kanade, Shohei Nobuhara, and Yaser Sheikh

  ICCV 2015  Oral presentation — 56/1698 = 3.3%

  Paper (PDF) Supplementary Project Page

  Press:
• 
  MAP Visibility Estimation for Large-Scale Dynamic 3D Reconstruction

  Hanbyul Joo, Hyun Soo Park, and Yaser Sheikh

  CVPR 2014  Oral presentation — 104/1807 = 5.8%

  Paper (PDF) Project Page Dataset

  Press:
• 
  Graph-based Shape Matching for Deformable Objects

  Hanbyul Joo, Yekeun Jeong, Olivier Duchenne, and In So Kweon

  ICIP 2011

  Paper (PDF)
• 
  Graph-Based Robust Shape Matching for Robotic Application

  Hanbyul Joo, Yekeun Jeong, Olivier Duchenne, Seong-Young Ko, and In So Kweon

  ICRA 2009

  Paper (PDF)

DIY A Multiview Camera System: Panoptic Studio Teardown

Hanbyul Joo, Tomas Simon, Hyun Soo Park, Shohei Nobuhara, and Yaser Sheikh

In Conjunction with CVPR 2017

Tutorial Page Video

WIRED, 2018

[Video + Article]

Reuters, 2017

[Video]

The Verge, 2016

[Video, Article]

Reuters, 2015

[Video + Article]

Daily Planet, Discovery, 2015

[Video]

Docuprime, EBS (Korean TV), 2017

[Video]

• Arirang TV, The Evolution of AI and AI: Reading Humans, 2025
• IEEE Spectrum, Robots Learn to Speak Body Language, 2017
• ZDNet, CMU researchers create a huge dome that can read body language, 2017
• TechCrunch, CMU researchers create a huge dome that can read body language, 2017
• BBC News, The dome which could help machines understand behaviour, 2017
• CMU News, Scientists put human interaction under the microscope, 2017
• SPIEGEL ONLINE (German), The panoptic Studio: Computer decipher the secrets of body language, 2015
• Co.DESIGN, Inside A Robot Eyeball, Science Will Decode Our Body Language, 2015
• Voice of America, New Studio Yields Most Detailed Motion Capture in 3D, 2015
• CNet, Tomorrow Daily: New video capture tech, 2014
• NBCNews, Camera-Studded Dome Tracks Your Every Move With Precision, 2014
• IEEE Spectrum, Camera-Filled Dome Recreates Full 3-D Motion Scenes, 2014
• Discovery News, Amazing 3-D Flicks from Dome of 500 Cameras?, 2014
• Gizmodo, A Dome Packed With 480 Cameras Captures Detailed 3D Images In Motion, 2014
• The Verge, Scientists build a real Panopticon that captures your every move in 3D, 2014
• ScienceDaily, Hundreds of videos used to reconstruct 3-D motion without markers, 2014
• Phys.org, Researchers combine hundreds of videos to reconstruct 3D motion without markers, 2014
• Engadget, Watch a dome full of cameras capture 3D motion in extreme detail, 2014
• PetaPixel, Researchers Use a 480-Camera Dome to More Accurately Capture 3D Motion, 2014
• Gizmag, Camera-studded dome used to reconstruct 3D motion, 2014
• The Register, Boffins fill a dome with 480 cameras for 3D motion capture, 2014
• The Engineer, 3D motion captured without markers, 2014
• Popular Photography, Carnegie Mellon Packs 480 Cameras In A Dome To Perfectly Track 3D Motion, 2014
• CMU News, Carnegie Mellon Combines Hundreds of Videos To Reconstruct 3D Motion Without Use of Markers, 2014