Chen-Hsuan Lin - Learning 3D Registration and Reconstruction from the Visual World

Sep 21st 2021 at MIT CSAIL Abstract: Humans learn to develop strong senses for 3D geometry by looking around in the visual world. Through pure visual perception, not only can we recover a mental 3D representation of what we are looking at, but meanwhile we can also recognize the location we are looking at the scene from. In this talk, I will discuss the problems of learning geometric alignment and dense 3D reconstruction, and the general importance of factorizing geometric information from visual data. I will discuss learning 3D shape priors from static RGB images from single-view supervision, as well as the problem of joint 3D registration and reconstruction: given a video sequence, how one can exploit pretrained 3D shape priors to register and refine 3D shape reconstruction, as well as a generic rendering prior from Neural Radiance Fields (NeRF) for learning neural 3D scene representations from noisy/unknown camera poses. Baking in suitable geometric priors allows learning models to effectively recov