Total Relighting: Learning to Relight Portraits for Background Replacement

SIGGRAPH 2021 Technical Paper: Total Relighting: Learning to Relight Portraits for Background Replacement - Rohit Pandey*, Sergio Orts-Escolano*, Chloe LeGendre*, Christian Haene, Sofien Bouaziz, Christoph Rhemann, Paul Debevec, and Sean Fanello Project Page: We propose a novel system for portrait relighting and background replacement, which maintains high-frequency boundary details and accurately synthesizes the subject’s appearance as lit by novel illumination, thereby producing realistic composite images for any desired scene. Our technique includes foreground estimation via alpha matting, relighting, and compositing. We demonstrate that each of these stages can be tackled in a sequential pipeline without the use of priors (e.g. known background or known illumination) and with no specialized acquisition techniques, using only a single RGB portrait image and a novel, target HDR lighting environment as inputs. We train our model using relit portraits