LiteAR: A Framework to Estimate Lighting for Mixed Reality Sessions for Enhanced Realism

We propose an end-to-end learning based method to estimate irradiance in real-time given a single input limited field of view image from a mobile phone camera. We further develop a technique inspired by physically based rendering to take advantage of spatially varying environment to illuminate virtual objects in augmented reality sessions to make them look more realistic. We integrate the Inertial Measurement Unit sensor to dynamically estimate illumination, making the mixed reality experience interactive. Our solution runs in real-time on mobile phones, with significantly lower computational requirements and enhanced realism in comparison to state-of-the-art methods.