Robust photometric alignment for asymmetric camera system

Multiple image sensors have now become the defacto offering from all major mobile phone manufacturers. Their applications vary from improving the image quality to providing new features to the end-users. One of these multi-sensor system is a tele-wide asymmetric camera configuration which is aimed at providing better zoom experience with more crisper images. The system is basically a combination of tele-photo lens which captures images at a higher optical zoom with narrow field of view and a wide-view lens which captures a wider field of view. Most of the leading OEMs provide only a smooth transition between the two lens systems while zooming into a scene since fusing the information from the two lens at all zoom ratios has some significant challenges. One such major challenge is accurate photometric alignment which is required to compensate the difference in auto exposure(AE), auto white balance(AWB), and the inherent differences in terms of both intrinsic and extrinsic parameters of the two camera sub-systems. Conventional methods in literature for photometric alignment make several assumptions and are also not very adaptable to different lighting conditions. To the best of our knowledge, accurate photometric alignment intended towards fusion of images from asymmetric tele-wide cameras that works in unconstrained scenarios and at all zoom ratios, is not very well studied. In this paper, we propose a computationally efficient algorithm for brightness and color correction that is suitable for fusion of images from multiple asymmetric cameras. The proposed method is evaluated quantitatively using the RMSE metric and qualitatively as well. Further the evaluation is carried out for several scenes captured under different lighting conditions, undergoing camera and object motion and with different focus and exposure values without calibration of inputs. Experimental results show that the proposed method is superior to the existing methods for photometric alignment. It is well suited as a robust image pre-conditioning step towards image-fusion and superresolution applications.