Now showing 1 - 10 of 25
  • Placeholder Image
    Publication
    Efficient Motion Deblurring with Feature Transformation and Spatial Attention
    (01-09-2019)
    Purohit, Kuldeep
    ;
    Convolutional Neural Networks (CNN) have recently advanced the state-of-the-art in generalized motion deblurring. Literature suggests that restoration of high-resolution blurred images requires a design with a large receptive field, which existing networks achieve by increasing the number of generic convolution layers, kernel-size, or the scales at which the image is processed. However, increasing the network capacity in this form comes with the burden of increased model size and lower speed. To resolve this, we propose a novel architecture composed of dynamic convolutional modules, namely feature transformation (FT) and spatial attention (SA). An FT module addresses the camera shifts responsible for the global blur in the input image, while a SA module addresses spatially varying blur due to dynamic objects and depth changes. Qualitative and quantitative comparisons on deblurring benchmarks demonstrate that our network outperforms prior art across factors of accuracy, compactness, and speed, enabling real-time deblurring.
  • Placeholder Image
    Publication
    Mixed-dense connection networks for image and video super-resolution
    (20-07-2020)
    Purohit, Kuldeep
    ;
    Mandal, Srimanta
    ;
    Efficiency of gradient propagation in intermediate layers of convolutional neural networks is of key importance for super-resolution task. To this end, we propose a deep architecture for single image super-resolution (SISR), which is built using efficient convolutional units we refer to as mixed-dense connection blocks (MDCB). The design of MDCB combines the strengths of both residual and dense connection strategies, while overcoming their limitations. To enable super-resolution for multiple factors, we propose a scale-recurrent framework which reutilizes the filters learnt for lower scale factors recursively for higher factors. This leads to improved performance and promotes parametric efficiency for higher factors. We train two versions of our network to enhance complementary image qualities using different loss configurations. We further employ our network for video super-resolution task, where our network learns to aggregate information from multiple frames and maintain spatio-temporal consistency. The proposed networks lead to qualitative and quantitative improvements over state-of-the-art techniques on image and video super-resolution benchmarks.
  • Placeholder Image
    Publication
    Distillation-guided Image Inpainting
    (01-01-2021)
    Suin, Maitreya
    ;
    Purohit, Kuldeep
    ;
    Image inpainting methods have shown significant improvements by using deep neural networks recently. However, many of these techniques often create distorted structures or blurry inconsistent textures. The problem is rooted in the encoder layers' ineffectiveness in building a complete and faithful embedding of the missing regions from scratch. Existing solutions like course-to-fine, progressive refinement, structural guidance, etc. suffer from huge computational overheads owing to multiple generator networks, limited ability of handcrafted features, and sub-optimal utilization of the information present in the ground truth. We propose a distillation-based approach for inpainting, where we provide direct feature level supervision while training. We deploy cross and self-distillation techniques and design a dedicated completion-block in encoder to produce more accurate encoding of the holes. Next, we demonstrate how an inpainting network's attention module can improve by leveraging a distillation-based attention transfer technique and further enhance coherence by using a pixel-adaptive global-local feature fusion. We conduct extensive evaluations on multiple datasets to validate our method. Along with achieving significant improvements over previous SOTA methods, the proposed approach's effectiveness is also demonstrated through its ability to improve existing inpainting works.
  • Placeholder Image
    Publication
    AIM 2020 Challenge on Efficient Super-Resolution: Methods and Results
    (01-01-2020)
    Zhang, Kai
    ;
    Danelljan, Martin
    ;
    Li, Yawei
    ;
    Timofte, Radu
    ;
    Liu, Jie
    ;
    Tang, Jie
    ;
    Wu, Gangshan
    ;
    Zhu, Yu
    ;
    He, Xiangyu
    ;
    Xu, Wenjie
    ;
    Li, Chenghua
    ;
    Leng, Cong
    ;
    Cheng, Jian
    ;
    Wu, Guangyang
    ;
    Wang, Wenyi
    ;
    Liu, Xiaohong
    ;
    Zhao, Hengyuan
    ;
    Kong, Xiangtao
    ;
    He, Jingwen
    ;
    Qiao, Yu
    ;
    Dong, Chao
    ;
    Luo, Xiaotong
    ;
    Chen, Liang
    ;
    Zhang, Jiangtao
    ;
    Suin, Maitreya
    ;
    Purohit, Kuldeep
    ;
    ;
    Li, Xiaochuan
    ;
    Lang, Zhiqiang
    ;
    Nie, Jiangtao
    ;
    Wei, Wei
    ;
    Zhang, Lei
    ;
    Muqeet, Abdul
    ;
    Hwang, Jiwon
    ;
    Yang, Subin
    ;
    Kang, Jung Heum
    ;
    Bae, Sung Ho
    ;
    Kim, Yongwoo
    ;
    Qu, Yanyun
    ;
    Jeon, Geun Woo
    ;
    Choi, Jun Ho
    ;
    Kim, Jun Hyuk
    ;
    Lee, Jong Seok
    ;
    Marty, Steven
    ;
    Marty, Eric
    ;
    Xiong, Dongliang
    ;
    Chen, Siang
    ;
    Zha, Lin
    ;
    Jiang, Jiande
    ;
    Gao, Xinbo
    ;
    Lu, Wen
    ;
    Wang, Haicheng
    ;
    Bhaskara, Vineeth
    ;
    Levinshtein, Alex
    ;
    Tsogkas, Stavros
    ;
    Jepson, Allan
    ;
    Kong, Xiangzhen
    ;
    Zhao, Tongtong
    ;
    Zhao, Shanshan
    ;
    Hrishikesh, P. S.
    ;
    Puthussery, Densen
    ;
    Jiji, C. V.
    ;
    Nan, Nan
    ;
    Liu, Shuai
    ;
    Cai, Jie
    ;
    Meng, Zibo
    ;
    Ding, Jiaming
    ;
    Ho, Chiu Man
    ;
    Wang, Xuehui
    ;
    Yan, Qiong
    ;
    Zhao, Yuzhi
    ;
    Chen, Long
    ;
    Sun, Long
    ;
    Wang, Wenhao
    ;
    Liu, Zhenbing
    ;
    Lan, Rushi
    ;
    Umer, Rao Muhammad
    ;
    Micheloni, Christian
    This paper reviews the AIM 2020 challenge on efficient single image super-resolution with focus on the proposed solutions and results. The challenge task was to super-resolve an input image with a magnification factor × 4 based on a set of prior examples of low and corresponding high resolution images. The goal is to devise a network that reduces one or several aspects such as runtime, parameter count, FLOPs, activations, and memory consumption while at least maintaining PSNR of MSRResNet. The track had 150 registered participants, and 25 teams submitted the final results. They gauge the state-of-the-art in efficient single image super-resolution.
  • Placeholder Image
    Publication
    Multilevel weighted enhancement for underwater image dehazing
    (01-06-2019)
    Purohit, Kuldeep
    ;
    Mandal, Srimanta
    ;
    Attenuation and scattering of light are responsible for haziness in images of underwater scenes. To reduce this effect, we propose an approach for single-image dehazing by multilevel weighted enhancement of the image. The underlying principle is that enhancement at different levels of detail can undo the degradation caused by underwater haze. The depth information is captured implicitly while going through different levels of details due to the depth-variant nature of haze. Hence, we judiciously assign weights to different levels of image details and reveal that their linear combination along with the coarsest information can successfully restore the image. Results demonstrate the efficacy of our approach as compared to state-of-the-art underwater dehazing methods.
  • Placeholder Image
    Publication
    AIM 2019 challenge on bokeh effect synthesis: Methods and results
    (01-10-2019)
    Ignatov, Andrey
    ;
    Patel, Jagruti
    ;
    Timofte, Radu
    ;
    Zheng, Bolun
    ;
    Ye, Xin
    ;
    Huang, Li
    ;
    Tian, Xiang
    ;
    Dutta, Saikat
    ;
    Purohit, Kuldeep
    ;
    Kandula, Praveen
    ;
    Suin, Maitreya
    ;
    ;
    Xiong, Zhiwei
    ;
    Huang, Jie
    ;
    Dong, Guanting
    ;
    Yao, Mingde
    ;
    Liu, Dong
    ;
    Yang, Wenjin
    ;
    Hong, Ming
    ;
    Lin, Wenying
    ;
    Qu, Yanyun
    ;
    Choi, Jae Seok
    ;
    Park, Woonsung
    ;
    Kim, Munchurl
    ;
    Liu, Rui
    ;
    Mao, Xiangyu
    ;
    Yang, Chengxi
    ;
    Yan, Qiong
    ;
    Sun, Wenxiu
    ;
    Fang, Junkai
    ;
    Shang, Meimei
    ;
    Gao, Fei
    ;
    Ghosh, Sujoy
    ;
    Sharma, Prasen Kumar
    ;
    Sur, Arijit
    This paper reviews the first AIM challenge on bokeh effect synthesis with the focus on proposed solutions and results. The participating teams were solving a real-world image-to-image mapping problem, where the goal was to map standard narrow-aperture photos to the same photos captured with a shallow depth-of-field by the Canon 70D DSLR camera. In this task, the participants had to restore bokeh effect based on only one single frame without any additional data from other cameras or sensors. The target metric used in this challenge combined fidelity scores (PSNR and SSIM) with solutions' perceptual results measured in a user study. The proposed solutions significantly improved baseline results, defining the state-of-the-art for practical bokeh effect simulation.
  • Placeholder Image
    Publication
    Region-Adaptive dense network for efficient motion deblurring
    (01-01-2020)
    Purohit, Kuldeep
    ;
    In this paper, we address the problem of dynamic scene deblurring in the presence of motion blur. Restoration of images affected by severe blur necessitates a network design with a large receptive field, which existing networks attempt to achieve through simple increment in the number of generic convolution layers, kernel-size, or the scales at which the image is processed. However, these techniques ignore the nonuniform nature of blur, and they come at the expense of an increase in model size and inference time. We present a new architecture composed of region adaptive dense deformable modules that implicitly discover the spatially varying shifts responsible for non-uniform blur in the input image and learn to modulate the filters. This capability is complemented by a self-attentive module which captures non-local spatial relationships among the intermediate features and enhances the spatially varying processing capability. We incorporate these modules into a densely connected encoder-decoder design which utilizes pre-trained Densenet filters to further improve the performance. Our network facilitates interpretable modeling of the spatially-varying deblurring process while dispensing with multi-scale processing and large filters entirely. Extensive comparisons with prior art on benchmark dynamic scene deblurring datasets clearly demonstrate the superiority of the proposed networks via significant improvements in accuracy and speed, enabling almost real-time deblurring.
  • Placeholder Image
    Publication
    NTIRE 2019 challenge on video deblurring: Methods and results
    (01-06-2019)
    Nah, Seungjun
    ;
    Timofte, Radu
    ;
    Baik, Sungyong
    ;
    Hong, Seokil
    ;
    Moon, Gyeongsik
    ;
    Son, Sanghyun
    ;
    Lee, Kyoung Mu
    ;
    Wang, Xintao
    ;
    Chan, Kelvin C.K.
    ;
    Yu, Ke
    ;
    Dong, Chao
    ;
    Loy, Chen Change
    ;
    Fan, Yuchen
    ;
    Yu, Jiahui
    ;
    Liu, DIng
    ;
    Huang, Thomas S.
    ;
    Sim, Hyeonjun
    ;
    Kim, Munchurl
    ;
    Park, Dongwon
    ;
    Kim, Jisoo
    ;
    Chun, Se Young
    ;
    Haris, Muhammad
    ;
    Shakhnarovich, Greg
    ;
    Ukita, Norimichi
    ;
    Zamir, Syed Waqas
    ;
    Arora, Aditya
    ;
    Khan, Salman
    ;
    Khan, Fahad Shahbaz
    ;
    Shao, Ling
    ;
    Gupta, Rahul Kumar
    ;
    Chudasama, Vishal
    ;
    Patel, Heena
    ;
    Upla, Kishor
    ;
    Fan, Hongfei
    ;
    Li, Guo
    ;
    Zhang, Yumei
    ;
    Li, Xiang
    ;
    Zhang, Wenjie
    ;
    He, Qingwen
    ;
    Purohit, Kuldeep
    ;
    ;
    Kim, Jeonghun
    ;
    Tofighi, Mohammad
    ;
    Guo, Tiantong
    ;
    Monga, Vishal
    This paper reviews the first NTIRE challenge on video deblurring (restoration of rich details and high frequency components from blurred video frames) with focus on the proposed solutions and results. A new REalistic and Diverse Scenes dataset (REDS) was employed. The challenge was divided into 2 tracks. Track 1 employed dynamic motion blurs while Track 2 had additional MPEG video compression artifacts. Each competition had 109 and 93 registered participants. Total 13 teams competed in the final testing phase. They gauge the state-of-the-art in video deblurring problem.
  • Placeholder Image
    Publication
    Spatially-attentive patch-hierarchical network for adaptive motion deblurring
    (01-01-2020)
    Suin, Maitreya
    ;
    Purohit, Kuldeep
    ;
    This paper tackles the problem of motion deblurring of dynamic scenes. Although end-to-end fully convolutional designs have recently advanced the state-of-the-art in non-uniform motion deblurring, their performance-complexity trade-off is still sub-optimal. Existing approaches achieve a large receptive field by increasing the number of generic convolution layers and kernel-size, but this comesat the expense of of the increase in model size and inference speed. In this work, we propose an efficient pixel adaptive and feature attentive design for handling large blur variations across different spatial locations and process each test image adaptively. We also propose an effective content-aware global-local filtering module that significantly improves performance by considering not only global dependencies but also by dynamically exploiting neighboring pixel information. We use a patch-hierarchical attentive architecture composed of the above module that implicitly discovers the spatial variations in the blur present in the input image and in turn, performs local and global modulation of intermediate features. Extensive qualitative and quantitative comparisons with prior art on deblurring benchmarks demonstrate that our design offers significant improvements over the state-of-the-art in accuracy as well as speed.
  • Placeholder Image
    Publication
    AIM 2019 challenge on real-world image super-resolution: Methods and results
    (01-10-2019)
    Lugmayr, Andreas
    ;
    Danelljan, Martin
    ;
    Timofte, Radu
    ;
    Fritsche, Manuel
    ;
    Gu, Shuhang
    ;
    Purohit, Kuldeep
    ;
    Kandula, Praveen
    ;
    Suin, Maitreya
    ;
    ;
    Joon, Nam Hyung
    ;
    Won, Yu Seung
    ;
    Kim, Guisik
    ;
    Kwon, Dokyeong
    ;
    Hsu, Chih Chung
    ;
    Lin, Chia Hsiang
    ;
    Huang, Yuanfei
    ;
    Sun, Xiaopeng
    ;
    Lu, Wen
    ;
    Li, Jie
    ;
    Gao, Xinbo
    ;
    Bell-Kligler, Sefi
    This paper reviews the AIM 2019 challenge on real world super-resolution. It focuses on the participating methods and final results. The challenge addresses the real world setting, where paired true high and low-resolution images are unavailable. For training, only one set of source input images is therefore provided in the challenge. In Track 1: Source Domain the aim is to super-resolve such images while preserving the low level image characteristics of the source input domain. In Track 2: Target Domain a set of high-quality images is also provided for training, that defines the output domain and desired quality of the super-resolved images. To allow for quantitative evaluation, the source input images in both tracks are constructed using artificial, but realistic, image degradations. The challenge is the first of its kind, aiming to advance the state-of-the-art and provide a standard benchmark for this newly emerging task. In total 7 teams competed in the final testing phase, demonstrating new and innovative solutions to the problem.