Image inpainting using a weighted self supervised learning architecture. Work published at CGVCVIP 2022.
Image inpainting is the process of regenerating lost parts of the image. Supervised algorithm-based methods have shown excellent results but have two significant drawbacks. They do not perform well when tested with unseen data. They fail to capture the global context of the image, resulting in a visually unappealing result. We propose a novel self-supervised learning framework for image-inpainting: Weighted Self-Supervised Learning (WSSL) to tackle these problems. We designed WSSL to learn features from multiple weighted pretext tasks. These features are then utilized for the downstream task, image-inpainting. To improve the performance of our framework and produce more visually appealing images, we also present a novel loss function for image inpainting. The loss function takes advantage of both reconstruction loss and perceptual loss functions to regenerate the image Our experimentation shows WSSL outperforms previous methods, and our loss function helps produce better results.
The weight names follow the following naming convention
<task-1>-<task-2>-<task-1-weight>-<task-2-weight>. They are the final weights to be used by the model for inference.
| Downstream task weight combination | Download link |
|---|---|
| Sharpness-Saturation-70-30 | here |
| Sharpness-Saturation-90-10 | here |
| Rotation-Saturation-30-70 | here |
NOTE: There may be minor variations in the PSNR and SSIM observed depending on the input image and mask combination used during inference.
If you find our work useful, please cite using
Gupta, S., Ravishankar, R. K., Gangaraju, M., Dwarkanath, P., & Subramanyam, N. (2022). WSSL: Weighted Self-Supervised Learning Framework for Image-Inpainting. In Y. Xiao, A. Abraham, G. Chao Peng & J. Roth (Eds.), Proceedings of the International Conferences Computer Graphics, Visualization, Computer Vision and Image Processing, Connected Smart Cities, Big Data Analytics, Data Mining and Computational Intelligence and Theory and Practice in Modern Computing 2022 (pp.111-119). Lisbon, Portugal: IADIS Press. ISBN: 978-989-8704-42-9
@incollection{GuptaRavishankarGangarajuDwarkanathSubramanyam2022,
author = {Gupta, S. and Ravishankar, R. K. and Gangaraju, M. and Dwarkanath, P. and Subramanyam, N.},
booktitle = {Proceedings of the International Conferences Computer Graphics, Visualization, Computer Vision and Image Processing, Connected Smart Cities, Big Data Analytics, Data Mining and Computational Intelligence and Theory and Practice in Modern Computing 2022 .},
editor = {Xiao, A. Abraham and G. Chao Peng and J. Roth},
pages = {111-119},
publisher = {IADIS Press.},
title = {WSSL: Weighted Self-Supervised Learning Framework for Image-Inpainting},
year = {2022},
}This work is licensed under the MIT License