Deep Learning for Image Super-Resolution

Authors

  • Gowrinath Jarugula

Keywords:

Image super-resolution, deep learning, convolutional neural networks, generative adversarial networks, transformer models

Abstract

Image super-resolution (SR) is an important problem in computer vision that aims to increase the spatial resolution of an image and to recover fine details from an input low-resolution observation. Conventional interpolation and reconstruction-based approaches can hardly obtain high-quality reconstructions because of their lack of flexibility in modeling intricate image priors. The development of deep learning has transformed the SR into data-driven domain learning hierarchical representations and complex transformation between low-level features and high-level representations from large-scale databases. This paper comprises a review of deep learning-based SR methods, classifying them into different categories according to the network architectures (e.g., convolutional neural networks, generative adversarial networks and transformer models), training strategies, and loss functions. In this paper consider trade-offs for reconstruction accuracy and perceptual quality, we describe evaluation measures, we indicate challenges namely computational cost and needing for real-world degradation modelling and generalization over a variety of scenarios. Finally, And suggest some appealing future research directions towards more efficient, robust and practical model for practical tasks.

Downloads

Download data is not yet available.

References

Dong, C., Loy, C. C., He, K., & Tang, X. (2014). Learning a deep convolutional network for image super-resolution. European Conference on Computer Vision (ECCV), 184–199. https://doi.org/10.1007/978-3-319-10593-2_13

Kim, J., Kwon Lee, J., & Mu Lee, K. (2016). Accurate image super-resolution using very deep convolutional networks. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1646–1654. https://doi.org/10.1109/CVPR.2016.182

Lim, B., Son, S., Kim, H., Nah, S., & Mu Lee, K. (2017). Enhanced deep residual networks for single image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 136–144. https://arxiv.org/abs/1707.02921

Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., … Shi, W. (2017). Photo-realistic single image super-resolution using a generative adversarial network. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4681–4690. https://doi.org/10.1109/CVPR.2017.19

Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., … Change Loy, C. (2018). ESRGAN: Enhanced super-resolution generative adversarial networks. European Conference on Computer Vision (ECCV) Workshops. https://arxiv.org/abs/1809.00219

Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. (2018). Residual dense network for image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2472–2481. https://doi.org/10.1109/CVPR.2018.00260

Hui, Z., Gao, X., Yang, Y., & Wang, X. (2018). Fast and accurate single image super-resolution via information distillation network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 723–731. https://doi.org/10.1109/CVPR.2018.00082

Haris, M., Shakhnarovich, G., & Ukita, N. (2018). Deep back-projection networks for super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1664–1673. https://doi.org/10.1109/CVPR.2018.00179

Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017). Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 26(7), 3142–3155. https://doi.org/10.1109/TIP.2017.2695227

Dai, T., Cai, J., Zhang, Y., Xia, S. T., & Zhang, L. (2019). Second-order attention network for single image super-resolution. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 11065–11074. https://doi.org/10.1109/CVPR.2019.01128

Li, J., Liang, X., Wei, Y., Xu, T., & Feng, J. (2021). SwinIR: Image restoration using swin transformer. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 1833–1844. https://arxiv.org/abs/2108.10257

Ledig, C., Shi, W., & Theis, L. (2016). Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802. https://arxiv.org/abs/1609.04802

Mechrez, R., Talmi, I., & Zelnik-Manor, L. (2018). The contextual loss for image transformation with non-aligned data. European Conference on Computer Vision (ECCV), 768–783. https://doi.org/10.1007/978-3-030-01231-1_46

Johnson, J., Alahi, A., & Fei-Fei, L. (2016). Perceptual losses for real-time style transfer and super-resolution. European Conference on Computer Vision (ECCV), 694–711. https://doi.org/10.1007/978-3-319-46475-6_43

Blau, Y., & Michaeli, T. (2018). The perception-distortion tradeoff. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 6228–6237. https://doi.org/10.1109/CVPR.2018.00654

Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018). The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 586–595. https://doi.org/10.1109/CVPR.2018.00068

Timofte, R., Agustsson, E., Van Gool, L., Yang, M., & Zhang, L. (2017). NTIRE 2017 challenge on single image super-resolution: Methods and results. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 114–125. https://doi.org/10.1109/CVPRW.2017.22

Huang, Y., Zhang, W., Li, J., & Wang, H. (2020). Unsupervised real-world super-resolution via domain-distance aware training. Advances in Neural Information Processing Systems, 33, 1293–1304. https://arxiv.org/abs/2007.15066

Haris, M., Shakhnarovich, G., & Ukita, N. (2020). Deep back-projection networks for super-resolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(8), 2841–2857. https://doi.org/10.1109/TPAMI.2019.2924482

Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. (2020). Residual non-local attention networks for image restoration. International Journal of Computer Vision, 128, 2239–2265. https://doi.org/10.1007/s11263-020-01368-8

Downloads

Published

31.07.2022

How to Cite

Gowrinath Jarugula. (2022). Deep Learning for Image Super-Resolution. International Journal of Intelligent Systems and Applications in Engineering, 10(2s), 353–365. Retrieved from https://www.ijisae.org/index.php/IJISAE/article/view/8063

Issue

Vol. 10 No. 2s (2022)

Section

Research Article

License

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.

IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.