Document Type : Reseach Article
Authors
Department of Computer, Shiraz Branch, Islamic Azad University, Shiraz, Iran
Abstract
Super resolution algorithms attempt to reconstruct high resolution images from low resolution images and it can be considered as a preprocessing step for object recognition and image classification. Various algorithms have been introduced for single-image super resolution, but these algorithms often face important challenges such as poorly matching the reconstructed image with the original image. This article introduces a preprocessing operation to improve the performance of the super resolution process. In the proposed method, the low-resolution images are enhanced before entering to the resolution change module. Calculating the brightness of the pixels in the image channels, creating the luminance map and removing atmospheric light, applying the transmittance map by using the luminance coefficients, and recovering the natural image in all three-color channels are the above preprocessing steps. The proposed method succeeded in increasing the PSNR parameter by 4.35%, 10.62%, and 8.31%, as well as 0.23%, 3.10%, and 7.91% of the SSIM parameter for Set5, Set14, and BSD100 benchmark datasets compared to its closest state-of-the-art methods.
Keywords
- Burger W, Burge MJ. “Digital image processing: an algorithmic introduction using Java”. Springer; 2016.
- Huang S, Sun J, Yang Y, Fang Y, Lin P, Que Y. "Robust single-image super-resolution based on adaptive edge-preserving smoothing regularization”. IEEE Transactions on Image Processing, Vol. 27(6), pp. 2650-2663, 2018. http://doi.org/ 10.1109/TIP.2018.2809472
- Yue L, Shen H, Li J, Yuan Q, Zhang H, Zhang L. "Image super-resolution: the techniques, applications, and future”. Signal Processing 2016, 128, pp. 389-408, 2016. https://doi.org/10.1016/j.sigpro.2016.05.002
- Haris M, Watanabe T, Fan L, Widyanto MR, Nobuhara H. “Super resolution for UAV images via adaptive multiple sparse representation and its application to 3-d reconstruction”. IEEE Transactions on Geoscience and Remote Sensing 2017, 55(7) , pp. 4047-4058, 2017. http://doi: 10.1109/TGRS.2017.2687419
- Ayas S, Ekinci M. "Learning based single image super resolution using discrete wavelet transform”. Ininternational conference on computer analysis of images and patterns, pp. 462-472, Springer, cham; 2017.
- Waleed Gondal M, Scholkopf B, Hirsch M. "The unreasonable effectiveness of texture transfer for single image super-resolution”. InProceedings of the European Conference on Computer Vision (ECCV) 2018, pp. 80-97, Springer, Cham.
- Lei J, Zhang S, Luo L, Xiao J, Wang H. "Super-resolution enhancement of UAV images based on fractional calculus and POCS”. Geo-spatial information science 2018, 21(1), pp. 56-66, 2018. https://doi.org/10.1080/10095020.2018.1424409
- Liu H, Fu Z, Han J, Shao L, Liu H. "Single satellite imagery simultaneous super-resolution and colorization using multi-task deep neural networks”. Journal of visual communication and image representation 2018, 53, pp. 20-30, 2018. https://doi.org/10.1016/j.jvcir.2018.02.016
- Han W, Chu J, Wang L, Pan C. "Edge-directed single image super-resolution via cross-resolution sharpening function learning”. Multimedia tools and applications 2017, 76(8), pp. 11143-11155, 2017. https://doi.org/10.1007/s11042-016-3656-z
- Bei Y, Damian A, Hu S, Menon S, Ravi N, Rudin C. “New techniques for preserving global structure and denoising with low information loss in single-image super-resolution”. In the IEEE conference on computer vision and pattern recognition (CVPR) workshops 2018, 874-881, 2018.
- Brifman A, Romano Y, Elad M. "Unified Single-Image and Video Super-Resolution via Denoising Algorithms”. IEEE Transactions on Image Processing 2019, 28(12), 6063-6076, 2019. http://doi.org/10.1109/TIP.2019.2924173.
- Ding N, Liu YP, Fan LW, Zhang CM. "Single Image Super-Resolution via Dynamic Lightweight Database with Local-Feature Based Interpolation”. Journal of Computer Science and Technology 2019, 34(3), 537-549, 2019. https://doi.org/10.1007/s11390-019-1925-9
- Yang W, Zhang X, Tian Y, Wang W, Xue JH, Liao Q. “Deep learning for single image super-resolution: A brief review”. IEEE Transactions on Multimedia 2019, 21(12), 3106-3121. http://doi.org/10.1109/TMM.2019.2919431
- Lu H, Li Y, Nakashima S, Serikawa S. "Single image dehazing through improved atmospheric light estimation”. Multimedia Tools and Applications 2016, 75(24), 17081-17096. https://doi.org/10.1007/s11042-015-2977-7
- Song J, Zhang L, Shen P, Peng X, Zhu G. "Single low-light image enhancement using luminance map”. In Chinese Conference on Pattern Recognition 2016, pp. 101-110. Springer, https://doi.org/10.1007/978-981-10-3005-5_9
- Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Bengio Y. “Generative adversarial nets”. In Advances in neural information processing systems 2014, pp. 2672-2680, 2014.
- Chang HW, Zhang Q W, Wu QG, Gan Y. "Perceptual image quality assessment by independent feature detector”. Neurocomputing 2015, 151, pp. 1142-1152. https://doi.org/10.1016/j.neucom.2014.04.081
- Tuna C, Unal G, Sertel E. "Single-frame super resolution of remote-sensing images by convolutional neural networks”. International Journal of Remote Sensing 2018, 39(8), 2463-2479.https://doi.org/10.1080/01431161.2018.1425561
- Li F, Xin L, Guo Y, Gao J, Jia X. "A framework of mixed sparse representations for remote sensing images”. IEEE Transactions on Geoscience and Remote Sensing 2017, 55(2), 1210-1221. http://doi.org/10.1109/TGRS.2016.2621123
- Fan C, Wu C, Li G, Ma J. "Projections onto convex sets super-resolution reconstruction based on point spread function estimation of low-resolution remote sensing images”. Sensors 2017, 17(2), 362. https://doi.org/10.3390/s17020362
- Lv Z, Jia Y, Zhang Q. "Joint image registration and point spread function estimation for the super-resolution of satellite images”. Signal processing: image communication 2017, 58, 199-211. https://doi.org /10.1016/j.image. 2017. 08.006
- Cruz C, Mehta R, Katkovnik V, Egiazarian KO. "Single image super-resolution based on wiener filter in similarity domain”. IEEE Transactions on Image Processing 2018, 27(3), 1376-1389. http://doi.org/10.1109/TIP.2017.2779265
- Lin G, Wu Q, Chen L, Qiu L, Wang X, Liu T, Chen X. “Deep unsupervised learning for image super-resolution with generative adversarial network”. Signal Processing: Image Communication 2018, 68, 88-100. https://doi.org/10.1016/j.image.2018.07.003
- Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Shi W. “Photo-realistic single image super-resolution using a generative adversarial network”. In Proceedings of the IEEE conference on computer vision and pattern recognition 2017, 4681-4690.
- Zhang X, Song H, Zhang K, Qiao J, Liu Q. "Single image super-resolution with enhanced Laplacian pyramid network via conditional generative adversarial learning”. Neurocomputing 2020, 398, 531-538. https://doi.org/10.1016/ j.neucom .2019.04.097
- Qiao J, Song H, Zhang K, Zhang X. "Conditional generative adversarial network with densely-connected residual learning for single image super-resolution”. Multimedia Tools and Applications 2021, 80(3), 4383-4397. https://doi.org/10.1007/s11042-020-09817-2
- Chen W, Liu C, Yan Y, Jin L, Sun X, Peng X. "Guided Dual Networks for Single Image Super-Resolution”. IEEE Access 2020, 8 :93608-93620. doi: 10.1109/ACCESS.2020.2995175.
- Haris M, Shakhnarovich G, Ukita N. "Deep Back-Projection Networks for Single Image Super-resolution”. IEEE Transactions on Pattern Analysis and Machine Intelligence 2019, 43(12), 4323-4337. http://doi.org/10.1109/ TPAMI.2020.3002836
- Hu Y, Li J, Huang Y, Gao X. "Channel-wise and spatial feature modulation network for single image super-resolution”. IEEE Transactions on Circuits and Systems for Video Technology 2019, 30(11), 3911-3927. http://doi.org/10.1109/ TCSVT.2019.2915238
- Xu X, Li X. SCAN: "Spatial Color Attention Networks for Real Single Image Super-Resolution”. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops 2019.
- Gao S, Zhuang X. "Multi-scale deep neural networks for real image super-resolution”. In proceedings of the IEEE conference on computer vision and pattern recognition workshops 2019.
- Xie C, Liu Y, Zeng W, Lu X. "An improved method for single image super-resolution based on deep learning”. Signal, image and video processing 2019, 13(3) :557-565. https://doi.org/10.1007/s11760-018-1382-x
- Yang W, Wang W, Zhang X, Sun S, Liao Q. “Lightweight feature fusion network for single image super-resolution”. IEEE Signal Processing Letters 2019, 26(4), 538-542. http://doi.org/10.1109/LSP.2018.2890770
- Dai T, Cai J, Zhang Y, Xia ST, Zhang L. "Second-order attention network for single image super-resolution”. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2019, pp. 11065-11074.
- Wang Y, Wang L, Wang H, Li P. "End-to-end image super-resolution via deep and shallow convolutional networks”. IEEE Access 2019, 7, 31959-31970. http://doi.org/10.1109 /ACCESS.2019.2903582
- Xu W, Chen R, Huang B, Zhang X, Liu C. "Single image super-resolution based on global dense feature fusion convolutional network”. Sensors 2019, 19(2), 316. https://doi.org/10.3390 /s19020316
- Fang F, Li J, Zeng T. "Soft-Edge Assisted Network for Single Image Super-Resolution”. IEEE Transactions on Image Processing 2020, 29, 4656-4668. http://doi.org/10.1109/TIP.2020.2973769
- https://deepai.org/dataset/set5-super-resolution (Accessed 20 May 2022)
- https://deepai.org/dataset/set14-super-resolution (Accessed 20 May 2022)
- https://deepai.org/dataset/bsd100-4x-upscaling (Accessed 20 May 2022)
- Guo X, Li Y, Ling H. "LIME: Low-light image enhancement via illumination map estimation”. IEEE Transactions on Image Processing 2017, 26(2), 982-993. http://doi.org/10.1109/TIP.2016.2639450
)