Document Type : Review Article
Authors
1 Ferdowsi University of Mashhad
2 McMaster University
Abstract
With the increasing demand for image-based applications, the efficient and reliable evaluation of image quality has increased in importance. Measuring the image quality is of fundamental importance for numerous image processing applications, where the goal of image quality assessment (IQA) methods is to automatically evaluate the quality of images in agreement with human quality judgments. Numerous IQA methods have been proposed over the past years to fulfill this goal. In this paper, a survey of the quality assessment methods for conventional image signals, as well as the newly emerged ones, which includes the high dynamic range (HDR) and 3-D images, is presented. A comprehensive explanation of the subjective and objective IQA and their classification is provided. Six widely used subjective quality datasets, and performance measures are reviewed. Emphasis is given to the full-reference image quality assessment (FR-IQA) methods, and 9 often-used quality measures (including mean squared error (MSE), structural similarity index (SSIM), multi-scale structural similarity index (MS-SSIM), visual information fidelity (VIF), most apparent distortion (MAD), feature similarity measure (FSIM), feature similarity measure for color images (FSIMC), dynamic range independent measure (DRIM), and tone-mapped images quality index (TMQI)) are carefully described, and their performance and computation time on four subjective quality datasets are evaluated. Furthermore, a brief introduction to 3-D IQA is provided and the issues related to this area of research are reviewed.
Keywords
[1] F. Xiao, J.E. Farrel, and B.A. Wandell, "Psychophysical thresholds and digital camera sensitivity: the thousand-photon limit," Proc. SPIE, Vol. 5678, pp. 75-84, Feb. 2005.
[2] J. Chen, T.N. Pappas, A. Mojsilovic, and B.E. Rogowitz, "Perceptually-tuned multiscale color-texture segmentation," in IEEE Int. Conf. on Image Processing, Oct. 2004.
[3] X.Zhang, D.A. Silverstein, J.E. Farrell, and B.A. Wandell, "Color image quality metric S-CIELAB and its application on halftone texture visibility," presented at IEEE Computer Conf., Feb. 1997.
[4] N., Damera-Venkata, T.D., Kite, W.S., Geisler, B. L., Evans, and A.C., Bovik, "Image quality assessment based on a degradation model," IEEE Trans. Image Processing, Vol. 9, pp. 636-650, April 2000.
[5] G. Piella, H. Heijmans, "A new quality metric for image fusion," presented at IEEE Int. Conf. on Image Processing, Sept. 2003.
[6] H.H. Barret, "Objective assessment of image quality: effects of quantum noise and object variability," J. Opt. Soc. Am. A, Vol. 7, pp. 1266-1278, 1990.
[7] H.H. Barret, J.L. Denny, R.F. Wanger, and K.J. Myers, "Objective assessment of image quality. II. Fisher information, Fourier crosstalk, and figures of merit for task performance," J. Opt. Soc. Am. A, Vol.12, pp. 834-852, 1995.
[8] Z. Wang, A.C. Bovik, and E.P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Trans. Image
Processing, Vol. 13, pp. 600 612, April 2004.
[9] Z. Wang, E.P. Simoncelli, and A.C. Bovik, "Multi-scale structural similarity for image quality assessment," presented at IEEE Asilomar Conf. on Signals, Systems, and Computers, Nov. 2003.
[10] H.R. Sheikh, A.C. Bovik, "Image information and visual quality," IEEE Trans. Image Processing, Vol. 15, pp. 430-444, Feb. 2006.
[11] E.C. Larson, D.M. Chandler, "Most apparent distortion: full-reference image quality assessment and the role of strategy," J. Electron. Imag, Vol. 19, pp. 011006:1 011006:21, Jan. 2010.
[12] L. Zhang, L. Zhang, X. Mou, and D. Zhang, "FSIM: A feature similarity index for image quality assessment," IEEE Trans. Image Processing, Vol. 20, pp. 2378-2386, Aug. 2011.
[13] T.O. Aydin, R. Mantiuk, K. Myszkowski, and H.P. Seidel, "Dynamic range independent image quality assessment," ACM Trans. Graph. , Vol.27, pp.1 -10, Aug. 2008.
[14] H. Yeganeh, Z. Wang, "Objective quality assessment of tone-mapped images," IEEE Trans. Image Processing, vol. 22, pp. 657-667, Feb. 2013.
[15] H.R. Sheikh, K. Seshadrinathan, A.K. Moorthy, Z. Wang, and A.C. Bovik, "LIVE image quality assessment database," [Online]. Available: http://live.ece.utexas.edu/research/quality/subjective.htm.
[16] E.C. Larson, D.M. Chandler, "Categorical image quality dataset," [Online]. Available: http://vision.okstate.edu/csiq.
[17] N. Ponomarenko, K. Egiazarian, "Tampere image database 2008 TID2008," [Online]. Available: http://www.ponomarenko.info/tid2008.htm.
[18] H. Yeganeh, Z. Wang, "TMQI: Tone-mapped image quality index," [Online]. Available: http://ece.uwaterloo.ca/~z70wang/research/tmqi/.
[19] ITU-R Recommendation BT.500-11 , "Methodology for the subjective assessment of the quality of television pictures," ITU, Geneva, Switzerland, 2002.
[20] ITU-R Recommendation BT.710-4, "Subjective assessment methods for image quality in high-definition television," ITU, Geneva, Switzerland, 1998.
[21] ITU-T Recommendation P.910, "Subjective video quality assessment methods for multimedia applications," ITU, Geneva, Switzerland, 2008.
[22] ITU-R Recommendation BT.814-1, "Specification and alignment procedures for setting of brightness and contrast of displays," ITU, 1994.
[23] ITU-R Recommendation BT.1129-2, "Subjective assessment of standard definition digital television (SDTV) systems," ITU, 1998.
[24] ITU-R Recommendation BT.1361, "Worldwide unified colorimetry and related characteristics of future television and imaging systems," ITU, 1998.
[25] ITU-R Recommendation BT.815-1, "Specification of a signal for measurement of the contrast ratio of displays," ITU, 1994.
[26] R. Mantiuk, A. Tomaszewska, and R. Mantiuk,
Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015
80
"Comparison of four subjective methods for image quality assessment," Computer Graphics Forum, Vol.31, pp. 2478-2491, 2012.
[27] H. Gulliksen, L.R. Tucker, "A general procedure for obtaining paired comparisons from multiple rank orders," Psychometrika, Vol. 26, pp. 1 73-183, June 1961.
[28] D.A. Silverstein, J.E. Farrell, "Efficient method for paired comparison," J. Electron. Imag., Vol. 10, pp. 394-398, April 2001.
[29] A.M. van Dijk, J.B. Martens, and A.B. Watson, "Quality assessment of coded images using numerical category scaling," Proc. SPIE, Vol. 2451, pp. 90-101, Feb. 1995.
[30] Z. Wang, A.C. Bovik, “Modern image quality assessment. Synthesis Lectures on Image”, Video & Multimedia Processing, Morgan & Claypool Publishers, 2006.
[31] H. R. Sheikh, A.C. Bovik, and L. Cormack, "No-reference quality assessment using nature scene statistics: JPEG2000," IEEE Trans. Image Processing, Vol.14, pp.1918-1927 , Nov. 2005.
[32] L. Liang, S. Wang, J. Chen, S. Ma, D. Zhao, and W. Gao, "No-reference perceptual image quality metric using gradient profiles for JPEG2000," Signal Processing: Image Communication , Vol. 25, pp. 502-516, Aug. 2010.
[33] T. Brando, M.P. Queluz, "No-reference image quality assessment based on DCT domain statistics," Signal Processing, Vol. 88, pp. 822-833, April 2008.
[34] Z. Wang, H.R. Sheikh, and A.C. Bovik, "No-reference perceptual quality assessment of JPEG compressed images," presented at IEEE Int. Conf. on Image Processing, Sept. 2002.
[35] R. Ferzli , L.J. Karam, "A no-reference objective image sharpness metric based on the notion of just noticeable blur (JNB)," IEEE Trans. Image Processing, Vol. 18, pp. 717-728, April 2009.
[36] Z. Wang, A.C. Bovik, "Reduced-and no-reference image quality assessment: the natural scene statistic model approach," in IEEE Signal Processing Magazine, Vol. 28, Nov. 2011, pp. 29-40.
[37] A. Rehman, Z. Wang, "Reduced-reference image quality assessment by structural similarity estimation," IEEE Trans. Image Processing, Vol. 21, pp. 3378-3389, Aug. 2012.
[38] Z. Wang, E.P. Simoncelli, "Reduced-reference image quality assessment using a wavelet-domain natural image statistic model," Proc. SPIE, Vol. 5666, pp. 149-159, 2005.
[39] Z. Wang, G. Wu, H.R. Sheikh, E.P. Simoncelli, E.H. Yang, and A.C. Bovik, "Qualityaware images," IEEE Trans. Image Processing, Vol. 15, pp. 1680-1689, June 2006.
[40] Q. Li, Z. Wang, "Reduced-reference image quality assessment using divisive normalization-based image representation," IEEE Journal of Selected Topics in Signal Processing, Vol.3, pp. 202-211, April 2009.
[41] S. Wolf, M.H. Pinson, "Spatial-temporal
distortion metric for in-service quality monitoring of any digital video system," Proc. SPIE, Vol. 3845, pp. 266-277, Nov. 1999.
[42] T.M. Kusuma, H.J. Zepernick, "A reduced-reference perceptual quality metric for in-service image quality assessment," Joint First Workshop on Mobile Future and Symposium on Trends in Commun., pp. 71 -74, Oct. 2003.
[43] I.P. Gunawan, M. Ghanbari, "Reduced reference picture quality estimation by using local harmonic amplitude information," presented at London Commun. Symposium, Sept. 2003.
[44] K. Chono, Y.C. Lin, D. Varodayan, Y. Miyamoto, and B. Girod, "Reduced-reference image quality assessment using distributed source coding," in IEEE Int. Conf. on Multimedia and Expo, April 2008.
[45] M. Carnec, P. Le Callet, and D. Barba, "An image quality assessment method based on perception of structural information," in IEEE Int. Conf. on Image Processing, Sept 2003.
[46] M. Carnec, P. Le Callet, and D. Barba, "Visual features for image quality assessment with reduced reference," in IEEE Int. Conf. on Image Processing, Sept. 2005.
[47] L. Ma, S. Li, and K.N. Ngan, "Visual horizontal effect for image quality assessment," IEEE Signal Processing Letters, Vol. 17, pp.627-630, July 2010.
[48] F. Zhang, W. Liu, W. Lin, and K.N. Ngan, "Spread spectrum image watermarking based on perceptual quality metric," IEEE Trans. Image Processing, Vol. 20, pp. 3207-3218, Nov. 2011.
[49] Y. Niu, M. Kyan, L. Ma, A. Beghdadi, and S. Krishnan, "A visual saliency modulated just noticeable distortion profile for image watermarking," in European Signal Processing Conf., 2011.
[50] Z. Wang, A.C. Bovik, "Mean squared error: Love it or leave it? A new look at signal fidelity measures," in IEEE Signal Processing Magazine, Vol. 26, Jan. 2009, pp. 98-117.
[51] D.M. Chandler, S.S. Hemami, "A57 dataset," [Online].Available:http://foulard.ece.cornell.edu/dmc27/vsnr/vsnr.html.
[52] Z. Wang, “Rate scalable foveated image and video communications”. PhD thesis, Dept. of ECE, the University of Texas at Austin, Dec. 2001.
[53] Z. Wang , A.C. Bovik, "A universal image quality index," IEEE Signal Processing Letters, Vol. 9, pp. 81 -84, March 2002.
[54] A.M. Alattar, E.T. Lin, and M.U. Celik, "Digital watermarking of low bit-rate advanced simple profile MPEG-4 compressed video," IEEE Trans. Circ. Syst. Video Tech. , Vol. 13, pp. 787-800, Aug. 2003.
[55] E. Christophe, D. Leger, and C. Mailhes, "Quality criteria benchmark for hyperspectral imagery," IEEE Trans. Geosci. Remote Sensing, Vol. 43, pp. 2103-2114, Sept. 2005.
[56] L. Snidaro, G.L. Foresti, "A multi-camera approach to sensor evaluation in video
Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015
81
surveillance," presented at IEEE Int. Conf. on Image Processing, Sept. 2005.
[57] Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, "The SSIM index for image quality assessment,",[Online].Available: http://ece.uwaterloo.ca/~z70wang/research/ssim/
[58] Z. Wang, E.P. Simoncelli, and A.C. Bovik, "Multi-scale structural similarity for image quality assessment," [Online]. Available: http://ece.uwaterloo.ca/~z70wang/research/ssim/msssim.
[59] A. Srivastava, A.B. Lee, E.P. Simoncelli, and S.-C. Zhu, "On advances in statistical modeling of natural images," J. Math. Imag. Vis. , Vol. 18, pp. 17-33, Jan. 2003.
[60] M.J. Wainwright, E.P. Simoncelli, and A.S. Wilsky, "Random cascades on wavelet trees and their use in analyzing and modeling natural images," Appl. Comput. Harmon. Anal., Vol. 11, pp. 89-123, July 2001.
[61] V. Strela, J. Portilla, and E. P. Simoncelli, "Image denoising using a local Gaussian scale mixture model in the wavelet domain," Proc. SPIE, Vol. 4119, pp. 363-371, Dec. 2000.
[62] H.R. Sheikh, A.C. Bovik, "Visual information fidelity (VIF) measure for image quality assessment," [Online]. Available: http://live.ece.utexas.edu/research/quality/vifvec_release.
[63] J. Mannos, D.J. Sakrison, "The effects of a visual fidelity criterion of the encoding of images," IEEE Trans. Inf. Theory, vol. 20, pp. 525-536, July 1974.
[64] S. Daly, "Subroutine for the generation of a human visual contrast sensitivity function," Eastman Kodak Tech. Report 233203y, 1987.
[65] E C. Larson , D.M. Chandler, "Full-reference image quality assessment and the role of strategy: The most apparent distortion,"[Online]. Available: http://vision.okstate.edu/mad/.
[66] D. Marr, Vision. MIT Press, July 2010.
[67] D. Marr, E. Hildreth, "Theory of edge detection," Proc. R. Soc. Lond. B, Vol. 207, pp. 187-217, Feb. 1980.
[68] M.C. Morrone, D.C. Burr, "Feature detection in human vision: A phase-dependent energy model," Proc. R. Soc. Lond. B, Vol. 235, pp. 221 -245, Dec. 1988.
[69] M.C. Morrone, J. Ross, D.C. Burr, and R.A. Owens, "Mach bands are phase dependent," Nature, Vol. 324, pp. 250-253, Nov. 1986.
[70] M.C. Morrone, R.A. Owens, "Feature detection from local energy," Pattern Recognit. Letters, Vol. 6, pp. 303-313, Dec. 1987.
[71] P. Kovesi, "Image features from phase congruency," Videre: J. Comp. Vis. Res., Vol.1, pp. 1 -26, 1999.
[72] L. Henriksson, A. Hyvärinen, and S. Vanni, "Representation of cross-frequency spatial phase relationships in human visual cortex," J. Neurosci., Vol. 29, pp. 14342-14351, Nov. 2009.
[73] R. Jain, R. Kasturi, and B.G. Schunck, “Machine
vision.” ,McGraw-Hill, 1995.
[74] B. Jähne, H. Haussecker, and P. Geissler, “Handbook of computer vision and applications” , Academic Press, 1999.
[75] L. Zhang, L. Zhang, X. Mou, and D. Zhang, "FSIM: A feature similarity index for image quality assessment," [Online]. Available: http://www.comp.polyu.edu.hk/~cslzhang/IQA/FSIM/FSIM. tm.
[76] A. Toet , M. P. Lucassen, "A new universal colour image fidelity metric," Displays, Vol. 24, pp. 197-207, Dec. 2003.
[77] O.D. Faugeras, "Digital color image processing within the framework of a human visual model," IEEE Trans. Acoust. Speech Signal Processing, Vol. 27, pp. 380-393, Aug. 1979.
[78] P. Le Callet , D. Barba, "Perceptual color image quality metric using adequate error pooling for coding scheme evaluation," Proc. SPIE, vol. 4662, May 2002.
[79] Y.K Lai, J.Guo, and C.C.J. Kuo, "Perceptual fidelity measure of digital color images," Proc. SPIE, Vol. 3299, pp. 221-231, 2002.
[80] M.S. Lian, "Image evaluation using a color visual difference predictor (CVDP)," Proc. SPIE, Vol. 4299, June 2001.
[81] J. Preiss, F. Fernandes, and P. Urban, "Color-image quality assessment: From prediction to optimization," IEEE Trans. Image Processing, Vol. 23, pp. 1366-1378, March 2014.
[82] A. Kolaman, O. Yadid-Pecht, "Quaternion structural similarity: A new quality index for color images," IEEE Trans. Image Processing, Vol. 21, pp. 1526-1536, April 2012.
[83] C.C. Yang , S.H. Kwok, "Efficient gamut clipping for color image processing using LHS and YIQ," Opt. Eng., Vol. 42, pp. 701 -711, March 2003.
[84] E. Reinhard, M. Stark, P. Shirley, and J. Ferwerda, "Photographic tone reproduction for digital images," ACM Trans. Graph. , Vol. 21, pp. 267-276, 2002.
[85] G.W. Larson, H. Rushmeier, and C. Piatko, "A visibility matching tone reproduction operator for high dynamic range scenes," IEEE Trans. Visual. Comp. Graph. , Vol. 3, pp. 291 -306, 1997.
[86] R. Fattal, D. Lischinski, and M. Werman, "Gradient domain high dynamic range compression," ACM Trans. Graph. , Vol. 21, pp. 249-256, July 2002.
[87] F. Drago, K. Myszkowski, T. Annen, and N. Chiba, "Adaptive logarithmic mapping for displaying high contrast scenes," Computer Graphics Forum, Vol. 22, pp. 419-426, Sept. 2003.
[88] F. Drago, W. L. Martens, K. Myszkowski, and H.P. Siedel, "Perceptual evaluation of tone mapping operators," presented at ACM SIGGRAPH 2003 Sketches & Appl., 2003.
[89] J. Kuang, H. Yamaguchi, C. Liu, G.M. Johnson, and M.D. Fairchild, "Evaluating HDR rendering algorithms," ACM Trans. Appl. Perception, Vol. 4, July 2007.
[90] P. Ledda, A. Chalmers, T. Troscianko, and H.
Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015
82
Seetzen, "Evaluation of tone mapping operators using a high dynamic range display," ACM Trans. Graph. , Vol. 24, pp. 640-648, July 2005.
[91] A. Yoshida, V. Blanz, K. Myszkowski, and H.P. Siedel, "Perceptual evaluation of tone mapping operators with real-world scenes," Proc. SPIE, Vol. 5666, pp. 192-203, 2005.
[92] M. Čadik, M. Wimmer, L. Neumann, and A. Artusi, "Image attributes and quality for evaluation of tone mapping operators," in Proc. 14th Pacific Conf. on Comput. Graph. Appl., pp. 35-44, 2006..
[93] M. Barkowsky, P. Le Callet, "On the perceptual similarity of realistic looking tone mapped high dynamic range images," in IEEE Int. Conf. on Image Processing, Sept. 2010.
[94] R. Mantiuk, S.J. Daly, K. Myszkowski, and H.P. Siedel, "Predicting visible differences in high dynamic range images: model and its calibration," Proc. SPIE, Vol. 5666, pp. 204- 214, March 2005.
[95] A.B. Watson, "Visual detection of spatial contrast patterns: Evaluation of five simple models," Opt. Express, Vol. 6, pp. 12-33, 2000.
[96] [96] R. J. Deeley, N. Drasdo, and W. N. Charman, "A simple parametric model of the human ocular modulation transfer function," Ophthalmic and Physiol. Opt. , Vol. 11, pp. 91 -93, 1991.
[97] S.J. Daly, "Visible differences predictor: an algorithm for the assessment of image fidelity," Proc. SPIE, Vol. 1666, Aug 1992.
[98] T.O. Aydin, R. Mantiuk, K. Myszkowski, and H.P. Seidel, "Dynamic range independent metrics online," [Online]. Available: http://driiqm.mpi-inf.mpg.de/.
[99] J.P. Guilford, “Psychometric methods”, 2nd ed. McGraw-Hill, Dec. 1954.
[100] Y. Le Grand, “Light, color and vision”. Dover, 1957.
[101] P.G.J. Barten, “Contrast sensitivity of the human eye and its effects on image quality”, Vol. PM72. SPIE Press, Dec. 1999.
[102] W.J. Crozier, "On the variability of critical illumination for flicker fusion and intensity discrimination," J. General Physiol., Vol. 19, pp. 503-522, Jan. 1936.
[103] D.H. Kelly, "Effects of sharp edges on the visibility of sinusoidal gratings," J. Opt. Soc. Am., Vol. 60, pp. 98-102, 1970.
[104] M. Čadik, P. Slavik, "The naturalness of reproduced high dynamic range images," presented at 9th Int. Conf. on Inf. Visual., July 2005.
[105] "Computer vision test images," [Online]. Available: http://www.cs.cmu.edu/afs/cs/project/cil/www/v images.html.
[106] "UCID - uncompressed colour image database," [Online]. Available: http://homepages.lboro.ac.uk/~cogs/datasets/ucid/ucid.html.
[107] V. Mante, R.A. Frazor, V. Bonin, W.S. Geisler, and M. Carandini, "Independence of luminance and contrast in natural scenes and in the early visual
system," Nature Neurosci., Vol. 8, pp. 1690-1697, Nov. 2005.
[108] M. Song, D. Tao, C. Chen, J. Bu, J. Luo, and C. Zhang, "Probabilistic exposure fusion," IEEE Trans. Image Processing, Vol. 21, pp. 341 -357, Jan. 2012.
[109] P. Le Callet, F. Autrusseau, "Subjective quality assessment IRCCyN/IVC database," [Online]. Available: http://www.irccyn.ec-nantes.fr/ivcdb/.
[110] Y. Horita, K. Shibata, and Y. Kawayoke, "MICT image quality evaluation database," [Online]. Available: http://mict.eng.u-toyama.ac.jp/mictdb.html.
[111] Video quality expert group (VQEG), "Final report from the video quality experts group on the validation of objective models of video quality assessment II," [Online]. Available: http://www.vqeg.org.
[112] "Theatrical market statistics," MPAA. Washington, DC, USA [Online]. Available: http://www.mpaa.org/wp-content/uploads/2014 /03 /MPAA-Theatrical-Market-Statistics-2013_032514-v2.pdf., 2011.
[113] "List of 3-D movies," [Online]. Available: http://en.wikipedia.org/wiki/List_of_3- D_films, 2005.
[114] "ESPN 3-D broadcasting schedule," ESPN. Bristol, CT, USA [Online]. Available: http://espn.go.com/espntv/3d/.
[115] H. Lee, S. Cho, K. Yun, N. Hur, and J. Kim, "A backward-compatible, mobile, personalized 3DTV broadcasting system based on T-DMB," in Three-dimensional television, H. M. Ozaktas and L. Onural, Eds.: Springer Berlin Heidelberg, pp. 11 -28, 2008.
[116] C.T.E.R. Hewage, S.T. Worral, S. Dogan, and A.M. Kondoz, "Prediction of stereoscopic video quality using objective quality models of 2-D video," Electron. Lett., Vol. 44, pp. 963-965, Jul. 2008.
[117] C.T.E.R. Hewage, S.T. Worral, S. Dogan, S. Villette, and A.M. Kondoz, "Quality evaluation of color plus depth map-based stereoscopic video," IEEE Journal of Selected Topics in Signal Processing, Vol. 3, pp. 304-318, April 2009.
[118] S. Winkler , D. Min, "Stereo/multiview picture quality: Overview and recent advances," Signal Processing: Image Communication, Vol. 28, pp. 1358-1373, Nov. 2013.
[119] P. Lebreton, A. Raake, M. Barkowsky, and P. Le Callet, "Evaluating depth perception of 3D stereoscopic videos," IEEE Journal of Selected Topics in Signal Processing, Vol. 6, Oct. 2012.
[120] P.J. Seuntiëns, I.E. Heynderickx, W.A. IJsselsteijn, P.M.J. van den Avoort, J. Berentsen, I.J. Dalm, M.T. Lambooij, and W. Oosting, "Viewing experience and naturalness of 3D images," Proc. SPIE, Vol. 6016, Nov. 2005.
[121] W.A. IJsselsteijn, "Presence in depth," PhD dissertation: Eindhoven University of Technology, 2004.
[122] J. Hakala, "The added value of stereoscopy in
Majlesi Journal of Electrical Engineering Vol. 9, No. 1, March 2015
83
still images," Master's Thesis: Alato University, 2010.
[123] M.T.M. Lambooij, W.A. IJsselsteijn, and M.F. Fortuin, "Visual discomfort and visual fatigue of stereoscopic displays: A review," Journal of Imaging Technology and Science, Vol. 53, pp. 1 -14, 2009.
[124] ITU-R Recommendation BT.2021, "Subjective methods for the assessment of stereoscopic 3DTV systems," International Telecommunication Union, Geneva, Switzerland 2012.
[125] A.K. Moorthy, C.C. Su, A. Mittal, and A.C. Bovik, "Subjective evaluation of stereoscopic image quality," Signal Processing: Image Communication, Vol. 28, pp. 870- 883, Sept. 2013.
[126] A. Benoit, P. Le Callet, P. Campisi, and R. Cousseau, "Quality assessment of stereoscopic images," EURASIP Journal on Image and Video Processing,2008.
[127] Y.H. Lin, J.L. Wu, "Quality assessment of stereoscopic 3D image compression by binocular integration behaviors," IEEE Trans. Image Processing, Vol. 23, pp. 1527-1542, April 2014.
[128] P. Gorley, N. Holliman, "Stereoscopic image quality metrics and compression," Proc. SPIE, Vol. 6803, Feb. 2008.
[129] L. Shen, J. Yang, and Z. Zhang, "Stereo picture quality estimation based on a multiple channel HVS model," in Int. Congr. Image and Signal Processing. Tianjin, pp. 1 -4, 2009.
[130] C.T.E.R. Hewage and M.G. Martini, "Reduced-reference quality metric for 3D depth map transmission," in 3DTV-Conf. True Vis., Capture, Transmiss. Display 3D Video. Tampere, pp. 1 -4, 2010.
[131] R. Bensalma, M.-C. Larabi, "A perceptual metric for stereoscopic image quality assessment based on the binocular energy," Multidimen. Syst. Signal Processing, Vol. 24, pp. 281 -316, June 2013.
[132] V. De Silva, H.K. Arachchi, E. Ekmekcioglu, and A. Kondoz, "Toward an impairment metric for stereoscopic video: A full-reference video quality metric to assess compressed stereoscopic video," IEEE Trans. Image Processing, Vol. 22, pp. 3392-3404, Sept. 2013.
[133] A. Maalouf, M.C. Larabi, "CYCLOP: A stereo color image quality assessment metric," in IEEE Int. Conf. Acoust., Speech and Signal Processing. Prague, pp. 1161 -1164, 2011.
[134] R. Akhter, Z.M. Parvez Sazzad, Y. Horita, and J. Baltes, "No-reference stereoscopic image quality assessment," Proc. SPIE, Vol. 7524, Feb. 2010.
[135] M.J. Chen, L.K. Cormack, and A.C. Bovik, "No-reference quality assessment of natural stereopairs," IEEE Trans. Image Processing, Vol. 22, pp. 3379-3391, Sept. 2013.
[136] A.K. Moorthy, C.C. Su, and A.C. Bovik, "LIVE 3D image quality dataset," [Online]. Available: http://live.ece.utexas.edu/research/quality/live_3dimage.html, 2013.
[137] A. Benoit, P. Le Callet, P. Campisi, and R.
Cousseau, "IVC 3D images dataset," [Online]. Available:http://130.66.64.103/spip.php?article876&lang=.
[138] Z. Mai, C. Doutre, P. Nasiopoulos, and R.K. Ward, "Rendering 3 -D high dynamic range images: Subjective evaluation of tone-mapping methods and preferred 3-D image attributes," [Online]. Available:http://www.ece.ubc.ca/~zicongm/subjective_test_3dtmo/3D MO_HTMLReport .html, 2012
)