Unified implementation of global high dynamic range image tone-mapping algorithms

Ishtiaq Rasool Khan; Susanto Rahardja; Ishtiaq Rasool Khan; Susanto Rahardja

doi:10.3934/mbe.2022215

Mathematical Biosciences and Engineering

2022, Volume 19, Issue 5: 4643-4656. doi: 10.3934/mbe.2022215

Previous Article Next Article

Research article

Unified implementation of global high dynamic range image tone-mapping algorithms

Ishtiaq Rasool Khan ^{1
,
,},
Susanto Rahardja ²

1.
Department of Computer Science and Artificial Intelligence, College of Computer Science and Engineering, University of Jeddah, Jeddah 21589, Saudi Arabia
2.
School of Marine Science and Technology, Northwestern Polytechnical University, 127 West Youyi Road, Xi'an, Shaanxi 710072, China

Received: 30 September 2021 Revised: 04 December 2021 Accepted: 19 January 2022 Published: 07 March 2022

High dynamic range (HDR) images and video require tone-mapping for display on low dynamic range (LDR) screens. Many tone-mapping operators have been proposed to convert HDR content to LDR, but almost each has a different implementation structure and requires a different execution time. We propose a unified structure that can represent any global tone-mapping algorithm with an array of just 256 coefficients. These coefficients extracted offline for every HDR image or video frame can be used to convert them to LDR in real time using linear interpolation. The produced LDR images are identical to the images produced by the original implementation of the algorithm. This unified implementation can replicate any global tone-mapping function and requires very low and fixed execution time, which is independent of algorithm and type of content and depends only on image size. Experimental studies are presented to show the accuracy and time efficiency of the proposed implementation.
- high dynamic range imaging,
- tone-mapping,
- unified implementation of TMO,
- HDR display,
- generic TMO design
Citation: Ishtiaq Rasool Khan, Susanto Rahardja. Unified implementation of global high dynamic range image tone-mapping algorithms[J]. Mathematical Biosciences and Engineering, 2022, 19(5): 4643-4656. doi: 10.3934/mbe.2022215

Related Papers:

Abstract

High dynamic range (HDR) images and video require tone-mapping for display on low dynamic range (LDR) screens. Many tone-mapping operators have been proposed to convert HDR content to LDR, but almost each has a different implementation structure and requires a different execution time. We propose a unified structure that can represent any global tone-mapping algorithm with an array of just 256 coefficients. These coefficients extracted offline for every HDR image or video frame can be used to convert them to LDR in real time using linear interpolation. The produced LDR images are identical to the images produced by the original implementation of the algorithm. This unified implementation can replicate any global tone-mapping function and requires very low and fixed execution time, which is independent of algorithm and type of content and depends only on image size. Experimental studies are presented to show the accuracy and time efficiency of the proposed implementation.

References

[1]	E. Reinhard, G. Ward, S. Pattanaik, P. Debevec, High Dynamic Range Imaging: Acquisition, Display and Image-Based Lighting, 2nd Edition. Morgan Kaufmann, 2010.
[2]	X. Cerda-Company, C. A. Parraga, X. Otazu, Which tone-mapping operator is the best? A comparative study of perceptual quality, J. Opt. Soc. Am., 35 (2018), 626–638. https://doi.org/10.1364/JOSAA.35.000626 doi: 10.1364/JOSAA.35.000626
[3]	G. W. Larson, H. Rushmeier, C. Piatko, A visibility matching tone reproduction operator for high dynamic range scenes, IEEE Trans. Vis. Comput. Graph., 3 (1997), 291–306. https://doi.org/10.1109/2945.646233 doi: 10.1109/2945.646233
[4]	I. R. Khan, S. Rahardja, M. M. Khan, M. M. Movania, F. Abed, A Tone-Mapping Technique Based on Histogram Using a Sensitivity Model of the Human Visual System, IEEE Trans. Ind. Electron., 65 (2018), 3469–3479. https://doi.org/10.1109/TIE.2017.2760247 doi: 10.1109/TIE.2017.2760247
[5]	E. Reinhard, M. Stark, P. Shirley, J. Ferwerda, Photographic tone reproduction for digital images, ACM Trans. Graph., 21 (2002), 267–276. https://doi.org/10.1145/566654.566575 doi: 10.1145/566654.566575
[6]	J. De Praeter, A. J. Díaz-Honrubia, T. Paridaens, G. Van Wallendael, P. Lambert, Simultaneous encoder for high-dynamic-range and low-dynamic-range video, IEEE Trans. Consum. Electron., 62 (2016), 420–428. https://doi.org/10.1109/TCE.2016.7838095 doi: 10.1109/TCE.2016.7838095
[7]	M. H. Kim, J. Kautz, Consistent tone reproduction, Proc. Tenth IASTED Int. Conf. Comput. Graph. Imag., ACTA Press Anaheim, 2008,152–159.
[8]	F. Drago, K. Myszkowski, T. Annen, N. Chiba, Adaptive Logarithmic Mapping for Displaying High Contrast Scenes, Comput. Graph. Forum, 22 (2003), 419–426. https://doi.org/10.1111/1467-8659.00689 doi: 10.1111/1467-8659.00689
[9]	M. Qiao, M. K. Ng, Tone mapping for high-dynamic-range images using localized gamma correction, J. Electron. Imag., 24 (2015), 013010. https://doi.org/10.1117/1.JEI.24.1.013010 doi: 10.1117/1.JEI.24.1.013010
[10]	J. Duan, M. Bressan, C. Dance, G. Qiu, Tone-mapping high dynamic range images by novel histogram adjustment, Pattern Recognit., 43 (2010), 1847–1862. https://doi.org/10.1016/j.patcog.2009.12.006 doi: 10.1016/j.patcog.2009.12.006
[11]	A. Boschetti, N. Adami, R. Leonardi, M. Okuda, High dynamic range image tone mapping based on local Histogram Equalization, IEEE Int. Conf. Multimedia Expo (ICME), (2010), 1130–1135. https://doi.org/10.1109/ICME.2010.5583305
[12]	J. Han, I. R. Khan, S. Rahardja, Lighting condition adaptive tone mapping method, ACM SIGGRAPH Posters, (2018). https://doi.org/10.1145/3230744.3230773
[13]	H. Li, X. Jia, L. Zhang, Clustering based content and color adaptive tone mapping, Comput. Vis. Image Underst., 168 (2018), 37–49. https://doi.org/10.1016/j.cviu.2017.11.001 doi: 10.1016/j.cviu.2017.11.001
[14]	W. U. Lee, S. Park, S. J. Ko, Image fusion-based tone mapping using gaussian mixture model clustering, IEEE Int. Conf. Consum. Electron., (2020).
[15]	M. Oskarsson, Temporally Consistent Tone Mapping of Images and Video Using Optimal K-means Clustering, J. Math. Imag. Vis., 57 (2017), 225–238. https://doi.org/10.1007/s10851-016-0677-1 doi: 10.1007/s10851-016-0677-1
[16]	S. N. Pattanaik, J. A. Ferwerda, M. D. Fairchild, D. P. Greenberg, A multiscale model of adaptation and spatial vision for realistic image display, Conf. Comput. Graph. Interact. Tech. (SIGGRAPH), (1998), 287–298. https://doi.org/10.1145/280814.280922
[17]	J. Kuang, G. M. Johnson, M. D. Fairchild, iCAM06: A refined image appearance model for HDR image rendering, J. Vis. Commun. Image Represent., 18 (2007), 406–414. https://doi.org/10.1016/j.jvcir.2007.06.003 doi: 10.1016/j.jvcir.2007.06.003
[18]	N. H. Nguyen, T. Van Vo, Y. Jeong, Y. Moon, C. Lee, Optimized Tone Mapping of HDR Images via HVS Model-Based 2D Histogram Equalization, APSIPA Trans. Signal Inf. Process., (2018), 700–704. https://doi.org/10.23919/APSIPA.2018.8659452
[19]	I. R. Khan, W. Aziz, S. O. Shim, Tone-Mapping Using Perceptual-Quantizer and Image Histogram, IEEE Access, 8 (2020), 31350–31358. https://doi.org/10.1109/ACCESS.2020.2973273 doi: 10.1109/ACCESS.2020.2973273
[20]	T. Jinno, M. Okuda, N. Adami, New local tone mapping and two-layer coding for HDR images, IEEE Int. Conf. Acoust. Speech Signal Process., (ICASSP), (2012), 765–768. https://doi.org/10.1109/ICASSP.2012.6287996
[21]	B. K. Kim, R. H. Park, S. K. Chang, Tone mapping with contrast preservation and lightness correction in high dynamic range imaging, Signal Image Video Process., 10 (2016), 1425–1432. https://doi.org/10.1007/s11760-016-0942-1 doi: 10.1007/s11760-016-0942-1
[22]	S. Rahardja, F. Farbiz, C. Manders, Z. Huang, J. Ling, I. Khan, et al., Eye hdr: Gaze-adaptive system for displaying high-dynamic-range images, SIGGRAPH ASIA Art Gallery & Emerging Technologies: Adaptation, (2009), 68. https://doi.org/10.1145/1665137.1665187
[23]	C. Guo, X. Jiang, Deep Tone-Mapping Operator Using Image Quality Assessment Inspired Semi-Supervised Learning, IEEE Access, 9 (2021), 73873–73889. https://doi.org/10.1109/ACCESS.2021.3080331 doi: 10.1109/ACCESS.2021.3080331
[24]	Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Process., 13 (2004), 600–612. https://doi.org/10.1109/TIP.2003.819861 doi: 10.1109/TIP.2003.819861
[25]	L. Zhang, L. Zhang, X. Mou, D. Zhang, FSIM: A feature similarity index for image quality assessment, IEEE Trans. Image Process., 20 (2011), 2378–2386. https://doi.org/10.1109/TIP.2011.2109730 doi: 10.1109/TIP.2011.2109730
[26]	H. Yeganeh, Z. Wang, Objective quality assessment of tone-mapped images, IEEE Trans. Image Process., 22 (2013), 657–667. https://doi.org/10.1109/TIP.2012.2221725 doi: 10.1109/TIP.2012.2221725
[27]	Z. H. Nafchi, A. Shahkolaei, R. Farrahi Moghaddam, M. Cheriet, FSITM: A Feature Similarity Index For Tone-Mapped Images, IEEE Signal Process. Lett., 22 (2015), 1026–1029. https://doi.org/10.1109/LSP.2014.2381458 doi: 10.1109/LSP.2014.2381458
[28]	BT.500: Methodologies for the subjective assessment of the quality of television images, available from: https://www.itu.int/rec/R-REC-BT.500 (accessed Feb. 18, 2022).
[29]	G. Somanath, D. Kurz, HDR Environment Map Estimation for Real-Time Augmented Reality, IEEE/CVF Conf. Comput. Vision Pattern Recognit. (CVPR), (2021), 11298–11306. https://doi.org/10.1109/CVPR46437.2021.01114
[30]	A. Khaldieh, S. Ploumis, M. T. Pourazad, P. Nasiopoulos, V. Leung, Tone mapping for video gaming applications, 2018 IEEE Int. Conf. Consumer Electron. (ICCE), (2018), 1–12. https://doi.org/10.1109/ICCE.2018.8326304
[31]	M. Purohit, M. Singh, A. Kumar, B. K. Kaushik, Enhancing the Surveillance Detection Range of Image Sensors Using HDR Techniques, IEEE Sensors J., 21 (2021), 19516–19528. https://doi.org/10.1109/JSEN.2021.3091018 doi: 10.1109/JSEN.2021.3091018
[32]	S. T. A. Ali, A. H. Usama, I. R. Khan, M. M. Khan, A. Siddiq, Mobile registration number plate recognition using artificail intelligence, 2021 IEEE Int. Conf. Image Process. (ICIP), (2021), 944–948. https://doi.org/10.1109/ICIP42928.2021.9506699
[33]	J. Ahmed, B. Gao, W. L. Woo, Sparse Low-Rank Tensor Decomposition for Metal Defect Detection Using Thermographic Imaging Diagnostics, IEEE Trans. Ind. Inform., 17 (2021), 1810–1820. https://doi.org/10.1109/TII.2020.2994227 doi: 10.1109/TII.2020.2994227
[34]	B. Hu, B. Gao, W. L. Woo, L. Ruan, J. Jin, Y. Yang, et al., A Lightweight Spatial and Temporal Multi-Feature Fusion Network for Defect Detection, IEEE Trans. Image Process., 30 (2021), 472–486. https://doi.org/10.1109/TIP.2020.3036770 doi: 10.1109/TIP.2020.3036770
[35]	S. S. Kaderuppan, E. W. L. Wong, A. Sharma, W. L. Woo, Smart Nanoscopy: A Review of Computational Approaches to Achieve Super-Resolved Optical Microscopy, IEEE Access, 8 (2020), 214801–214831. https://doi.org/10.1109/ACCESS.2020.3040319 doi: 10.1109/ACCESS.2020.3040319

Reader Comments

Your name:*

Email:*
© 2022 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)