Time-Resolved 3D cardiopulmonary MRI reconstruction using spatial transformer network

Qing Zou; Zachary Miller; Sanja Dzelebdzic; Maher Abadeer; Kevin M. Johnson; Tarique Hussain; Qing Zou; Zachary Miller; Sanja Dzelebdzic; Maher Abadeer; Kevin M. Johnson; Tarique Hussain

doi:10.3934/mbe.2023712

Mathematical Biosciences and Engineering

2023, Volume 20, Issue 9: 15982-15998. doi: 10.3934/mbe.2023712

Previous Article Next Article

Research article Special Issues

Time-Resolved 3D cardiopulmonary MRI reconstruction using spatial transformer network

1.
Division of Pediatric Cardiology, Department of Pediatrics, The University of Texas Southwestern Medical Center, Dallas, TX, USA
2.
Department of Radiology, The University of Texas Southwestern Medical Center, Dallas, TX, USA
3.
Advanced Imaging Research Center, The University of Texas Southwestern Medical Center, Dallas, TX, USA
4.
Department of Biomedical Engineering, University of Wisconsin, Madison, WI, USA
5.
Department of Medical Physics, University of Wisconsin School of Medicine and Public Health, Madison, WI, USA
6.
Department of Radiology, University of Wisconsin School of Medicine and Public Health, Madison, WI, USA

Academic Editor: Shangce Gao

Received: 15 June 2023 Revised: 25 July 2023 Accepted: 27 July 2023 Published: 04 August 2023

The accurate visualization and assessment of the complex cardiac and pulmonary structures in 3D is critical for the diagnosis and treatment of cardiovascular and respiratory disorders. Conventional 3D cardiac magnetic resonance imaging (MRI) techniques suffer from long acquisition times, motion artifacts, and limited spatiotemporal resolution. This study proposes a novel time-resolved 3D cardiopulmonary MRI reconstruction method based on spatial transformer networks (STNs) to reconstruct the 3D cardiopulmonary MRI acquired using 3D center-out radial ultra-short echo time (UTE) sequences. The proposed reconstruction method employed an STN-based deep learning framework, which used a combination of data-processing, grid generator, and sampler. The reconstructed 3D images were compared against the start-of-the-art time-resolved reconstruction method. The results showed that the proposed time-resolved 3D cardiopulmonary MRI reconstruction using STNs offers a robust and efficient approach to obtain high-quality images. This method effectively overcomes the limitations of conventional 3D cardiac MRI techniques and has the potential to improve the diagnosis and treatment planning of cardiopulmonary disorders.
- cardiopulmonary MRI,
- spatial transformer network,
- 3D UTE sequence
Citation: Qing Zou, Zachary Miller, Sanja Dzelebdzic, Maher Abadeer, Kevin M. Johnson, Tarique Hussain. Time-Resolved 3D cardiopulmonary MRI reconstruction using spatial transformer network[J]. Mathematical Biosciences and Engineering, 2023, 20(9): 15982-15998. doi: 10.3934/mbe.2023712

Related Papers:

Abstract

The accurate visualization and assessment of the complex cardiac and pulmonary structures in 3D is critical for the diagnosis and treatment of cardiovascular and respiratory disorders. Conventional 3D cardiac magnetic resonance imaging (MRI) techniques suffer from long acquisition times, motion artifacts, and limited spatiotemporal resolution. This study proposes a novel time-resolved 3D cardiopulmonary MRI reconstruction method based on spatial transformer networks (STNs) to reconstruct the 3D cardiopulmonary MRI acquired using 3D center-out radial ultra-short echo time (UTE) sequences. The proposed reconstruction method employed an STN-based deep learning framework, which used a combination of data-processing, grid generator, and sampler. The reconstructed 3D images were compared against the start-of-the-art time-resolved reconstruction method. The results showed that the proposed time-resolved 3D cardiopulmonary MRI reconstruction using STNs offers a robust and efficient approach to obtain high-quality images. This method effectively overcomes the limitations of conventional 3D cardiac MRI techniques and has the potential to improve the diagnosis and treatment planning of cardiopulmonary disorders.

References

[1]	A. Hansell, J. Walk, J. Soriano, What do chronic obstructive pulmonary disease patients die from? Amultiple cause coding analysis, Eur. Respir. J., 22 (2003), 809–814. https://doi.org/10.1183/09031936.03.00031403 doi: 10.1183/09031936.03.00031403
[2]	R. Rubinsztajn, R. Chazan, An analysis of the causes of mortality and co-morbidities in hospitalised patients with chronic obstructive pulmonary disease, Adv. Respir. Med., 79 (2011), 343–346. https://doi.org/10.5603/ARM.27637 doi: 10.5603/ARM.27637
[3]	P. Sicard, Y. O. Khaniabadi, S. Perez, M. Gualtieri, A. De Marco, Effect of O$_3$, PM$_1$$_0$ and PM$_2$$_.$$_5$ on cardiovascular and respiratory diseases in cities of France, Iran and Italy, Environ. Sci. Pollut. Res., 26 (2019), 32645–32665. https://doi.org/10.1007/s11356-019-06445-8 doi: 10.1007/s11356-019-06445-8
[4]	L. C. Saunders, P. J. Hughes, S. Alabed, D. J. Capener, H. Marshall, J. Vogel-Claussen, et al., Integrated cardiopulmonary MRI assessment of pulmonary hypertension, J. Magn. Reson. Imaging, 55 (2022), 633–652. https://doi.org/10.1002/jmri.27849 doi: 10.1002/jmri.27849
[5]	O. Bane, S. J. Shah, M. J. Cuttica, J. D. Collins, S. Selvaraj, N. R. Chatterjee, et al., A non-invasive assessment of cardiopulmonary hemodynamics with MRI in pulmonary hypertension, Magn. Reson. Imaging, 33 (2015), 1224–1235. https://doi.org/10.1016/j.mri.2015.08.005 doi: 10.1016/j.mri.2015.08.005
[6]	J. A. Tkach, N. S. Higano, M. D. Taylor, R. A. Moore, M. Hossain, G. Huang, et al., Quantitative cardiopulmonary magnetic resonance imaging in neonatal congenital diaphragmatic hernia, Pediatr. Radiol., 52 (2022), 2306–2318. https://doi.org/10.1007/s00247-022-05384-w doi: 10.1007/s00247-022-05384-w
[7]	J. Y. Cheng, T. Zhang, M. T. Alley, M. Uecker, M. Lustig, J. M. Pauly, et al., Comprehensive multi-dimensional MRI for the simultaneous assessment of cardiopulmonary anatomy and physiology, Sci. Rep., 7 (2017), 1–15. https://doi.org/10.1038/s41598-017-04676-8 doi: 10.1038/s41598-017-04676-8
[8]	A. Vonk Noordegraaf, F. Haddad, H. J. Bogaard, P. M. Hassoun, Noninvasive imaging in the assessment of the cardiopulmonary vascular unit, Circulation, 131 (2015), 899–913. https://doi.org/10.1161/CIRCULATIONAHA.114.006972 doi: 10.1161/CIRCULATIONAHA.114.006972
[9]	R. A. Lafountain, J. S. da Silveira, J. Varghese, G. Mihai, D. Scandling, J. Craft, et al., Cardiopulmonary exercise testing in the MRI environment, Physiol. Meas., 37 (2016), N11. https://doi.org/10.1088/0967-3334/37/4/N11 doi: 10.1088/0967-3334/37/4/N11
[10]	R. Jogiya, A. Schuster, A. Zaman, M. Motwani, M. Kouwenhoven, E. Nagel, et al., Three-dimensional balanced steady state free precession myocardial perfusion cardiovascular magnetic resonance at 3T using dual-source parallel RF transmission: initial experience, J. Cardiovasc. Magn. Reson., 16 (2014), 1–10. https://doi.org/10.1186/s12968-014-0090-0 doi: 10.1186/s12968-014-0090-0
[11]	C. Lin, M. A. Bernstein, 3D magnetization prepared elliptical centric fast gradient echo imaging, Magn. Reson. Med., 59 (2008), 434–439.
[12]	S. Potthast, L. Mitsumori, L. A. Stanescu, M. L. Richardson, K. Branch, T. J. Dubinsky, et al., Measuring aortic diameter with different MR techniques: Comparison of three-dimensional (3D) navigated steady-state free-precession (SSFP), 3D contrast-enhanced magnetic resonance angiography (CE-MRA), 2D T2 black blood, and 2D cine SSFP, J. Magn. Reson. Imaging, 31 (2010), 177–184. https://doi.org/10.1002/jmri.22016 doi: 10.1002/jmri.22016
[13]	M. J. Kim, D. G. Mitchell, K. Ito, P. N. Kim, Hepatic MR imaging: comparison of 2D and 3D gradient echo techniques, Abdom. Imaging, 26 (2001), 269–276. https://doi.org/10.1007/s002610000177 doi: 10.1007/s002610000177
[14]	S. J. Kruger, S. B. Fain, K. M. Johnson, R. V. Cadman, S. K. Nagle, Oxygen-enhanced 3D radial ultrashort echo time magnetic resonance imaging in the healthy human lung, NMR Biomed., 27 (2014), 1535–1541. https://doi.org/10.1002/nbm.3158 doi: 10.1002/nbm.3158
[15]	W. Zha, S. J. Kruger, K. M. Johnson, R. V. Cadman, L. C. Bell, F. Liu, et al., Pulmonary ventilation imaging in asthma and cystic fibrosis using oxygen-enhanced 3D radial ultrashort echo time MRI, J. Magn. Reson. Imaging, 47 (2018), 1287–1297. https://doi.org/10.1002/jmri.25877 doi: 10.1002/jmri.25877
[16]	K. S. Sodhi, A. Bhatia, P. Rana, J. L. Mathew, Impact of radial percentage k-space filling and signal averaging on native lung MRI image quality in 3D radial UTE acquisition: A pilot study, Acad. Radiol., 2023 (2023). https://doi.org/10.1016/j.acra.2023.01.029 doi: 10.1016/j.acra.2023.01.029
[17]	M. D. Robson, P. D. Gatehouse, M. Bydder, G. M. Bydder, Magnetic resonance: an introduction to ultrashort TE (UTE) imaging, J. Comput. Assisted Tomogr., 27 (2003), 825–846. https://doi.org/10.1097/00004728-200311000-00001 doi: 10.1097/00004728-200311000-00001
[18]	R. Shekhar, V. Zagrodsky, Cine MPR: interactive multiplanar reformatting of four-dimensional cardiac data using hardware-accelerated texture mapping, IEEE Trans. Inf. Technol. Biomed., 7 (2003), 384–393. https://doi.org/10.1109/TITB.2003.821320 doi: 10.1109/TITB.2003.821320
[19]	K. T. Block, H. Chandarana, S. Milla, M. Bruno, T. Mulholland, G. Fatterpekar, et al., Towards routine clinical use of radial stack-of-stars 3D gradient-echo sequences for reducing motion sensitivity, J. Korean Soc. Magn. Reson. Med., 18 (2014), 87–106. https://doi.org/10.13104/jksmrm.2014.18.2.87 doi: 10.13104/jksmrm.2014.18.2.87
[20]	J. E. Park, Y. H. Choi, J. E. Cheon, W. S. Kim, I. O. Kim, Y. J. Ryu, et al., Three-dimensional radial vibe sequence for contrast-enhanced brain imaging: an alternative for reducing motion artifacts in restless children, AJR Am. J. Roentgenol., 210 (2018), 876–882. https://doi.org/10.2214/AJR.17.18490 doi: 10.2214/AJR.17.18490
[21]	J. Cao, D. Zhao, C. Tian, T. Jin, F. Song, Adopting improved adam optimizer to train dendritic neuron model for water quality prediction, Math. Biosci. Eng., 20 (2023), 9489–9510. https://doi.org/10.3934/mbe.2023417 doi: 10.3934/mbe.2023417
[22]	T. Jin, F. Li, H. Peng, B. Li, D. Jiang, Uncertain barrier swaption pricing problem based on the fractional differential equation in caputo sense, Soft Comput., 2023 (2023), 1–16. https://doi.org/10.1007/s00500-023-08153-5 doi: 10.1007/s00500-023-08153-5
[23]	Q. Zou, A. H. Ahmed, P. Nagpal, S. Kruger, M. Jacob, Dynamic imaging using a deep generative storm (gen-storm) model, IEEE Trans. Med. Imaging, 40 (2021), 3102–3112. https://doi.org/10.1109/TMI.2021.3065948 doi: 10.1109/TMI.2021.3065948
[24]	R. Hou, F. Li, IDPCNN: Iterative denoising and projecting CNN for MRI reconstruction, J. Comput. Appl. Math., 406 (2022), 113973.
[25]	Z. Wang, R. Wu, Y. Xu, Y. Liu, R. Chai, H. Ma, A two-stage CNN method for MRI image segmentation of prostate with lesion, Biomed. Signal Process. Control, 82 (2023), 104610. https://doi.org/10.1016/j.bspc.2023.104610 doi: 10.1016/j.bspc.2023.104610
[26]	D. I. Zaridis, E. Mylona, N. Tachos, V. C. Pezoulas, G. Grigoriadis, N. Tsiknakis, et al., Region-adaptive magnetic resonance image enhancement for improving CNN-based segmentation of the prostate and prostatic zones, Sci. Rep., 13 (2023), 714. https://doi.org/10.1038/s41598-023-27671-8 doi: 10.1038/s41598-023-27671-8
[27]	S. Shojaei, M. S. Abadeh, Z. Momeni, An evolutionary explainable deep learning approach for Alzheimer's MRI classification, Expert Syst. Appl., 220 (2023), 119709. https://doi.org/10.1016/j.eswa.2023.119709 doi: 10.1016/j.eswa.2023.119709
[28]	O. Özkaraca, O. İ. Bağrıaçık, H. Gürüler, F. Khan, J. Hussain, J. Khan, et al., Multiple brain tumor classification with dense CNN architecture using brain MRI images, Life, 13 (2023), 349. https://doi.org/10.3390/life13020349 doi: 10.3390/life13020349
[29]	S. Hardaha, D. R. Edla, S. R. Parne, A survey on convolutional neural networks for MRI analysis, Wireless Pers. Commun., 128 (2023), 1065–1085. https://doi.org/10.1007/s11277-022-09989-0 doi: 10.1007/s11277-022-09989-0
[30]	N. R. Huttinga, C. A. Van den Berg, P. R. Luijten, A. Sbrizzi, MR-MOTUS: model-based non-rigid motion estimation for MR-guided radiotherapy using a reference image and minimal k-space data, Phys. Med. Biol., 65 (2020), 015004. https://doi.org/10.1088/1361-6560/ab554a doi: 10.1088/1361-6560/ab554a
[31]	Z. Miller, L. Torres, S. Fain, K. Johnson, Motion compensated extreme MRI: Multi-scale low rank reconstructions for highly accelerated 3D dynamic acquisitions (MoCo-MSLR), preprint, arXiv: 2205.00131.
[32]	Q. Zou, L. A. Torres, S. B. Fain, N. S. Higano, A. J. Bates, M. Jacob, Dynamic imaging using motion-compensated smoothness regularization on manifolds (MoCo-SToRM), Phys. Med. Biol., 67 (2022), 144001. https://doi.org/10.1109/ISBI52829.2022.9761440 doi: 10.1109/ISBI52829.2022.9761440
[33]	M. Jaderberg, K. Simonyan, A. Zisserman, Spatial transformer networks, in Advances in Neural Information Processing Systems, 28 (2015).
[34]	F. Ong, X. Zhu, J. Y. Cheng, K. M. Johnson, P. E. Larson, S. S. Vasanawala, et al., Extreme MRI: Large-scale volumetric dynamic imaging from continuous non-gated acquisitions, Magn. Reson. Med., 84 (2020), 1763–1780. https://doi.org/10.1002/mrm.28235 doi: 10.1002/mrm.28235
[35]	O. Maier, S. H. Baete, A. Fyrdahl, K. Hammernik, S. Harrevelt, L. Kasper, et al., CG‐SENSE revisited: Results from the first ISMRM reproducibility challenge, Magn. Reson. Med., 85 (2021), 1821–1839. https://doi.org/10.1002/mrm.28569 doi: 10.1002/mrm.28569
[36]	N. A. Gumerov, R. Duraiswami, Fast multipole method based filtering of non-uniformly sampled data, preprint, arXiv: 1611.09379.
[37]	M. J. Muckley, R. Stern, T. Murrell, F. Knoll, Torchkbnufft: A high-level, hardware-agnostic non-uniform fast fourier transform, in ISMRM Workshop on Data Sampling & Image Reconstruction, (2020), 22.
[38]	D. Strong, T. Chan, Edge-preserving and scale-dependent properties of total variation regularization, Inverse Probl., 19 (2003), S165. https://doi.org/10.1088/0266-5611/19/6/059 doi: 10.1088/0266-5611/19/6/059
[39]	S. Imambi, K. B. Prakash, G. Kanagachidambaresan, Pytorch, in Programming with TensorFlow: Solution for Edge Computing Applications, (2021), 87–104. https://doi.org/10.1007/978-3-030-57077-4
[40]	D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, preprint, arXiv: 1412.6980.
[41]	A. F. Agarap, Deep learning using rectified linear units (RELU), preprint, arXiv: 1803.08375.
[42]	K. M. Johnson, S. B. Fain, M. L. Schiebler, S. Nagle, Optimized 3D ultrashort echo time pulmonary MRI, Magn. Reson. Med., 70 (2013), 1241–1250. https://doi.org/10.1002/mrm.24570 doi: 10.1002/mrm.24570
[43]	M. Buehrer, K. P. Pruessmann, P. Boesiger, S. Kozerke, Array compression for MRI with large coil arrays, Magn. Reson. Med., 57 (2007), 1131–1139. https://doi.org/10.1002/mrm.21237 doi: 10.1002/mrm.21237
[44]	X. Zhu, M. Chan, M. Lustig, K. M. Johnson, P. E. Larson, Iterative motion-compensation reconstruction ultra-short TE (iMoCo UTE) for high-resolution free-breathing pulmonary MRI, Magn. Reson. Med., 83 (2020), 1208–1221. https://doi.org/10.1002/mrm.27998 doi: 10.1002/mrm.27998
[45]	S. Napel, M. P. Marks, G. D. Rubin, M. D. Dake, C. H. McDonnell, S. M. Song, et al., CT angiography with spiral CT and maximum intensity projection, Radiology, 185 (1992), 607–610. https://doi.org/10.1148/radiology.185.2.1410382 doi: 10.1148/radiology.185.2.1410382
[46]	J. F. Gruden, S. Ouanounou, S. Tigges, S. D. Norris, T. S. Klausner, Incremental benefit of maximum-intensity-projection images on observer detection of small pulmonary nodules revealed by multidetector CT, Am. J. Roentgenol., 179 (2002), 149–157. https://doi.org/10.2214/ajr.179.1.1790149 doi: 10.2214/ajr.179.1.1790149

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)