Non-negative Tucker decomposition with double constraints for multiway dimensionality reduction

Xiang Gao; Linzhang Lu; Qilong Liu; Xiang Gao; Linzhang Lu; Qilong Liu

doi:10.3934/math.20241058

AIMS Mathematics

2024, Volume 9, Issue 8: 21755-21785. doi: 10.3934/math.20241058

Previous Article Next Article

Research article

Non-negative Tucker decomposition with double constraints for multiway dimensionality reduction

1.
School of Mathematical Sciences, Guizhou Normal University, Guiyang 550025, China
2.
School of Mathematical Sciences, Xiamen University, Xiamen 361005, China

Received: 23 May 2024 Revised: 28 June 2024 Accepted: 01 July 2024 Published: 08 July 2024
MSC : 15A69, 62H30, 68U10

Nonnegative Tucker decomposition (NTD) is one of the renowned techniques in feature extraction and representation for nonnegative high-dimensional tensor data. The main focus behind the NTD-like model was how to factorize the data to get ahold of a high quality data representation from multidimensional directions. However, existing NTD-like models do not consider relationship and properties between the factor matrix of columns while preserving the geometric structure of the data space. In this paper, we managed to capture nonlinear local features of data space and further enhance expressiveness of the NTD clustering method by syncretizing organically approximately orthogonal constraint and graph regularized constraint. First, based on the uni-side and bi-side approximate orthogonality, we flexibly proposed two novel approximately orthogonal NTD with graph regularized models, which not only in part make the factor matrix tend to be orthogonality, but also preserve the geometrical information from high-dimensional tensor data. Second, we developed the iterative updating algorithm dependent on the multiplicative update rule to solve the proposed models, and provided its convergence and computational complexity. Finally, we used numerical experimental results to demonstrate the effectiveness, robustness, and efficiency of the proposed new methods on the real-world image datasets.
- tensor factorization,
- non-negative Tucker decomposition,
- graph regularized,
- approximate orthogonality
Citation: Xiang Gao, Linzhang Lu, Qilong Liu. Non-negative Tucker decomposition with double constraints for multiway dimensionality reduction[J]. AIMS Mathematics, 2024, 9(8): 21755-21785. doi: 10.3934/math.20241058

Related Papers:

Abstract

Nonnegative Tucker decomposition (NTD) is one of the renowned techniques in feature extraction and representation for nonnegative high-dimensional tensor data. The main focus behind the NTD-like model was how to factorize the data to get ahold of a high quality data representation from multidimensional directions. However, existing NTD-like models do not consider relationship and properties between the factor matrix of columns while preserving the geometric structure of the data space. In this paper, we managed to capture nonlinear local features of data space and further enhance expressiveness of the NTD clustering method by syncretizing organically approximately orthogonal constraint and graph regularized constraint. First, based on the uni-side and bi-side approximate orthogonality, we flexibly proposed two novel approximately orthogonal NTD with graph regularized models, which not only in part make the factor matrix tend to be orthogonality, but also preserve the geometrical information from high-dimensional tensor data. Second, we developed the iterative updating algorithm dependent on the multiplicative update rule to solve the proposed models, and provided its convergence and computational complexity. Finally, we used numerical experimental results to demonstrate the effectiveness, robustness, and efficiency of the proposed new methods on the real-world image datasets.

References

[1]	P. Deng, T. Li, H. Wang, S. Horng, Z. Yu, X. Wang, Tri-regularized non-negative matrix tri-factorization for co-clustering, Knowl. Based Syst., 226 (2021), 107101. https://doi.org/10.1016/j.knosys.2021.107101 doi: 10.1016/j.knosys.2021.107101
[2]	S. Li, W. Li, J. Hu, Y. Li, Semi-supervised bi-orthogonal constraints dual-graph regularized NMF for subspace clustering, Appl. Intell., 52 (2022), 3227–3248. https://doi.org/10.1007/s10489-021-02522-z doi: 10.1007/s10489-021-02522-z
[3]	B. Cai, G. Lu, Tensor subspace clustering using consensus tensor low-rank representation, Inf. Sci., 609 (2022), 46–59. https://doi.org/10.1016/j.ins.2022.07.049 doi: 10.1016/j.ins.2022.07.049
[4]	M. Wall, A. Rechtsteiner, L. Rocha, Singular value decomposition and principal component analysis, In: D. P. Berrar, W. Dubitzky, M. Granzow, A practical approach to microarray data analysis, Springer, 2003, 91–109. https://doi.org/10.1007/0-306-47815-3_5
[5]	S. Roweis, L. Saul, Nonlinear dimensionality reduction by locally linear embedding, Science, 290 (2000), 2323–2326. https://doi.org/10.1126/science.290.5500.2323 doi: 10.1126/science.290.5500.2323
[6]	W. Yin, Z. Ma, LE and LLE regularized nonnegative Tucker decomposition for clustering of high dimensional datasets, Neurocomputing, 364 (2019), 77–94. https://doi.org/10.1016/j.neucom.2019.06.054 doi: 10.1016/j.neucom.2019.06.054
[7]	A. Gersho, R. Gray, Vector quantization and signal compression, Springer, 2012. https://doi.org/10.1007/978-1-4615-3626-0
[8]	S. Wold, K. Esbensen, P. Geladi, Principal component analysis, Chemometr. Intell. Lab. Syst., 2 (1987), 37–52. https://doi.org/10.1016/0169-7439(87)80084-9 doi: 10.1016/0169-7439(87)80084-9
[9]	Y. Zhao, C. Jiao, M. Wang, J. Liu, J. Wang, C. Zheng, Htrpca: hypergraph regularized tensor robust principal component analysis for sample clustering in tumor omics data, Interdiscip. Sci., 14 (2022), 22–33. https://doi.org/10.1007/s12539-021-00441-8 doi: 10.1007/s12539-021-00441-8
[10]	D. Lee, H. Seung, Learning the parts of objects by non-negative matrix factorization, Nature, 401 (1999), 788–791. https://doi.org/10.1038/44565 doi: 10.1038/44565
[11]	D. Lee, H. Seung, Algorithms for non-negative matrix factorization, Adv. Neural Inf. Process. Syst., 13 (2000), 556–562.
[12]	P. De Handschutter, N. Gillis, A consistent and ﬂexible framework for deep matrix factorizations, Pattern Recogn., 134 (2023), 109102. https://doi.org/10.1016/j.patcog.2022.109102 doi: 10.1016/j.patcog.2022.109102
[13]	Z. Wang, P. Dellaportas, I. Kosmidis, Bayesian tensor factorisations for time series of counts, Mach. Learn., 113 (2023), 3731–3750. https://doi.org/10.1007/s10994-023-06441-7 doi: 10.1007/s10994-023-06441-7
[14]	B. Chen, J. Guan, Z. Li, Unsupervised feature selection via graph regularized non-negative CP decomposition, IEEE Trans. Pattern Anal. Mach. Intell., 45 (2022), 2582–2594. https://doi.org/10.1109/TPAMI.2022.3160205 doi: 10.1109/TPAMI.2022.3160205
[15]	M. Che, Y. Wei, Randomized algorithms for the approximations of Tucker and the tensor train decompositions, Adv. Comput. Math., 45 (2019), 395–428. https://doi.org/10.1007/s10444-018-9622-8 doi: 10.1007/s10444-018-9622-8
[16]	T. Kolda, B. Bader, Tensor decompositions and applications, SIAM Rev., 51 (2009), 455–500. https://doi.org/10.1137/07070111X doi: 10.1137/07070111X
[17]	Y. Kim, S. Choi, Nonnegative Tucker decomposition, 2007 IEEE Conference on Computer Vision and Pattern Recognition, 2007. https://doi.org/10.1109/CVPR.2007.383405
[18]	H. Huang, Z. Ma, G. Zhang, Dimensionality reduction of tensors based on manifold-regularized tucker decomposition and its iterative solution, Int. J. Mach. Learn. Cybern., 13 (2022), 509–522. https://doi.org/10.1007/s13042-021-01422-5 doi: 10.1007/s13042-021-01422-5
[19]	J. Zhang, Y. Han, J. Jiang, Semi-supervised tensor learning for image classification, Multimedia Syst., 23 (2017), 63–73. https://doi.org/10.1007/s00530-014-0416-7 doi: 10.1007/s00530-014-0416-7
[20]	X. Zhang, M. Ng, Sparse nonnegative Tucker decomposition and completion under noisy observations, arXiv, 2022. https://doi.org/10.48550/arXiv.2208.08287
[21]	Q. Liu, L. Lu, Z. Chen, Nonnegative Tucker decomposition with graph regularization and smooth constraint for clustering, Pattern Recogn., 148 (2023), 110207. https://doi.org/10.1016/j.patcog.2023.110207 doi: 10.1016/j.patcog.2023.110207
[22]	Y. Qiu, G. Zhou, Y. Zhang, S. Xie, Graph regularized nonnegative Tucker decomposition for tensor data representation, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing, 2019, 8613–8617. https://doi.org/10.1109/ICASSP.2019.8683766 doi: 10.1109/ICASSP.2019.8683766
[23]	Y. Qiu, G. Zhou, Y. Wang, Y. Zhang, S. Xie, A generalized graph regularized non-negative Tucker decomposition framework for tensor data representation, IEEE Trans. Cybern., 52 (2020), 594–607. https://doi.org/10.1109/TCYB.2020.2979344 doi: 10.1109/TCYB.2020.2979344
[24]	D. Chen, G. Zhou, Y. Qiu, Y. Yu, Adaptive graph regularized non-negative Tucker decomposition for multiway dimensionality reduction, Multimedia Tools Appl., 83 (2024), 9647–9668. https://doi.org/10.1007/s11042-023-15622-4 doi: 10.1007/s11042-023-15622-4
[25]	X. Li, M. Ng, G. Cong, Y. Ye, Q. Wu, MR-NTD: manifold regularization nonnegative tucker decomposition for tensor data dimension reduction and representation, IEEE Trans. Neural Networks Lear. Syst., 28 (2016), 1787–1800. https://doi.org/10.1109/TNNLS.2016.2545400 doi: 10.1109/TNNLS.2016.2545400
[26]	Z. Huang, G. Zhou, Y. Qiu, Y. Yun, Y. Dai, A dynamic hypergraph regularized non-negative Tucker decomposition framework for multiway data analysis, Int. J. Mach. Learn. Cybern., 13 (2022), 3691–3710. https://doi.org/10.1007/s13042-022-01620-9 doi: 10.1007/s13042-022-01620-9
[27]	W. Jing, L. Lu, Q. Liu, Graph regularized discriminative nonnegative Tucker decomposition for tensor data representation, Appl. Intell., 53 (2023), 23864–23882. https://doi.org/10.1007/s10489-023-04738-7 doi: 10.1007/s10489-023-04738-7
[28]	Y. Qiu, G. Zhou, X. Chen, D. Zhang, X. Zhao, Q. Zhao, Semi-supervised non-negative Tucker decomposition for tensor data representation, Sci. China Technol. Sci., 64 (2021), 1881–1892. https://doi.org/10.1007/s11431-020-1824-4 doi: 10.1007/s11431-020-1824-4
[29]	L. Ren, R. Hu, Y. Liu, D. Li, J. Wu, Y. Zang, et al., Improving fraud detection via imbalanced graph structure learning, Mach. Learn., 113 (2023), 1069–1090. https://doi.org/10.1007/s10994-023-06464-0 doi: 10.1007/s10994-023-06464-0
[30]	M. Zhao, W. Li, L. Li, P. Ma, Z. Cai, R. Tao, Three-order tensor creation and Tucker decomposition for infrared small-target detection, IEEE Trans. Geosci. Remote Sens., 60 (2021), 1–16. https://doi.org/10.1109/TGRS.2021.3057696 doi: 10.1109/TGRS.2021.3057696
[31]	T. Jiang, M. K. Ng, J. Pan, G. Song, Nonnegative low rank tensor approximations with multidimensional image applications, Numer. Math., 153 (2023), 141–170. https://doi.org/10.1007/s00211-022-01328-6 doi: 10.1007/s00211-022-01328-6
[32]	C. Ding, X. He, H. Simon, On the equivalence of non-negative matrix factorization and spectral clustering, Proceedings of the 2005 SIAM International Conference on Data Mining, 2005,606–610. https://doi.org/10.1137/1.9781611972757.70 doi: 10.1137/1.9781611972757.70
[33]	J. Pan, M. Ng, Y. Liu, X. Zhang, H. Yan, Orthogonal nonnegative Tucker decomposition, SIAM J. Sci. Comput., 43 (2021), B55–B81. https://doi.org/10.1137/19M1294708 doi: 10.1137/19M1294708
[34]	B. Li, G. Zhou, A. Cichocki, Two efficient algorithms for approximately orthogonal nonnegative matrix factorization, IEEE Signal Process. Lett., 22 (2015), 843–846. https://doi.org/10.1109/LSP.2014.2371895 doi: 10.1109/LSP.2014.2371895
[35]	Y. Qiu, W. Sun, Y. Zhang, X. Gu, G. Zhou, Approximately orthogonal nonnegative Tucker decomposition for flexible multiway clustering, Sci. China Technol. Sci., 64 (2021), 1872–1880. https://doi.org/10.1007/s11431-020-1827-0 doi: 10.1007/s11431-020-1827-0
[36]	D. Cai, X. He, J. Han, T. Huang, Graph regularized nonnegative matrix factorization for data representation, IEEE Trans. Pattern Anal. Mach. Intell., 33 (2010), 1548–1560. https://doi.org/10.1109/TPAMI.2010.231 doi: 10.1109/TPAMI.2010.231
[37]	F. Shang, L. Jiao, J. Shi, F. Wang, M. Gong, Fast affinity propagation clustering: a multilevel approach, Pattern Recogn., 45 (2012), 474–486. https://doi.org/10.1016/j.patcog.2011.04.032 doi: 10.1016/j.patcog.2011.04.032
[38]	F. Shang, L. Jiao, F. Wang, Graph dual regularization non-negative matrix factorization for co-clustering, Pattern Recogn., 45 (2012), 2237–2250. https://doi.org/10.1016/j.patcog.2011.12.015 doi: 10.1016/j.patcog.2011.12.015

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)