Semi-supervised graph regularized concept factorization with the class-driven constraint for image representation

Yuelin Gao; Huirong Li; Yani Zhou; Yijun Chen; Yuelin Gao; Huirong Li; Yani Zhou; Yijun Chen

doi:10.3934/math.20231468

AIMS Mathematics

2023, Volume 8, Issue 12: 28690-28709. doi: 10.3934/math.20231468

Previous Article Next Article

Research article Special Issues

Semi-supervised graph regularized concept factorization with the class-driven constraint for image representation

1.
School of Mathematics and Information Science, North Minzu University, Yinchuan, 750021, China
2.
School of Mathematics and Computer Application, Shangluo University, Shangluo 726000, China
3.
Engineering Research Center of Qinling Health Welfare Big Data, Universities of Shaanxi Province, Shangluo 726000, China
4.
School of Health Management, Shangluo University, Shangluo 726000, China
5.
Library, Xian Aeronautical University, Xi'an 710077, China

Received: 21 August 2023 Revised: 27 September 2023 Accepted: 06 October 2023 Published: 23 October 2023
MSC : 15A23

As a popular dimensionality reduction technique, concept factorization (CF) has been widely applied in image clustering. However, CF fails to extract the intrinsic structure of data space and does not utilize the label information. In this paper, a new semi-supervised graph regularized CF (SGCF) method is proposed, which makes full use of the limited label information and the graph regularization to improve the algorithm of clustering performance. Particularly, SGCF associates the class label information of data points with their new representations by using the class-driven constraint, and this constraint forces the new representations of data points to be more similar within the same class while different between classes. Furthermore, SGCF extracts the geometric structure of the data space by incorporating graph regularization. SGCF not only reveals the geometrical structure of the data space, but also takes into the limited label information account. We drive an efficient multiplicative update algorithm for SGCF to solve the optimization, and analyze the proposed SGCF method in terms of the convergence and computational complexity. Clustering experiments show the effectiveness of the SGCF method in comparison to other state-of-the-art methods.
- concept factorization,
- semi-supervised learning,
- class-driven constraint,
- label information
Citation: Yuelin Gao, Huirong Li, Yani Zhou, Yijun Chen. Semi-supervised graph regularized concept factorization with the class-driven constraint for image representation[J]. AIMS Mathematics, 2023, 8(12): 28690-28709. doi: 10.3934/math.20231468

Related Papers:

Abstract

As a popular dimensionality reduction technique, concept factorization (CF) has been widely applied in image clustering. However, CF fails to extract the intrinsic structure of data space and does not utilize the label information. In this paper, a new semi-supervised graph regularized CF (SGCF) method is proposed, which makes full use of the limited label information and the graph regularization to improve the algorithm of clustering performance. Particularly, SGCF associates the class label information of data points with their new representations by using the class-driven constraint, and this constraint forces the new representations of data points to be more similar within the same class while different between classes. Furthermore, SGCF extracts the geometric structure of the data space by incorporating graph regularization. SGCF not only reveals the geometrical structure of the data space, but also takes into the limited label information account. We drive an efficient multiplicative update algorithm for SGCF to solve the optimization, and analyze the proposed SGCF method in terms of the convergence and computational complexity. Clustering experiments show the effectiveness of the SGCF method in comparison to other state-of-the-art methods.

References

[1]	H. P. Kriegel, P. Kroger, A. Zimek, Clustering high-dimensional data: A survey on subspace clustering, pattern-based clustering, and correlation clustering, ACM T. Knowl. Discov. D., 3 (2009), 1–58. http://doi.org/10.1145/1497577.1497578 doi: 10.1145/1497577.1497578
[2]	G. Cui, Y. Li, Nonredundancy regularization based nonnegative matrix factorization with manifold learning for multiview data representation, Inform. Fusion, 82 (2022), 86–98. http://doi.org/10.1016/j.inffus.2021.12.001 doi: 10.1016/j.inffus.2021.12.001
[3]	J. A. Lee, M. Verleysen, Nonlinear dimensionality reduction, New York: Springer, 2007. https://doi.org/10.1007/978-0-387-39351-3
[4]	I. T. Jolliffe, Principal component analysis, New York: Springer, 1986. http://doi.org/10.1007/978-1-4757-1904-8
[5]	D. Kalman, A singularly valuable decomposition: The SVD of a matrix, Coll. Math. J., 27 (1996), 2–23. https://doi.org/10.1080/07468342.1996.11973744 doi: 10.1080/07468342.1996.11973744
[6]	D. D. Lee, H. S. Seung, Learning the parts of objects by non-negative matrix factorization, Nature, 401 (1999), 788–791. https://doi.org/10.1038/44565 doi: 10.1038/44565
[7]	D. Lee, H. S. Seung, Algorithms for non-negative matrix factorization, NIPS'00: Proceedings of the 13th international conference on neural information processing systems, 2000, 535–541.
[8]	G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, Y. Ma, Robust recovery of subspace structures by low-rank representation, IEEE T. Pattern Anal. Mach. Intell., 35 (2013), 171–184. https://doi.org/10.1109/TPAMI.2012.88 doi: 10.1109/TPAMI.2012.88
[9]	J. Liu, Y. Chen, J. Zhang, Z. Xu, Enhancing low-rank subspace clustering by manifold regularization, IEEE T. Image Process., 23 (2014), 4022–4030. https://doi.org/10.1109/TIP.2014.2343458 doi: 10.1109/TIP.2014.2343458
[10]	S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, R Harshman, Indexing by latent semantic analysis, J. Am. Soc. Inform. Sci., 41 (1990), 391–407. https://doi.org/10.1002/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9 doi: 10.1002/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9
[11]	W. Xu, X. Liu, Y. Gong, Document clustering based on non-negative matrix factorization, SIGIR 2003: Proceedings of the 26th annual international ACM SIGIR conference on research and development in information, 2003, 267–273. https://doi.org/10.1145/860435.860485
[12]	F. Shahnaz, M. W. Berry, V. P. Pauca, R. J. Plemmons, Document clustering using nonnegative matrix factorization, Inform. Process. Manag., 42 (2006), 373–386. https://doi.org/10.1016/j.ipm.2004.11.005 doi: 10.1016/j.ipm.2004.11.005
[13]	X. Long, H. Lu, Y. Peng, W. Li, Graph regularized discriminative non-negative matrix factorization for face recognition, Multimed Tools Appl., 72 (2014), 2679–2699. https://doi.org/10.1007/s11042-013-1572-z doi: 10.1007/s11042-013-1572-z
[14]	H. Liu, Z. Wu, X. Li, D. Cai, T. S. Huang, Constrained nonnegative matrix factorization for image representation, IEEE T. Pattern Anal. Mach. Intell., 34 (2012), 1299–1311. https://doi.org/10.1109/TPAMI.2011.217 doi: 10.1109/TPAMI.2011.217
[15]	J. P. Brunet, P. Tamayo, T. R Golub, J. P. Mesirov, Metagenes and molecular pattern discovery using matrix factorization, PNAS, 101 (2004), 4164–4169. https://doi.org/10.1073/pnas.0308531101 doi: 10.1073/pnas.0308531101
[16]	W. Hua, X. He, Discriminative concept factorization for data representation, Neurocomputing, 74 (2011), 3800–3807. https://doi.org/10.1016/j.neucom.2011.07.020 doi: 10.1016/j.neucom.2011.07.020
[17]	X. Peng, D. Chen, D. Xu, Hyperplane-based nonnegative matrix factorization with label information, Inform. Sci., 493 (2019), 1–19. https://doi.org/10.1016/j.ins.2019.04.026 doi: 10.1016/j.ins.2019.04.026
[18]	W. Xu, Y. Gong, Document clustering by concept factorization, Proceedings of the 27th annual international ACM SIGIR conference on research and development in information, 2004, 202–209. https://doi.org/10.1145/1008992.1009029
[19]	D. Cai, X. He, J. Han, Locally consistent concept factorization for document clustering, IEEE T. Knowl. Data Eng., 23 (2011), 902–913. https://doi.org/10.1109/TKDE.2010.165 doi: 10.1109/TKDE.2010.165
[20]	H. Liu, Z. Yang, J. Yang, Z. Wu, X. Li, Local coordinate concept factorization factorization for image representation, IEEE T. Neur. Net. Lear. Syst., 25 (2014), 1071–1082. https://doi.org/10.1109/TNNLS.2013.2286093 doi: 10.1109/TNNLS.2013.2286093
[21]	D. Wei, X. Shen, Q. Sun, X. Gao, Z. Ren, Adaptive graph guided concept factorization on Grassmann manifold, Inform. Sci., 576 (2021), 725–742. https://doi.org/10.1016/j.ins.2021.08.040 doi: 10.1016/j.ins.2021.08.040
[22]	S. Peng, W. Ser, B. Chen, L. Sun, Z. Lin, Correntropy based graph regularized concept factorization for clustering, Neurocomputing, 316 (2018), 34–48. https://doi.org/10.1016/j.neucom.2018.07.049 doi: 10.1016/j.neucom.2018.07.049
[23]	H. Li, J. Zhang, J. Liu, Graph-regularized CF with local coordinate for image representation, J. Vis. Commun. Image Rep., 49 (2017), 392–400. https://doi.org/10.1016/j.jvcir.2017.10.005 doi: 10.1016/j.jvcir.2017.10.005
[24]	Y. He, H. Lu, L. Huang, S. Xie, Pairwise constrained concept factorization for data representation, Neural Networks, 52 (2014), 1–17. https://doi.org/10.1016/j.neunet.2013.12.007 doi: 10.1016/j.neunet.2013.12.007
[25]	L. Xue, S. Xiaobo, S. Zhenqiu, Y. Qiaolin, Z. Chunxia, Graph regularized multilayer concept factorization for data representation, Neurocomputing, 238 (2017), 139–151. https://doi.org/10.1016/j.neucom.2017.01.045 doi: 10.1016/j.neucom.2017.01.045
[26]	H. Cai, B. Liu, Y. Xiao, L. Y. Lin, Semi-supervised multi-view clustering based on constrained nonnegative matrix factorization, Knowl. Based Syst., 182 (2019), 104798. https://doi.org/10.1016/j.knosys.2019.06.006 doi: 10.1016/j.knosys.2019.06.006
[27]	M. Babaee, S. Tsoukalas, M. Babaee, G. Rigoll, M. Datcu, Discriminative nonnegative matrix factorization for dimensionality reduction, Neurocomputing, 173 (2016), 212–223. https://doi.org/10.1016/j.neucom.2014.12.124 doi: 10.1016/j.neucom.2014.12.124
[28]	S. Peng, W. Ser, B. Chen, Z. Lin, Robust semi-supervised nonnegative matrix factorization for image clustering, Pattern Recogn., 111 (2021), 107683. https://doi.org/10.1016/j.patcog.2020.107683 doi: 10.1016/j.patcog.2020.107683
[29]	H.g Li, J. Zhang, G. Shi, J. Liu, Graph-based discriminative nonnegative matrix factorization with label information, Neurocomputing, 266 (2017), 91–100. https://doi.org/10.1016/j.neucom.2017.04.067 doi: 10.1016/j.neucom.2017.04.067
[30]	Y. Yi, Y. Chen, J. Wang, G. Lei, J. Dai, H. Zhang, Joint feature representation and classification via adaptive graph semi-supervised nonnegative matrix factorization, Signal Process.-Image, 89 (2020), 115984. https://doi.org/10.1016/j.image.2020.115984 doi: 10.1016/j.image.2020.115984
[31]	Z. Shu, C. Zhao, P. Huang, Local regularization concept factorization and its semi-supervised extension for image representation, Neurocomputing, 158 (2015), 1–12. https://doi.org/10.1016/j.neucom.2015.02.014 doi: 10.1016/j.neucom.2015.02.014
[32]	Z. Xing, Y. Ma, X. Yang, F. Nie, Graph regularized nonnegative matrix factorization with label discrimination for data clustering, Neurocomputing, 440 (2021), 297–309. https://doi.org/10.1016/j.neucom.2021.01.064 doi: 10.1016/j.neucom.2021.01.064
[33]	H. Liu, G. Yang, Z. Wu, D. Cai, Constrained concept factorization for image represention, IEEE T. Cybernetics, 44 (2014), 1214–1224. https://doi.org/10.1109/TCYB.2013.2287103 doi: 10.1109/TCYB.2013.2287103
[34]	H. Li, Y. Gao, J. Liu, J. Zhang, C. Li, Semi-supervised graph regularized nonnegative matrix factorization with local coordinate for image representation, Signal Process.-Image, 102 (2022), 116589. https://doi.org/10.1016/j.image.2021.116589 doi: 10.1016/j.image.2021.116589
[35]	S. Peng, Z. Yang, F. Nie, B. Chen, Z. Lin, Correntropy based semi-supervised concept factorization with adaptive neighbors for clustering, Neural Networks, 154 (2022), 203–217. https://doi.org/10.1016/j.neunet.2022.07.021 doi: 10.1016/j.neunet.2022.07.021
[36]	W. Yan, B. Zhang, S. Ma, Z. Yang, A novel regularized concept factorization for document clustering, Knowl.-Based Syst., 135 (2017), 147–158. https://doi.org/10.1016/j.knosys.2017.08.010 doi: 10.1016/j.knosys.2017.08.010
[37]	D. Cai, X. He, J. Han, T. S. Huang, Graph regularized nonnegative matrix factorization for data representation, IEEE T. Pattern Anal., 33 (2011), 1548–1560. https://doi.org/10.1109/TPAMI.2010.231 doi: 10.1109/TPAMI.2010.231
[38]	C. Cortes, M. Mohri, On transductive regression, NIPS'06: Proceedings of the 19th international conference on neural information processing systems, 2006, 305–312.
[39]	F. R. K. Chung, Spectral graph theory, American Mathematical Society, 1997.
[40]	Y. H. Xiao, Z. f. Zhu, Y. Zhao, Y. C. Wei, Class-driven non-negative matrix factorization for image representation, J. Comput. Sci. Technol., 28 (2013), 751–761. https://doi.org/10.1007/s11390-013-1374-9 doi: 10.1007/s11390-013-1374-9
[41]	H. Li, J. Zhang, J. Liu, Class-driven concept factorization for image representation, Neurocomputing, 190 (2016), 197–208. https://doi.org/10.1016/j.neucom.2016.01.017 doi: 10.1016/j.neucom.2016.01.017
[42]	A. P. Dempster, N. M. Laird, D. B. Rubin, Maximum likelihood from incomplete data via the $ EM $ algorithm, J. Royal Stat. Soc. Ser. B, 39 (1977), 1–22. https://doi.org/10.1111/j.2517-6161.1977.tb01600.x doi: 10.1111/j.2517-6161.1977.tb01600.x

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)