As a popular data representation technique, Nonnegative matrix factorization (NMF) has been widely applied in edge computing, information retrieval and pattern recognition. Although it can learn parts-based data representations, existing NMF-based algorithms fail to integrate local and global structures of data to steer matrix factorization. Meanwhile, semi-supervised ones ignore the important role of instances from different classes in learning the representation. To solve such an issue, we propose a novel semi-supervised NMF approach via joint graph regularization and constraint propagation for edge computing, called robust constrained nonnegative matrix factorization (RCNMF), which learns robust discriminative representations by leveraging the power of both L2, 1-norm NMF and constraint propagation. Specifically, RCNMF explicitly exploits global and local structures of data to make latent representations of instances involved by the same class closer and those of instances involved by different classes farther. Furthermore, RCNMF introduces the L2, 1-norm cost function for addressing the problems of noise and outliers. Moreover, L2, 1-norm constraints on the factorial matrix are used to ensure the new representation sparse in rows. Finally, we exploit an optimization algorithm to solve the proposed framework. The convergence of such an optimization algorithm has been proven theoretically and empirically. Empirical experiments show that the proposed RCNMF is superior to other state-of-the-art algorithms.
Citation: Qing Yang, Jun Chen, Najla Al-Nabhan. Data representation using robust nonnegative matrix factorization for edge computing[J]. Mathematical Biosciences and Engineering, 2022, 19(2): 2147-2178. doi: 10.3934/mbe.2022100
As a popular data representation technique, Nonnegative matrix factorization (NMF) has been widely applied in edge computing, information retrieval and pattern recognition. Although it can learn parts-based data representations, existing NMF-based algorithms fail to integrate local and global structures of data to steer matrix factorization. Meanwhile, semi-supervised ones ignore the important role of instances from different classes in learning the representation. To solve such an issue, we propose a novel semi-supervised NMF approach via joint graph regularization and constraint propagation for edge computing, called robust constrained nonnegative matrix factorization (RCNMF), which learns robust discriminative representations by leveraging the power of both L2, 1-norm NMF and constraint propagation. Specifically, RCNMF explicitly exploits global and local structures of data to make latent representations of instances involved by the same class closer and those of instances involved by different classes farther. Furthermore, RCNMF introduces the L2, 1-norm cost function for addressing the problems of noise and outliers. Moreover, L2, 1-norm constraints on the factorial matrix are used to ensure the new representation sparse in rows. Finally, we exploit an optimization algorithm to solve the proposed framework. The convergence of such an optimization algorithm has been proven theoretically and empirically. Empirical experiments show that the proposed RCNMF is superior to other state-of-the-art algorithms.
| [1] | S. Liu, L. Liu, J. Tang, B. Yu, Y. Wang, W. Shi, Edge computing for autonomous driving: Opportunities and challenges, in Proceedings of the IEEE, 107 (2019), 1697–1716. doi: 10.1109/JPROC.2019.2915983. |
| [2] |
M. Wang, X. Hua, J. Tang, R. Hong, Beyond distance measurement: constructing neighborhood similarity for video annotation, IEEE Trans. Multimedia, 11 (2009), 465–476. doi: 10.1109/TMM.2009.2012919. doi: 10.1109/TMM.2009.2012919
|
| [3] |
Y. Song, W. Cai, H. Huang, D. Feng, Y. Wang, M. Chen, Bioimage classification with subcategory discriminant transform of high dimensional visual descriptors, BMC Bioinf., 17 (2016), 465. doi: 10.1186/s12859-016-1318-9. doi: 10.1186/s12859-016-1318-9
|
| [4] |
Z. Xing, Y. Ma, X. Yang, F. Nie, Graph regularized nonnegative matrix factorization with label discrimination for data clustering, Neurocomputing, 440 (2021), 297–309. doi: 10.1016/j.neucom.2021.01.064. doi: 10.1016/j.neucom.2021.01.064
|
| [5] |
H. Xiong, D. Kong, Elastic nonnegative matrix factorization, Pattern Recognit., 90 (2019), 464–475. doi: 10.1016/j.patcog.2018.07.007. doi: 10.1016/j.patcog.2018.07.007
|
| [6] | F. Nie, H. Huang, X. Cai, C. Ding, Efficient and robust feature selection via Joint ℓ2, 1-norms minimization, in Proceedings of the 23rd International Conference on Neural Information Processing Systems (NIPS), 2 (2010), 1813–1821. doi: 10.5555/2997046.2997098. |
| [7] |
R. Chatpatanasiri, B. Kijsirikul, A unified semi-supervised dimensionality reduction framework for manifold learning, Neurocomputing, 73 (2010), 1631–1640. doi: 10.1016/j.neucom.2009.10.024. doi: 10.1016/j.neucom.2009.10.024
|
| [8] |
Z. Li, J. Tang, X. He, Robust structured nonnegative matrix factorization for image representation, IEEE Trans. Neural Networks Learn. Syst., 29 (2018), 1947–1960. doi: 10.1109/TNNLS.2017.2691725. doi: 10.1109/TNNLS.2017.2691725
|
| [9] |
W. Yu, R. Wang, F. Nie, F. Wang, Q. Yu, X. Yang, An improved locality preserving projection with l1-norm minimization for dimensionality reduction, Neurocomputing, 316 (2018), 322–331. doi: 10.1016/j.neucom.2018.08.008. doi: 10.1016/j.neucom.2018.08.008
|
| [10] |
P. N. Belhumeur, J. P. Hepanha, D. J. Kriegman, Eigenfaces vs. fisherfaces: recognition using class specific linear projection, IEEE Trans. Pattern Anal. Mach. Intell., 19 (1997), 711–720. doi: 10.1109/34.598228. doi: 10.1109/34.598228
|
| [11] |
S. Yan, D. Xu, B. Zhang, H. Zhang, Q. Yang, S. Lin, Graph embedding and extensions: A general framework for dimensionality reduction, IEEE Trans. Pattern Anal. Mach. Intell., 29 (2007), 40–51. doi: 10.1109/TPAMI.2007.250598. doi: 10.1109/TPAMI.2007.250598
|
| [12] |
S. Roweis, L. K. Saul, Nonlinear dimensionality reduction by locally linear embedding, Science, 290 (2000), 2323–2326. doi: 10.1126/science.290.5500.2323. doi: 10.1126/science.290.5500.2323
|
| [13] |
J. B. Tenenbaum, V. Silva, J. C. Langford, A global geometric framework for nonlinear dimensionality reduction, Science, 290 (2000), 2319–2323. doi: 10.1126/science.290.5500.2319. doi: 10.1126/science.290.5500.2319
|
| [14] |
A. M. Martinez, A. C. Kak, PCA versus LDA, IEEE Trans. Pattern Anal. Mach. Intell., 23 (2001), 228–233. doi: 10.1109/34.908974. doi: 10.1109/34.908974
|
| [15] |
F. Nie, D. Xu, I. W. Tsang, C. Zhang, Flexible manifold embedding: A framework for semi-supervised and unsupervised dimension reduction, IEEE Trans. Image Process., 19 (2010), 1921–1932. doi: 10.1109/TIP.2010.2044958. doi: 10.1109/TIP.2010.2044958
|
| [16] | D. Zhang, Z. Zhou, S. Chen, Semi-supervised dimensionality reduction, in Peoceedings of the 2007 SIAM International Conference on Data Mining (SDM), (2007), 629–634. doi: 10.1137/1.9781611972771.73. |
| [17] | C. Boutsidis, P. Drineas, M. W Mahoney, P. Drineas, Unsupervised feature selection for the k-means clustering problem, in Proceedingds of the 22nd International Conference on Neural Information Processing Systems, (2009), 153–161. doi: 10.5555/2984093.2984111. |
| [18] | D. Cai, X. He, J. Han, Semi-supervised discriminant analysis, in 2007 IEEE 11th International Conference on Computer Vision (ICCV), (2007), 1–7. doi: 10.1109/ICCV.2007.4408856. |
| [19] |
J. Ye, R. Janardan, C. Park, H. Park, An optimization criterion for generalized discriminant analysis on under sampled problems, IEEE Trans. Pattern Anal. Mach. Intell., 26 (2004), 982–994. doi: 10.1109/TPAMI.2004.37. doi: 10.1109/TPAMI.2004.37
|
| [20] | X. Wang, Y. Liu, F. Nie, H. Huang, Discriminative unsupervised dimensionality reduction, in Proceedings of the 24th International Conference on Artificial Intelligence, (2015), 3925–3931. doi: 10.5555/2832747.2832796. |
| [21] | X. He, P. Niyogi, Locality preserving projections, in Proceedings of the 16th International Conference on Neural Information Processing Systems, (2003), 153–160. doi: 10.5555/2981345.2981365. |
| [22] | M. Belkin, P. Niyogi, Laplacian eigenmaps and spectral techniques for embedding and clustering, in Proceedings of the 14th International Conference on Neural Information Processing System, (2001), 585–591. doi: 10.5555/2980539.2980616. |
| [23] |
D. Wang, X. Gao, X. Wang, Semi-supervised nonnegative matrix factorization via constraint propagation, IEEE Trans. Cybern., 46 (2016), 233–244. doi: 10.1109/TCYB.2015.2399533. doi: 10.1109/TCYB.2015.2399533
|
| [24] |
D. D. Lee, H. S. Seung, Learning the parts of objects by non-negative matrix factorization, Nature, 401 (1999), 788–791. doi: 10.1038/44565. doi: 10.1038/44565
|
| [25] |
Y. X. Wang, Y. J. Zhang, Nonnegative matrix factorization: a comprehensive review, IEEE Trans. Knowl. Data Eng., 25 (2013), 1336–1353. doi: 10.1109/TKDE.2012.51. doi: 10.1109/TKDE.2012.51
|
| [26] | S. Li, X. Hou, H. Zhang, Q. Cheng, Learning spatially localized, parts-based representation, in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (2001), 1. doi: 10.1109/CVPR.2001.990477. |
| [27] | S. D. Kamvar, D. Klein, C. D. Manning, Spectral learning, in Proceedings of the 18th International Joint Conference on Artificial Intelligence, (2003), 561–566. doi: 10.5555/1630659.1630742. |
| [28] |
Q. Huang, X. Yin, S. Chen, Y. Wang, B. Chen, Robust nonnegative matrix factorization with structure regularization, Neurocomputing, 412 (2020), 72–90. doi: 10.1016/j.neucom.2020.06.049. doi: 10.1016/j.neucom.2020.06.049
|
| [29] | S. Y. Li, Y. Jiang, Z. H. Zhou, Partial multi-view clustering, in Proceedings of the 28th AAAI Conference on Artificial Intelligence, (2014), 1968–1974. doi: 10.5555/2892753.2892826. |
| [30] |
Y. Yi, J. Wang, W. Zhou, C. Zheng, J. Kong, S. Qiao, Non-negative matrix factorization with locality constrained adaptive graph, IEEE Trans. Circuits Syst. Video Techn., 30 (2020), 427–441. doi: 10.1109/TCSVT.2019.2892971. doi: 10.1109/TCSVT.2019.2892971
|
| [31] |
H. Liu, Z. Wu, X. Li, D. Cai, T. S. Huang, Constrained nonnegative matrix factorization for image representation, IEEE Trans. Pattern Anal. Mach. Intell., 34 (2012), 1299–1311. doi: 10.1109/TPAMI.2011.217. doi: 10.1109/TPAMI.2011.217
|
| [32] | J. P. Brunet, P. Tamayo, T. R. Golub, J. P. Mesirov, Metagenes and molecular pattern discovery using matrix factorization, in Proceedings of the National Academy of Sciences, 101 (2004), 4164–4169. doi: 10.1073/pnas.0308531101. |
| [33] |
C. Peng, Y. Chen, Z. Kang, C. Chen, Q. Cheng, Robust principal component analysis: A factorization-based approach with linear complexity, Inf. Sci., 513 (2020), 581–599. doi: 10.1016/j.ins.2019.09.074. doi: 10.1016/j.ins.2019.09.074
|
| [34] |
C. Ding, T. Li, M. I. Jordan, Convex and semi-nonnegative matrix factorizations, IEEE Trans. Pattern Anal. Mach. Intell., 32 (2010), 45–55. doi: 10.1109/TPAMI.2008.277. doi: 10.1109/TPAMI.2008.277
|
| [35] |
D. Cai, X. He, J. Han, T. S. Huang, Graph regularized nonnegative matrix factorization for data representation, IEEE Trans. Pattern Anal. Mach. Intell., 33 (2011), 1548–1560. doi: 10.1109/TPAMI.2010.231. doi: 10.1109/TPAMI.2010.231
|
| [36] |
Z. Li, J. Liu, H. Lu, Structure preserving non-negative matrix factorization for dimensionality reduction, Comput. Vision Image Understanding, 117 (2013), 1175–1189. doi: 10.1016/j.cviu.2013.04.003. doi: 10.1016/j.cviu.2013.04.003
|
| [37] |
Z. Zhang, K. Zhao, Low rank matrix approximation with manifold regularization, IEEE Trans. Pattern Anal. Mach. Intell., 35 (2013), 1717–1729. doi: 10.1109/TPAMI.2012.274. doi: 10.1109/TPAMI.2012.274
|
| [38] |
N. Lu, H. Miao, Structure constrained nonnegative matrix factorization for pattern clustering and classification, Neurocomputing, 171 (2016), 400–411. doi: 10.1016/j.neucom.2015.06.049. doi: 10.1016/j.neucom.2015.06.049
|
| [39] |
A. Cichocki, R. Zdunek, S. Amari, Csiszár's divergences for non-negative matrix factorization: Family of new algorithms, ICA Independent Component Analysis and Blind Signal Separation, (2006), 32–39. doi: 10.1007/11679363_5. doi: 10.1007/11679363_5
|
| [40] |
A. Cichocki, R. Zdunek, A. Phan, S. Amari, Nonnegative matrix and tensor factorizations: applications to exploratory multi-way data analysis and blind source separation, John Wiley & Sons, Ltd., (2009). doi:10.1002/9780470747278. doi: 10.1002/9780470747278
|
| [41] |
C. Fevotte, J. Idier, Algorithms for nonnegative matrix factorization with the β-divergence, Neural Comput., 23 (2011), 2421–2456. doi: 10.1162/NECO_a_00168. doi: 10.1162/NECO_a_00168
|
| [42] |
K. Devarajan, V. C. K. Cheung, A quasi-likelihood approach to nonnegative matrix factorization, Neural Comput., 28 (2016), 1663–1693. doi: 10.1162/NECO_a_00853. doi: 10.1162/NECO_a_00853
|
| [43] |
K. Devarajan, A statistical framework for non-negative matrix factorization based on generalized dual divergence, Neural Networks, 140 (2021), 309–324. doi: 10.1016/j.neunet.2021.03.020. doi: 10.1016/j.neunet.2021.03.020
|
| [44] |
Y. Chen, M. Rege, M. Dong, J. Hua, Non-negative matrix factorization for semi-supervised data clustering, Knowl. Inf. Syst., 17 (2008), 355–379. doi: 10.1007/s10115-008-0134-6. doi: 10.1007/s10115-008-0134-6
|
| [45] | N. Guan, X. Huang, L. Lan, Z. Luo, X. Zhang, Graph based semi-supervised non-negative matrix factorization for document clustering, in 2012 11th International Conference on Machine Learning and Applications (ICMLA), (2012), 404–408. doi: 10.1109/ICMLA.2012.73. |
| [46] |
H. Lee, J. Yoo, S. Choi, Semi-supervised nonnegative matrix factorization, IEEE Signal Process. Lett., 17 (2010), 4–7. doi: 10.1109/LSP.2009.2027163. doi: 10.1109/LSP.2009.2027163
|
| [47] |
X. Zhang, L. Zong, X. Liu, J. Luo, Constrained clustering with nonnegative matrix factorization, IEEE Trans. Neural Networks Learn. Syst., 27 (2016), 1514–1526. doi: 10.1109/TNNLS.2015.2448653. doi: 10.1109/TNNLS.2015.2448653
|
| [48] |
Z. Yang, Y. Xiang, K. Xie, Y. Lai, Adaptive method for nonsmooth nonnegative matrix factorization, IEEE Trans. Neural Networks Learn. Syst., 28 (2017), 948–960. doi: 10.1109/TNNLS.2016.2517096. doi: 10.1109/TNNLS.2016.2517096
|
| [49] |
Y. Yi, Y. Shi, H. Zhang, J. Wang, Jun Kong, Label propagation based semi-supervised non-negative matrix factorization for feature extraction, Neurocomputing, 149 (2015), 1021–1037. doi: 10.1016/j.neucom.2014.07.031. doi: 10.1016/j.neucom.2014.07.031
|
| [50] |
Y. Yi, Y. Chen, J. Wang, G. Lei, J. Dai, H. Zhang, Joint feature representation and classification via adaptive graph semi-supervised nonnegative matrix factorization, Signal Process.: Image Commun., 89 (2020), 115984. doi: 10.1016/j.image.2020.115984. doi: 10.1016/j.image.2020.115984
|
| [51] |
S. Li, Q. Liu, J. Dai, W. Wang, X. Gui, Y. Yi, Adaptive-weighted multiview deep basis matrix factorization for multimedia data analysis, Wireless Commun. Mobile Comput., 9 (2021), 1–12. doi: 10.1155/2021/5526479. doi: 10.1155/2021/5526479
|
| [52] |
Y. Jia, S. Kwong, J. Hou, W. Wu, Semi-supervised non-negative matrix factorization with dissimilarity and similarity regularization, IEEE Trans. Neural Networks Learn. Syst., 31 (2019), 2510–2521. doi: 10.1109/TNNLS.2019.2933223. doi: 10.1109/TNNLS.2019.2933223
|
| [53] |
Z. Xing, M. Wen, J. Peng, J. Feng, Discriminative semi-supervised non-negative matrix factorization for data clustering, Eng. Appl. Artif. Intell., 103 (2021), 104289. doi: 10.1016/j.engappai.2021.104289. doi: 10.1016/j.engappai.2021.104289
|
| [54] | D. Zhou, O. Bousquet, T. N. Lal, J. Weston, B. S. Cholkopf, Learning with local and global consistency, in Proceedings of the 16th International Conference on Neural Information Processing Systems, (2003), 321–328. doi: 10.5555/2981345.2981386. |
| [55] |
Z. Li, J. Liu, J. Tang, H. Lu, Robust structured subspace learning for data representation, IEEE Trans. Pattern Anal. Mach. Intell., 37 (2015), 2085–2098. doi: 10.1109/TPAMI.2015.2400461. doi: 10.1109/TPAMI.2015.2400461
|
| [56] |
J. Huang, F. Nie, H. Huang, C. Ding, Robust manifold nonnegative matrix factorization, ACM Trans. Knowl. Discovery Data, 8 (2014), 1–21. doi: 10.1145/2601434. doi: 10.1145/2601434
|
| [57] |
W. Liu, N. Zheng, Q. You, Nonnegative matrix factorization and its applications in pattern recognition, Chin. Sci. Bull., 51 (2006), 7–18. doi: 10.1007/s11434-005-1109-6. doi: 10.1007/s11434-005-1109-6
|
| [58] | D. Kong, C. Ding, H. Huang, Robust nonnegative matrix factorization using l21 norm, in Proceedings of the 20th ACM International Conference on Information and Knowledge Management, (2011), 673–682. doi: 10.1145/2063576.2063676. |
| [59] |
B. Wu, E. Wang, Z. Zhu, W. Chen, P. Xiao, Manifold NMF with L2, 1 norm for clustering, Neurocomputing, 273 (2018), 78–88. doi: 10.1016/j.neucom.2017.08.025. doi: 10.1016/j.neucom.2017.08.025
|
| [60] |
M. Babaee, S. Tsoukalas, M. Babaee, G. Rigoll, M. Datcu, Discriminative nonnegative matrix factorization for dimensionality reduction, Neurocomputing, 173 (2016), 212–223. doi: 10.1016/j.neucom.2014.12.124. doi: 10.1016/j.neucom.2014.12.124
|
| [61] |
Z. Lu, Y. Peng, Exhaustive and efficient constraint propagation: A graph-based learning approach and its applications, Int. J. Comput. Vision, 103 (2013), 306–325. doi: 10.1007/s11263-012-0602-z. doi: 10.1007/s11263-012-0602-z
|
| [62] | C. Ding, D. Zhou, X. He, H. Zha, R1-pca: Rotational invariant l1-norm principal component analysis for robust subspace factorization, in Proceedings of the 23rd International Conference on Machine Learning (ICML), (2006), 281–288. doi: 10.1145/1143844.1143880. |
| [63] |
X. Yin, S. Chen, E. Hu. Regularized soft K-means for discriminant analysis, Neurocomputing, 103 (2013), 29–42. doi: 10.1016/j.neucom.2012.08.021. doi: 10.1016/j.neucom.2012.08.021
|
Figures(9) / Tables(3)
Qing Yang, Jun Chen, Najla Al-Nabhan. Data representation using robust nonnegative matrix factorization for edge computing[J]. Mathematical Biosciences and Engineering, 2022, 19(2): 2147-2178. doi: 10.3934/mbe.2022100
DownLoad: