Semi-supervised graph regularized concept factorization with the class-driven constraint for image representation

1.   Introduction

In real-world applications, the image clustering problem has become an important research hotspot in the areas of machine learning and pattern recognition due to it enabled knowledge discovery. With the rapid development of modern information technology, we are usually faced with high-dimensional data generated in the real world ^[1,2]. In addition, this high-dimensional data not only increases the calculation and storage but also often encounters some problems such as "curse of dimension" and "small sample size", which will bring great difficulties to the subsequent processing of data ^[3]. In general, data representation plays a crucial role in the field of computer vision in order to discover the important underlying structures and useful information in high-dimensional data. Matrix factorization (MF) is one of these data representation techniques, and typical methods include principal component analysis (PCA) ^[4], singular value decomposition (SVD) ^[5], nonnegative matrix factorization (NMF) ^[6,7] and low rank representation (LRR) ^[8,9]. NMF is a popular one of the MF methods, which obtains a parts-based representation of original data space. The great advantage of this parts-based representation have been successfully applied in various areas, such as clustering and classification ^[10,11,12], face and object recognition ^[13,14], gene expression analysis ^[15] and others. However, NMF still has some intrinsic drawbacks ^[16,17]. For example, it is only applicable to nonnegative data matrix and cannot utilize the power of kernelization, and it fails to extract the geometric structure of data space and cannot make use of the label information.

In order to overcome these above shortcomings, many improved NMF algorithms have been successfully proposed in recent years. For example, Xu and Gong proposed a concept factorization (CF) method ^[18]. In CF, each cluster is represented by a linear combination of the data points, and each data point is linearly represented by the cluster centers. Although the CF method can be applied in any transformed space, it is difficult to discover the local geometric structures of the data space. Therefore, several graph regularized CF methods have been proposed to handle this issue, such as locally consistent CF (LCCF)^[19], local coordinate CF (LCF)^[20], adaptive graph guided CF (AGCF) ^[21], and so on. Specifically, LCCF extracts the intrinsic geometrical structures of data space by using a k-nearest neighbor graph, while LCF preserves the sparsity and locality simultaneously by introducing a local coordinate constraint and AGCF captures the local relationships of the data points by using the adaptive graph regularization constraint. However, most algorithms construct graphs directly on the original data, which can lead to sensitivity to noise ^[22]. Many studies have employed adaptive graphs to solve these problems ^[23,24,25], but they mainly focus primarily on the problem of data dimensionality reduction.

Undoubtedly, the improved CF models mentioned above are unsupervised learning methods, and they fail to utilize the limited label information. Using a small amount of label information of the data to improve the performance of the algorithms has become one of the hotspots of machine learning in current studies. Many machine learning experts have shown that a significant improvement can be achieved in learning accuracy when a small amount of the labeled data is used in conjunction with the large amounts of the unlabeled data ^{[26,27,28,29,30,31,32]}. In order to make better use of the label information, several works incorporated the limited label information into CF. For examples, Liu et al. incorporated the label constraints into NMF and CF, respectively, and proposed both constrained nonnegative matrix factorization (CNMF)^[14] and constrained concept factorization (CCF) ^[33]. The purpose of these constraints is that the samples from the same class should be merged together in the new representation space. Babaee et al proposed a discriminative NMF (DNMF) ^[27] by using the label information of a small amount of data as a discriminative constraint, which enforces the samples with the same label information to be aligned on the same axis in the new representation instead of a single point. Li et al. proposed a class-driven CF by utilizing the class-driven constraint, which forces the representations of samples to be more similar within the same class while different between other classes. In addition, some further research of the CF has also been proposed ^{[32,34,35,36]} based on different constraints in the past few years.

In this work, we incorporate the class-driven constraint and the graph regularization into CF, and propose a new semi-supervised graph regularized CF (SGCF) algorithm for the clustering application. Specifically, the main contributions of this work can be outlined as follows:

● In our proposed SGCF, we incorporate the limited label information and the graph regularization into CF as the additional constraints to enhance the performance of the clustering task.

● SGCF is a semi-supervised learning method, which associates the class labels of samples with their representations by using a class-driven constraint, and also extracts the intrinsic geometrical structure of the data space by introducing the graph regularization.

● The multiplicative update algorithm is used to optimize the objective function of SGCF. Meanwhile, the multiplicative update rules and the computational complexity of SGCF are derived, and its convergence is also proved.

● Extensive experiments on four real-word image data sets show that the effectiveness of the proposed SGCF when compared with the state-of-the-art methods.

The rest of this paper is organized as follows. Section 2 briefly reviews some methods that are closely related with our research, including NMF, CF, GNMF and the class-driven constraint. We then introduce our proposed SGCF model and its optimization method in Section 3, and its convergence proof is provided in Section 4. Next, we conduct extensive experiments in Section 5. Finally, we conclude the paper in Section 6.

2.   Related work

In this section, we briefly introduce the basic concepts of NMF, CF, GNMF and the class-driven constraint, which are closely related with our proposed SGCF method.

2.1.   NMF and CF

Given a data matrix $\textbf{X} = [x_{1}, x_{2}, \cdots, x_{n}]\in \textbf{R}_{+}^{m\times n}$, where $x_{i}\in \textbf{R}_{+}^m (1 \leq i\leq n)$ represents a nonnegative data point. NMF aims to decompose $\textbf{X}$ into two matrix factors, i.e., the basis matrix $\textbf{U}\in \textbf{R}_{+}^{m\times k}$ and the coefficient matrix $\textbf{V}\in \textbf{R}_{+}^{n\times k}$, which minimize the following optimization problem ^[6,7]:

where $\|\cdot\|_{F}$ denotes the Frobenius norm of the matrix. This formulation can be viewed column by column as follows:

where $x_{j}$ can be approximated by a linear combination of $\textbf{U}$ with the weight matrix $\textbf{V}$.

However, NMF cannot utilize the power of kernelization. Therefore, Xu and Gong proposed CF, which can be used in any data representation space ^[18]. The CF is an extension of NMF in that each basis vector $u_{j}$ is represented by a linear combination of the samples $x_{i}$:

where $w_{ij}\geq 0$. Therefore, let $\mathbf{W} = [w_{ij}]\in R^{n\times k}$, and CF can be expressed as follows:

In this way, CF can easily be performed in any data representation space, such as Hilbert space. Some detailed review can be found in ^{[18,19,20,35]}.

2.2.   Graph-regularized nonnegative matrix factorization (GNMF)

NMF cannot detect the intrinsic structure of the original data space, which is essential to the image representation. For example, Cai et al. proposed the graph-regularized NMF (GNMF) method, which removes this restriction by adding a regularizer based on the manifold assumption that if two samples $x_{i}, x_{j}$ are similar, then their weighted coefficient $v_{i}, v_{j}$ are similar ^[37]. The weight $s_{ij}$ is used to measure the similarity of two samples $x_{i}, x_{j}$. There are many choices to construct the weight matrix $\textbf{S}$ on the graph, such as dot-product weighting, heat kernel weighting, 0-1 weighting and so on ^[38]. Then, $Tr(V^{T}LV)$ can be used to preserve the local manifold structure of the data space, so the objective function of GNMF is given as follows:

where $\alpha > 0$, $Tr(\cdot)$ denotes the trace of a matrix. $\mathbf{L = H-S}$ is the graph Laplacian matrix ^[39], $\textbf{S}$ is the weight matrix and $\textbf{H}$ is diagonal matrix with the $i$th diagonal element $\textbf{H}_{ii} = \sum_{j}\textbf{S}_{ij}$. In general, the matrix $\mathbf{S}$ is introduced as follows:

where $N_{p}(x_{i})$ represents $p$-nearest neighbors set of $x_{i}$.

2.3.   Class-driven constraint

Consider a dataset consisting of $n$ samples $\textbf{X} = [x_{1}, x_{2}, \cdots, x_{l}, x_{l+1}, \cdots, x_{n}]$, among which the first $l$ samples $x_{1}, x_{2}, \cdots, x_{l}$ are labeled with one class form $c$ categories and the rest of $n-l$ samples $x_{l+1}, \cdots, x_{n}$ are unlabeled samples. In order to learn a discriminative basis set $\mathbf{U = [U_{1}, U_{2}, \cdots, U_{k}]}\in \mathbf{R}_{+}^{m\times k}$, where $\mathbf{U}_{i}$ is able to sparsely represent the $i-$th class well but not others, $r$ denotes the number of basis vectors of each subset and $k = r \times c$.

Suppose $\mathbf{V} = [v_{1}, v_{2}, \cdots, v_{n}]^{T}\in \mathbf{R}^{n\times k}$ denotes the representation matrix in Eq (2.3) and $\mathbf{D} = [d_{b_{1}}, d_{b_{2}}, \cdots, d_{b_{n}}]^{T}\in \mathbf{R}^{n\times k}$ denotes the indicator matrix of the inhomogeneous representation. Our target is to make use of class label information to learn discriminative basis vectors, which represent better for their own classes while worse for the others. Therefore, we expect that approximated part-based representation $v_{j}$ for sample $x_{j}$ with label $b_{j}$ will satisfy the following conditions ^[40]:

where $d_{b_{j}}$ chooses the inhomogeneous representation coefficients of $v_{j}$, i.e., coefficients corresponding to basis vectors other than $\mathbf{U}_{b_{j}}$. There are $n$ samples, suppose the discriminative basis set $\mathbf{U = [U_{1}, U_{2}, U_{3}]}$, $\textbf{U}_{i}\in \mathbf{R}^{m\times 2}(i.e., k = 6)$, where $x_{1}$ belongs to the first class, $x_{2}$ belongs to the second class, $x_{3}$ belongs to the third class and the other $n-3$ are unlabeled samples. Thus, the indicator matrix $\mathbf{D^{T}} = [d_{b_{1}}, d_{b_{2}}, d_{b_{3}}, \cdots, d_{b_{n}}]$ can be defined as

where if $x_{j}$ (such as $x_{4}$) is the unlabeled sample, we set all the elements in $d_{b_{j}}$ (such as $d_{b_{4}}$) to zero. We refer to Eq (2.6) as the class-driven constraint.

Li et al. incorporated the class-driven constraint Eq (2.6) into CF and proposed a class-driven CF (CDCF) as follows ^[41]:

The objective function is solved by the iterative updating method, and the updating rules of $\textbf{W}$ and $\textbf{V}$ are:

3.   SGCF

3.1.   The objective function

CDCF is a semi-supervised learning method, but it fails to extract the geometrical structure of data space ^[41]. GNMF adds the manifold regularization to NMF ^[37]. LCF considers the locality constraints in revealing the underlying concepts, but it ignores the manifold structure in the data space ^[20]. Therefore, in order to reveal the latent concepts consistent with manifold geometrical structure, we will incorporate this manifold regularization $Tr(V^{T}LV)$ to CDCF and obtain the proposed SGCF approach formulated as follows:

where $\alpha, \beta$ are nonnegative balance factors. The key novelty of SGCF is that SGCF not only captures the intrinsic geometrical structure of data space by using graph regularization but also considers the available label information by incorporating the class-driven constraint. Then, the discriminative ability of SGCF is greatly enhanced in clustering tasks.

3.2.   Optimization

Undoubtedly, the objective function of the proposed SGCF method in Eq (3.1) is nonconvex in all matrix variables $\mathbf{W}$ and $\mathbf{V}$ together, while it is convex with respect to the single matrix variables $\mathbf{W}$ or $\mathbf{V}$. In this situation, it is very hard to find the global optimal solution in Eq (3.1). However, a local minimum can be obtained for the objective function of SGCF by using the multiplicative update rules ^[28,37].

We need to minimize the objective function $O$ defined in Eq (3.1) to get its local minima, and the objective function of SGCF can be rewritten as:

where $\mathbf{K = X^{T}X}$. Specifically, assuming that $\mathbf{\Phi} = [\phi_{ik}]$ and $\mathbf{\Psi} = [\psi_{jk}]$ are two Lagrangian multipliers, then the corresponding Lagrangian function $L$ can be presented below:

The partial derivatives of $L$ with respect to $\mathbf{W}$ and $\mathbf{V}$, respectively, are:

Letting the derivatives be zero and using the Karush-Kuhn-Tucker (KKT) condition $\phi_{ik} w_{ik} = 0$ and $\psi_{jk}v_{jk} = 0$, we have:

Therefore, we have the following updating rules for $\mathbf{W}$ and $\mathbf{V}$:

From Eq (3.1), we can find that if the indicator matrix $\textbf{D}$ is a $n\times k$ null matrix or $\beta = 0$, SGCF reduces to LCCF ^[19], if $\alpha = 0$, SGCF reduces to CDCF ^[41] and if $\alpha = \beta = 0$, SGCF reduces to CF ^[18]. The procedure of SGCF is summarized in Algorithm 1.

The solution of the optimization problem Eq (3.1) is not unique. It is easy to verify that if $\mathbf{W}$ and $\mathbf{V}$ are a solution of the optimization problem Eq (3.1), then $\mathbf{WS}$ and $\mathbf{VS^{-1}}$ will also be a solution of the optimization problem Eq (3.1) for any positive diagonal matrix $\mathbf{S}$. Therefore, to make the solution unique, we can further require that $w^{T}\textbf{K}w = 1$, where $w$ is the column vector of $\mathbf{W}$ and $\mathbf{WV^{T}}$ does not change ^[19,33,41]. So, $\mathbf{W}$ and $\mathbf{V}$ should be updated as following:

3.3.   Computational complexity analysis

In this section, we discuss the computational cost of SGCF compared to standard CF and LCCF. Without loss of generality, the big $O$ notation is used to represent the complexity of algorithms. The number of three arithmetic operations including addition, multiplication and division are counted for each update step of NMF, CF, LCCF and proposed SGCF and the results are shown in Table 1 ($N$: the number of data points; $M$: the number of features; $K$: the number of factors; $p$: the number of nearest neighbors, $p \ll N$).

Due to the class-driven constraint, our SGCF method needs $NK$ more operation for addition and multiplication compared with LCCF in each iteration. For the graph Laplacian term, our SGCF and LCCF methods need to compute the weight matrix $S$. Because of $S$ is a sparse matrix, we only need $N(p+3)K$ more addition and multiplication comparing to CF in each iteration. Suppose the multiplicative updates stop after $t$ iterations, the overall computational complexity for NMF, CF, LCCF and SGCF will be $O(tMNK)$, $O(tN^{2}K+N^{2}M)$, $O(tN^{2}K+N^{2}M) +N^{2}p$ and $O(tN^{2}K+N^{2}M) +N^{2}p +NK$. Since $p$ is usually very small (around five) ^[19] and $K \ll min\{M, N \}$, SGCF, LCCF and CF will have the same computational complexity by using the big $O$ notation when dealing with the high-dimensional data.

4.   Convergence proof

We have the following theorem regarding the above updated rules, which guarantees the SGCF method converges to a local minimum.

Theorem 4.1. The objective function $O$ in Eq (3.1) is non-increasing under the updating rules in Eqs (3.4) and (3.5). The objective function is invariant under these updating rules if and only if $\mathbf{W}$ and $\mathbf{V}$ are at a stationary point.

In order to prove Theorem 4.1, we require to prove that the objective function $O$ of SGCF is non-increasing under the updated rules in Eqs (3.4) and (3.5). Due to the fact that the graph regularization and the class-driven constraint in the objective function $O$ of SGCF are only related to the coefficient matrix $\mathbf{V}$, and the iterative updated rule for Eq (3.4) is exactly the same as the updated rule for $\mathbf{W}$ in the CDCF ^[41] and CF ^[18], we omit the convergence proof of SGCF to show that the objective function $O$ of the optimization problem Eq (3.1) is non-increasing under the updated rule in Eq (3.4) ^[18,41]. Therefore, we only have to prove that the objective function $O$ in Eq (3.1) is non-increasing under the updated rule in Eq (3.5). Similar to the literature ^[28,37], we utilize a similar auxiliary function that has been used in the expectation maximization algorithm ^[42]. We begin by introducing a definition and some lemmas.

Definition 4.1. $G(x, x^{'})$ is the auxiliary function of $F(x)$ if the following conditions:

are satisfied.

Lemma 4.1. If $G(x, x^{'})$ is the auxiliary function of $F(x)$, then $F(x)$ is non-increasing under the update

Proof. $F(x^{(t+1)})\leq G(x^{(t+1)}, x^{(t)})\leq G(x^{(t)}, x^{(t)}) = F(x^{(t)})$.

Now, we prove the convergence of the iterative updating rules for $\mathbf{V}$ in Eq (3.5). First, we suppose that $v_{ab}$ is any entry in the matrices $\mathbf{V}$, and $F_{v_{ab}}$ denotes the part of $O$ only relevant to $v_{ab}$. Therefore, we have the derivative of $F_{ab}$ with respect to $v_{ab}$ as follows:

From Eq (3.5), we can observe that the updated rule is element-wise. That means we only need to prove $F_{z_{ab}}$ is non-increasing under the updated rule for Eq (3.5).

Lemma 4.2. Let $F'$ denotes the first order derivative with respect to $\mathbf{V}$. The function

is an auxiliary function of $F_{z_{ab}}$.

Proof. Evidently, $G(v, v) = F_{v_{ab}}(v)$, we only have to show that $G(v, v_{ab}^{(t)})\geq F_{v_{ab}}(v)$. To achieve this goal, we can compare the Taylor series expansion of $F_{v_{ab}}(v)$ at $v_{ab}^{t}$ with the auxiliary function $G(v, v_{ab}^{t})$:

Putting (4.4) to (4.6) and comparing with (4.5) to find that $G(v, v_{ab}^{(t)})\geq F_{v_{ab}}(v)$ is equivalent to

In order to prove the above inequality, we have:

and $(\beta \mathbf{D}/2)_{ab}\geq 0$. Therefore, Eq (4.7) holds and we have $G(v, v_{ab}^{(t)})\geq F_{v_{ab}}(v)$.

Now, we can show the convergence of Theorem 4.1 for $\textbf{V}$:

Proof of Theorem 4.1. For the updated rule for Eq (3.5) of $\textbf{V}$, we can replace $G(v, v_{ab}^{(t)})$ in Eq (4.2) by Eq (4.5) to obtain the updated rule, which is exactly the same with the iterative updated rule for $\mathbf{V}$.

Since Eq (4.5) is an auxiliary function, $F_{v_{ab}}$ is non-increasing under the updated rule for Eq (3.5) with Lemma 4.2.

5.   Experimental results

In this section, we report several experiments from the performance of the proposed SGCF and compare them with the existing methods on the COIL20, PIE, Yale B and MNIST datasets. The important statistic properties of these datasets are enumerated in Table 2. We will describe the details of these datasets individually later on. The compared methods as follows:

● Traditional K-means clustering algorithm (K-means);

● Nonnegative matrix factorization (NMF) ^[6,7];

● Concept factorization (CF) ^[18];

● Locally consistent concept factorization (LCCF) ^[19];

● Constrained concept factorization (CCF) ^[33];

● Constrained nonnegative matrix factorization (CNMF) ^[14];

● Discriminative nonnegative matrix factorization (DNMF) ^[27];

● Class-driven concept factorization (CDCF) ^[41];

Among them, K-means, NMF, CF and LCCF are unsupervised learning algorithms, while CCF, CDCF, CNMF, DNMF and our proposed SGCF are semi-supervised learning algorithms, and K-means is conduct on original datasets. For the semi-supervised learning algorithms, we randomly picked 20% samples from each class as the available label information in each trial, and used them to construct the label indicator matrix $D$. In order to achieve better representative results, we always set the new dimension $k$ of the data space equal to the cluster number $P$ by selecting a random subset of all classes. Then $K$-means was repeated 20 times in the new representation $\mathbf{V}$ and the best result was recorded. The number of nearest neighbors $p$ was set to be five in the graph regularization methods as suggest in ^[34,37], such as LCCF, CNMF, DNMF and SGCF. Two widely-used evaluation metrics, clustering accuracy (AC) and normalized mutual information (NMI), were used to evaluate the clustering performance, and detailed definitions of these evaluation metrics can be found in ^[14,28,37]. For each dateset, we made twenty independent trials and recorded the average clustering result.

5.1.   Experiments on COIL20 dataset

The COIL20 dataset consists of the gray-scale images of 20 objects, and each image is manually cropped to $32\times 32$. These objects were placed on a motorized turntable, which was rotated 360 degrees to various object poses with respect to a fixed camera. Figure 1 shows some images from the COIL20 dataset.

Form Table 3, we can find that proposed SGCF achieves the best average clustering performance. Compared with NMF, CF and LCCF, the SGCF can achieve 15.35%, 17.41% and 3.53% improvement in AC on average, respectively. The corresponding NMI on average increases are 22.22%, 22.87% and 4.81% improvement, respectively. Therefore, the performances of SGCF surpasses those of NMF, CF and LCCF. This is mainly due to the fact that NMF, CF and LCCF are unsupervised learning algorithms, while SGCF is a semi-supervised learning algorithm, which considers the label information by introducing the class-driven constraint.

5.2.   Experiments on PIE face dataset

The PIE dataset contains 11,554 face images of 68 different people, each of which has 42 facial images under different light and illumination conditions. Each image is manually cropped to $32\times 32$. For the fixed cluster number $P$, we randomly selected 100 images of each class in this experiment. Figure 2 shows some images from the PIE face dataset.

Table 4 shows that our proposed SGCF achieves the best performance for all different cluster number $P$. Specifically, the AC and NMI of LCCF are 32.57% and 12.35% with 5.00% and 8.88% gains over CF, respectively. The AC and NMI of SGCF are 62.08% and 44.74% with gains of 29.51% and 32.39% over LCCF, respectively. Therefore, the performance of SGCF is superior to those of LCCF and CF. The main reason is that CF and LCCF are unsupervised learning methods, while SGCF integrates the label information into CF by utilizing the class-driven constraint. Meanwhile, it is noteworthy to point out that SGCF obtains the best performance due to the fact that SGCF effectively takes both the class-driven constraint and graph regularization into account.

5.3.   Experiments on Yale B dataset

The Extended Yale B dataset contains 2,414 frontal face images from 38 objects, each of which contains 64 face images captured under controlled lighting in the laboratory. Each image is manually cropped to $32\times 32$. In this experiment, we randomly selected 50 images of each class for the fixed cluster number $P$. Figure 3 shows some images from the Yale B dataset.

From Table 5, it can be observed that SGCF achieves the best performance in clustering. The average clustering accuracies obtained by k-means, NMF, CF, LCCF, CCF, CDCF, CNMF, DNMF and SGCF are 25.67%, 28.14%, 26.67%, 49.10%, 30.35%, 39.82%, 32.59%, 35.21% and 65.42%, respectively. Compared with DNMF, SGCF can achieve 30.21% improvement in AC and 39.70% improvement in NMI. The important reason is that DNMF cannot capture the intrinsic geometrical structure of data space, while SGCF can efficiently detect the intrinsic structure of data space by using the graph regularization. Thus, the clustering performance can be greatly improved.

5.4.   Experiments on MNIST database

The MNIST dataset classifies 10 classes from zero to nine, which provides a total of 70,000 handwritten samples. Each image is resized to $28\times28$ gray-scale images. Similar to experiments in ^[17], we randomly choose 100 images of each class in this experiment. Figure 4 shows some handwritten samples from the MNIST dataset.

We can see from Table 6 that the proposed SGCF obtains the best performance compared with the others methods. Specifically, compared with CCF and CDCF, SGCF can achieve 16.47% and 5.09% improvement in AC, respectively. For NMI, SGCF can achieve 13.05% and 7.73% improvement, respectively. Therefore, the performance of SGCF is significantly superior to the CCF and CDCF. The major reason is that CCF and CDCF do not consider the graph regularization, while SGCF not only preserves the intrinsic structure of data space by using the graph regularization, but also considers the available label information by utilizing the class-driven constraints. Besides, we can see that the semi-supervised learning methods (i.e., CCF, CDCF, CNMF, DNMF and SGCF) consistently outperform other unsupervised learning methods, such as K-means, NMF, CF and LCCF. This indicates that the limited label information can improve the clustering performance for semi-supervised learning methods, which proves the effectiveness of SGCF yet again.

5.5.   Parameters analysis

In this subsection, we show how the tradeoff parameters $\alpha, \beta$ can influence the performance of clustering. Here, we randomly choose 50 images of each class and randomly select five classes from each dataset as a subset of dataset in these experiments. Following several existing works ^{[23,27,32,37]}, we separately analyze $\alpha$ and $\beta$ in SGCF. First, we empirically fix the parameters $\beta$, and then search the optimal parameters $\alpha$ from [0.01, 0.1, 1, 10,100, 1000]. Second, we fix the optimal parameters $\alpha$, and search the optimal parameters $\beta$ from a range of [0.01, 0.1, 1, 10,100, 1000, 10000] in SGCF. We independently run on 20 times for each experiment, and the results are shown in Figures 5 and 6. The parameter setting of LCCF, CDCF, DNMF and SGCF are shown in Table 7.

It's worth noting that the proposed SGCF is a semi-supervised learning method, and the performance of the algorithm is closely related to the proportion of labeled data. In order to demonstrate how the performance of SGCF varies with labeled data, we select the proportions of labeled data from each class with the label percentage varying from 10% to 50% as the labeled data constructs the label indicator matrix $D$ for each dataset. We randomly choose five classes from each dataset and independently run on 20 times for each experiment, and the experimental results are shown in Figure 7. From Figure 7, we can see that the clustering performance of CCF, CDCF, CNMF, DNMF and SGCF are generally improved with the increase of the proportion of the labeled data, while the unsupervised learning methods K-means, NMF, CF and LCCF have clustreing performances that are not changed because it's irrespective of the proportion of the labeled data variation. However, it can be found that the SGCF can achieve better performance when the proportion of the labeled data is small. Therefore, the proposed SGCF is a more robust method.

6.   Conclusions

In this paper, a new SGCF with the class-driven constraint was proposed. In this framework, the label information with the class-driven constraint was encoded in CF, which forces the new representations of samples to be more similar within the same class while different between unalike classes. To extract the geometric information of the original data space, a graph regularization was added to the objective function of SGCF. For the proposed SGCF method, iterative updated rules were proposed and the convergence was proven both theoretically and experimentally. The experimental results on four databases have demonstrated the better performance of SGCF in comparison to the other state-of-the-art matrix factorization methods.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

This work is supported by the Key Project of Ningxia Natural Science Foundation under Grant no. 2022AAC02043, the First-class Discipline Construction Fund project of Ningxia Higher Education under Grant no. NXYLXK2017B09, the Natural Science Foundation of Shaanxi Province under Grant no. 2020JM-630, the Major scientific Research Project of Northern University for Nationalities under Grant no. ZDZX201901, the Shangluo Universities Key Disciplines Project (Discipline name.Mathematics) and the Fundamental Research Fund of Shangluo University under Grant no. 18SKY009, 19SCX02.

Conflict of interest

The authors declare that they have no conflicts of interest related to this article.