FedSC: A federated learning algorithm based on client-side clustering

1.   Introduction

Nowadays, smart terminals, such as smart bracelets, are becoming increasingly popular. These devices are easy to carry and have many powerful sensors that can detect various inputs from the wearer's body ^[1,2]. These data have strong research value. Using these data to train machine learning models will hopefully make these smart terminals smarter to better serve people. As people pay increasing attention to the protection of data privacy ^[3,4], it is impossible and inadvisable to collect these private data, which leads to the problem of data silos ^[5]. However, if only local data is used for training, then the following problems emerge: 1) insufficient local data leads to model convergence difficulties and poor performance, and 2) limited diversity of data types hampers the model's generalization capability. As a solution, federated learning is proposed ^[6,7]. In federated learning, each participant uses their own private data to train the local model that is sent to the central server for aggregation to obtain the global model. Under the framework of federated learning, participants' private data are not exported locally, which protects participants' data privacy ^[8,9,10].

Although federated learning offers significant advantages in terms of data privacy protection compared to centralized training, it also faces numerous challenges ^[11,12]. Among these challenges, heterogeneity stands out as the most crucial one ^[13,14]. System heterogeneity arises when participants have varying storage capacities and computing power ^[15,16]. Statistical heterogeneity, on the other hand, emerges when participants exhibit different data type distributions and data volumes, leading to the non-IID (non-Independent and Identically Distributed) problem ^[17]. While hardware advancements have gradually addressed system heterogeneity, statistical heterogeneity remains a persistent challenge. Non-IID data are prevalent in real-life scenarios, and federated learning's performance experiences a notable decline when confronted with such data. The classical federated learning algorithm proposed by Google ^{[18,19,20,21]}, FedAvg (Federated Averaging), fails to achieve superior performance across all clients compared to models trained locally by individual clients. Some participants even experience minimal benefits or inferior performance when participating in federated learning, discouraging their willingness to engage in the process. The existing FedAvg algorithm in federated learning no longer meets practical requirements, necessitating the development of a novel algorithm to address the statistical heterogeneity challenge posed by non-IID data distributions. This research aims to tackle this challenge and provide a solution in this paper.

The negative effects of non-IID data on FedAvg can be explained by the model parameters.

We define $l(x_i, y_i; w)$ as the loss of the prediction on example $(x_i, y_i)$ made with model parameters $w$. We aim for:

In the centralized machine learning environment, let $w_t^{(c)}$ denote the weight after $t-th$ update in the centralized machine learning. Then, the model parameters are updated in the following way:

In the federal learning environment, we assume there are $K$ clients over which the data is partitioned, with $P_k$ as the set of indexes of data points on client $k$, with $n_k = |P_k|$. On each client, local training is conducted separately using local data:

Let $w_{t}^{(f)}$ denote the weight calculated:

Finally, we define:

The divergence between $w_{t}^{(k)}$, $w_{t}^{(f)}$ and $w_{t}^{(c)}$ can be visualized from the Figures 1 and 2. When data is IID, for each client $k$, the data distribution is almost identical to the global data distribution and the divergence between $w_{t}^{(k)}$ and $w_{t}^{(c)}$ is small. Therefore, $w_{t}^{(f)}$ obtained after aggregating different $w_{t}^{(k)}$ according to Eq (1.4) also has very small divergence with $w_{t}^{(c)}$. After many iterations, $w_{t}^{(f)}$ is still close to $w_{t}^{(c)}$ and the $weight_{divergence}$ is small. At this point, federated learning can perform very well. When data is non-IID, the large difference in the distribution of data owned by each client resulting in the divergence between $w_{t}^{(k)}$ and $w_{t}^{(k)}$ becomes larger and the divergence between $w_{t}^{(k)}$ and $w_{t}^{(c)}$ also becomes larger. Therefore, the divergence between $w_{t}^{(f)}$ and $w_{t}^{(c)}$ also becomes much larger and accumulates very fast. After many iterations, the $weight_{divergence}$ becomes larger and larger. From the above analysis, it is concluded that the negative impact of non-IID data on federated learning is mainly due to the difference in data distribution of clients. Based on this, we improve the FedAvg algorithm and propose a new algorithm, which clusters clients with similar data distribution to find local IID data in the non-IID data distribution to solve the non-IID problem.

Contributions of this paper:

1) In this paper, FedSC algorithm is proposed to improve the accuracy of federated learning in the case of data imbalance, which helps to solve the problem of data heterogeneity in federated learning.

2) This paper reproduces a variety of federated learning algorithms, such as FedNova ^[22], SCAFFOLD ^[23], FedProx ^[17], etc. Through the comparisons, it demonstrates that FedSC consistently delivers strong performance across different scenarios.

3) The data training of the FedSC algorithm is performed only locally and no local data transmission is involved, which ensures data privacy and security for all participants, which helps to solve the problem of "data silos".

The structure of this paper is as follows. Section 2 presents the related work. Section 3 describes the FedSC algorithm flow in detail as well as the implementation details of each part. Section 4 presents the experimental results. Section 5 gives a summary of the full text.

2.   Related work

Non-IID data distribution is common in real life ^[24], for example, different regions may have completely different vegetation distribution ^[25]. Due to data regulation and privacy concerns, meaningful real federated datasets are difficult to obtain ^[26]. In this paper, we use the Dirichlet distribution to simulate the non-IID distribution of data, where each client is allocated a proportion of the samples of each label according to Dirichlet distribution. Dirichlet distribution is an appropriate choice to simulate non-IID data distribution and has been used in many studies ^[22,27,28]. Specifically, we sample $p_k = Dir_N\left(\beta \right)$ and allocate a $p_{k, j}$ proportion of the instances of class $k$ to the party $j$. Here, $Dir_N\left(\beta \right)$ denotes the Dirichlet distribution and $\beta$ is a concentration parameter ($\beta > 0$). An advantage of this approach is that we can flexibly change the imbalance level by varying the concentration parameter $\beta$. If $\beta$ is set to a smaller value, then the partition is more unbalanced.

The biggest challenge of federated learning is that its performance deteriorates when data are distributed as non-IID ^[29,30]. Many methods have been put forward to solve this problem ^{[31,32,33,34]}. The FedProx algorithm is proposed in ^[17], which improves the local objective function based on the FedAvg algorithm. The FedProx algorithm uses an additional adjustment term in the local objective function to limit the distance between the local model and the global model so that the local model will not be too scattered. This helps to avoid local inconsistencies and improves the generalization of the model. In addition, FedProx can balance the distance between the global and local models by adjusting the regularization hyperparameters to better accommodate different data distributions. However, its shortcomings are also obvious. The client needs to carefully adjust the proportion of the adjustment item in the local objective function. If the proportion is too small, then the adjustment has little effect. If the proportion is too large, then the local update is very small and the local model converges slowly. ^[22] considers that heterogeneity in the clients' local datasets and computation speeds results in large variations in the number of local updates performed by each client in each communication round. Simple aggregation of such models causes the global model updates are biased. To tackle this challenge, FedNova improve FedAvg in the aggregation stage, which normalizes and scales the local updates of each party according to their number of local steps before updating the global model. Therefore, the client can iterate autonomously to make better use of the information from local data, improving the generalization capability of the model. The FedNova algorithm achieves better performance on some non-IID datasets. Scaffold ^[23] introduces two variates (server variate and client variate), which are used to estimate the update direction of the server model and the update direction of each client. Then the difference between these two variables is used to approximate the bias in local training. Finally, the local updates are corrected by adding the difference to the local training. The Scaffold algorithm makes the federated learning model more stable, avoiding instability caused by the non-IID datasets. However, due to additional control variables, Scaffold doubles the size of each communication round compared to FedAvg. In ^[35], a federated learning method named FedCPF is proposed to be applied to vehicle edge computing scenarios in 6G communication networks. The method improves the efficiency and accuracy of federated learning by improving the communication method and optimizing the local training process, while reducing the communication overhead. There are also some researchers who aim to improve the distribution of non-IID data by sharing a certain percentage of the data, such as FedShare ^[36], Hybrid-FL ^[37], etc. Although the participants can be seen as trustworthy, direct data transfer still risks privacy leakage, which is against the original purpose of federated learning. In ^[38], the author divides clients according to the cosine similarity between the weight-updates of different clients to realize federated training of multiple models. However, it generates different personalized models for different clients rather than a unified model. In ^[39], TiFL was proposed, which divides participants into multiple levels according to the different performance of each participant. TiFL selects clients from the same level in every round of training in order to alleviate the discrete problem caused by the heterogeneity of resources and quantity owned by participants. The concept of client-side clustering is also employed in this paper to address the issue of non-IID data distribution. By classifying and aggregating clients based on their data distribution, the goal is to achieve a more balanced data distribution within each cluster. This approach aims to minimize the adverse effects of non-IID data on the performance of federated learning algorithms.

3.   Algorithm

This section describes the FedSC algorithm in detail.

3.1.   Client clustering method based on local data type distribution

Non-IID data make local model parameters divergent in the federated learning so that the central server cannot aggregate a good model; however, federated learning can give satisfactory results with IID data. Thus, we cluster the clients according to their data distribution. The clients with high data distribution similarity are divided into a cluster; hence, data distribution in this cluster is similar to IID. At this time, federated optimization within each cluster can greatly improve the performance of the model and can improve the efficiency.

Let $n_{i}$ be the total amount of data samples owned by client $i$, $n_{ij}$ be the number of class $j$ data samples owned by client $i$, and define the data attribute of client i as $I_i = \left[n_{i1}/n_{i}, n_{i2}/n_{i}, ..., n_{ij}/n_{i}\right]$. Clients are clustered based on their data attributes.

A bottom-up clustering algorithm is proposed. Initially, each client is treated as an individual cluster, and clusters are progressively merged at each step. Ultimately, a cluster encompassing all samples is obtained. The detailed algorithm steps are as follows:

1) Treat each client as an individual cluster.

2) Calculate the distance between two clusters and merge the cluster with the smallest distance. In this case, the distance between two clusters is defined as the maximum distance between any pair of clients in the two clusters. Specifically, the distance between cluster $p$ and cluster $q$ is defined as follows:

where $d_{ij}$ represents the distance between the data attributes $I_i$ of client $i$ in cluster $p$ and the data attributes $I_j$ of client $j$ in cluster $q$.

3) Repeat step 2 and iteratively aggregate clusters until the number of clusters meets the requirements.

In this way, the data distribution among clients within the same cluster becomes similar. Consequently, federated learning can effectively operate within each cluster, yielding improved results.

3.2.   Serial training method for simulating centralized machine learning

In centralized machine learning, the model updates the model parameters by selecting a batch size of data at a time, as shown in Figure 3. All the data participate in the training process and each type of data makes its own contribution to the model fairly.

In FedAvg algorithm, the model parameters are updated by aggregating the model parameters of each client. The central server sends the initial model parameters $w_{t}^{(f)}$ to the client. The client uses $w_{t}^{(f)}$ as the model starting point to train with the local dataset and sends the trained model $w_{t}^{(k)}$ to the central server. The central server aggregates the obtained models to obtain $w_{t+1}^{(f)}$. The training process of federated learning is shown in Figure 4.

There is a wide gap between federated learning and centralized machine learning with regard to the method of updating model parameters. Due to the clustering of clients, the traditional training process of the federated learning algorithm is no longer applicable. Therefore, we have made improvements to address this issue. We make the data in each cluster contribute to the global model of federated learning by imitating the batch processing of centralized machine learning. Treat the model aggregated by the central server from client models in a cluster as a model generated from batch-size data in centralized machine learning. Then, the central server transmits the aggregated model to the clients in another cluster for training, which corresponds to using other batch-size data to continue training the model in centralized machine learning as Figure 5.

3.3.   Detailed steps are described below

First, each client sends its data attribute $I$ to the central server. Because the information about the proportion of the client data categories may be leaked during this process, the client encrypts the data attribute $I$ before sending it to the central server. After receiving data attributes from all clients and decrypting them, the central server uses the clustering algorithm to aggregate all clients into $N$ clusters. Next, the central server selects a $C-fraction$ of clients from the first cluster and send the initial global model parameters $w_0^{(f)}$ to them. After the clients train $E$ epochs with their private data, the obtained local model parameters are uploaded to the central server. The central server performs a weighted average of the obtained model parameters according to the number of data samples of the clients to obtain the global model parameter $w_1^{(f)}$. Then, the central server selects a $C-fraction$ of clients from the second cluster and sends the global model parameters to the selected clients. The selected client takes $w_1^{(f)}$ as the initial model parameter, uses the local dataset for training $E$ epochs, and sends the trained model parameters to the central server. After obtaining the model parameters of all clients selected from the second cluster, the central server aggregates them to obtain the global model parameter $w_2^{(f)}$. Subsequently, $w_2^{(f)}$ is sent to the clients selected from the third cluster, and so on. This process is repeated for $N$ aggregations, enabling the original model parameter to be trained with the data from $N$ clusters and obtain the model parameter $w_N^{(f)}$. The model parameter $w_N^{(f)}$ contains the knowledge learned from $N$ cluster data. This constitutes a communication process. Next, the central server sends the model parameters $w_N^{(f)}$ to the clients selected from the first cluster. This process is repeated until the model converges. In centralized machine learning, all data types are traversed in batch-size units. In this paper's proposed algorithm, all data types are traversed in units of data from clients selected from each cluster. In the FedSC algorithm, data training only occurs locally on the client side. The client and server solely exchange model parameters, without sharing any data. This approach mitigates the risk of data leakage to a certain extent and ensures the data security of clients. The schematic diagram of the FedSC algorithm is shown in Figure 6 and the pseudocode of the FedSC algorithm is shown in Algorithm 1.

4.   Experiment

To investigate the effectiveness of FedSC algorithms on non-IID data setting, we conduct extensive experiments on three public datasets, including two image datasets (i.e., MNIST ^[40], FEMNIST ^[41]) and one tabular dataset (we collect 6000 pieces each of 10 kinds of attack flows from the network security dataset CICIDS2017 ^[42] as the training set, and 1000 pieces each of the corresponding attack flows as the test set). The statistics of the datasets are summarized in Table 1. We use an MLP with three hidden layers as the base model. We also use the SGD optimizer with learning rate 0.01 and the batch size is set to 64. Furthermore, we reproduce FedAvg, FedProx, Scaffold, and FedNova algorithms and run all the algorithms for the same number of rounds for fair comparison. By default, the number of rounds is set to 100, the number of clients is set to 100($P$), the number of local epochs is set to 1($E$), the percentage of clients selected is set to 1($C$), the $\beta$ of Dirichlet distribution is set to 0.5($D$), the added noise is set to 0($Z$) and the number of clustering is set to 10($G$), unless state otherwise. Because the selection of clients in federated learning has a certain degree of randomness, the average accuracy rate is used as the standard instead of the highest accuracy rate. The model building and simulation in this paper are implemented on Python3.7. The pytorch-gpu1.10.1 framework is mainly used to build the model.

4.1.   The influence of the order of data samples on the algorithm

The order of data samples has an important impact on machine learning. Here, we explore the influence of data sample order on FedSC algorithm. When data samples are arranged in a certain rule, machine learning learns this rule as a feature, which leads to overfitting. The clients are clustered and experimented, and then the order of the clusters is shuffled and experimented again for comparison. The experimental results are shown in Figure 7.

It can be seen from Figure 7 that the order of data samples has little influence on FedSC algorithm. In the FedSC algorithm, clients are randomly selected in each round of communication and aggregated in the cluster. These operations effectively mitigate overfitting, minimize the influence of data sample order on the model, and enhance the model's generalization ability.

4.2.   The influence of the number of cluster on the algorithm

In the FedSC algorithm, it is important to determine the optimal number of client clusters. In this experiment, the data is divided into 100 clients, and the number of clusters G is varied as 1, 10, 50, and 100. The experimental results are shown in Figure 8.

As can be seen from Figure 8, it is reasonable that the more clusters there are, the better the final result will be. In the FedSC algorithm, the model aggregated within each cluster is treated as a model trained on a batch-size of data in traditional machine learning. When the number of client clusters is small, each cluster contains a larger number of clients, resulting in scattered data distributions among clients within the cluster. This makes it challenging to aggregate a high-quality model, leading to poor performance of the final global model. When the number of clusters is large, a better model can be aggregated, but a large number of serial training consumes a long time. When G = 1, the FedSC algorithm becomes the FedAvg algorithm, which is affected by the non-IID and cannot aggregate a good global model. When G = 100, each client is equivalent to batch-size in traditional machine learning, and better results can be obtained at the cost of consuming more time. In order to balance the effect of the model and the cost of time consumed, the following experiment set the number of client clusters to 10, that is, the number of data types.

4.3.   Comparison between different number of client settings

The number of clients participating in federated learning has been increasing due to the advantages it offers in addressing data uncontrollability and data leakage issues associated with traditional centralized training. Thus, we verify the effect of the number of clients on the algorithm; that is, the effect of the hyperparameter $P$ in the algorithm on the experiment, which controls the number of clients in federated learning. The results obtained are shown in Table 2 and Figure 9.

It can be seen from Table 2 and Figure 9 that the number of clients has a great impact on federated learning, but the FedSC algorithm is better than other algorithms. The accuracy of the FedAvg, FedNova, Scaffold and FedProx algorithms decreases significantly as the number of clients increases. For instance, when the number of clients increases from 10 to 1000, the accuracy decreases by almost half. When the total data volume is certain, the more clients there are, the less data volume each client has. Therefore, the local model trained by each client may not be accurate enough, resulting in a degradation of the performance of the global model. The accuracy of FedSC algorithm decreases as the number of clients increases, but it is still within an acceptable range. Too many clients also means more communication, which leads to increased communication overhead. Too many clients can also lead to an increase in federated learning training time, as all clients need to be trained before model aggregation can take place. Therefore, reasonable control of the number of clients is very important for the performance and efficiency of federated learning. It is important to ensure that the model can be adequately trained from diverse data while also ensuring training speed and communication efficiency.

4.4.   Comparison between different data distribution settings

Now, we verify the effect of different data distributions on federated learning, i.e., the effect of the hyperparameter $D$ on the experiments. When $D$ becomes small, the data distribution is biased towards non-IID. When $D$ becomes large, the data distribution is biased towards IID. The results are shown in Table 3 and Figure 10.

It can be seen from Table 3 and Figure 10 that the FedSC algorithm shows better results regardless of whether the data distribution is biased towards IID or non-IID. Additionally, all algorithms exhibit better performance when the data distribution is biased towards IID compared to when it is biased towards non-IID. In ^[36], it is pointed out that forcing the average fusion of model parameters with large differences will lead to a decrease in model accuracy. When the data distribution is biased towards non-IID, the dataset owned by each client are obviously different, resulting in different models trained by each client. At this time, using the FedAvg algorithm, the model parameters provided by each client vary greatly and the model after server aggregation will have significant deviations, leading to model performance degradation. Although the FedProx, FedNova and Scaffold algorithms have improved the FedAvg algorithm, the results are still less than satisfactory. In the FedSC algorithm, clients in each cluster have roughly the same data so that the local model of clients in each cluster is roughly the same. The central server aggregates these local models from each cluster, resulting in a global model that incorporates more comprehensive characteristics of such data. As a result, the FedSC algorithm achieves better and more stable performance.

4.5.   Comparison among different number of clients selected settings

Here, we verify the influence of the number of clients selected in each round of communication on the algorithm; that is, the influence of the hyperparameter $C$ in the algorithm on the experiment, which controls the number of parallel clients in federated learning. Each round, $C$ of all clients are randomly selected from all clients. The results obtained are shown in Table 4 and Figure 11.

It can be seen from Table 4 and Figure 11 that the FedSC algorithm consistently achieves a higher accuracy rate compared to other algorithms in all scenarios. This superiority can be attributed to the fact that the FedSC algorithm ensures that the data within each cluster is almost IID, resulting in similar parameters among local models. Consequently, a global model with improved performance can be effectively aggregated. However, the training process of the FedSC algorithm is very unstable and the curve fluctuates a lot when fewer clients are selected each time, which is a drawback of the FedSC. The Scaffold algorithm is very stable in the training process with fewer selected clients, but its communication per round increases due to the additional variables introduced by the Scaffold algorithm.

It can also be seen that regardless of the FedSC algorithm or other algorithms, the more clients selected each time, the smaller the fluctuation of the training result, the more stable the training process, and the smoother the training curve. This is due to the fact that selecting more clients increases the number of data samples seen in each training round, resulting in more accurate results. Therefore, there will be no large fluctuations after aggregation.

Increasing the number of clients selected in each round has a positive impact on all algorithms, as it improves stability and accuracy. Selecting more clients per round of training is an effective approach to enhance performance. However, because each client needs to upload or download model parameters from the central server for each round of training, selecting more clients means more traffic. The FedSC algorithm can achieve relatively good results when $C$ is small, which means that few clients are needed to be selected in federated learning to achieve the desired accuracy and fewer clients means a reduction in total traffic. This is helpful to solve the problem of communication bottlenecks in federated learning.

4.6.   Comparison among different number of local epochs settings

Here, we verify the effect of the number of rounds updated locally by each client on federated learning, i.e., the effect of the hyperparameter $E$ on the experiments. The results are shown in Table 5 and Figure 12.

As can be seen from Table 5 and Figure 12, with the change of the number of local training rounds, the FedSC algorithm keeps a strong momentum. The average accuracy of FedSC algorithm is still at a high level and the training process is relatively stable, which is sufficient to demonstrate the superiority of the proposed FedSC algorithm.

It can be seen that the increase in the number of local training rounds has a certain improvement for all algorithms. When there are fewer local training rounds, the client may not be trained sufficiently locally and the uploaded model parameters may not be accurate enough, thus affecting the performance of the global model. Increasing the number of local training rounds enables the local model to learn the features of the local data of the client better, and the effect of the global model is naturally better and more stable. Therefore, increasing the number of local training rounds is a good way to improve the accuracy and stability of the algorithm. However, a larger number of local training epochs increases the local computation of the client, which increases the training time and communication burden. Therefore, the selection of local training rounds is crucial to the performance of federated learning, and it needs to consider the factors such as data set, model complexity, computing and communication resources.

4.7.   The effect of noise on federated learning

Federated learning techniques are vulnerable to Byzantine failures ^[43,44,45], biased local datasets, and poisoning attacks. Here, we verify the performance of federated learning algorithm when the client is disturbed by noise. Gaussian noise is added to the dataset to make the client training results inaccurate. Gaussian noise is popular ^[46], especially in images. Add Gaussian noise to each pixel point in the image data and Gaussian noise to each data in the table data. The expression of Gaussian noise is $\hat{x}\sim Gau\left(\mu, \sigma \right)$, set the mean value of Gaussian noise $u = 0$ and change the variance of Gaussian noise $\sigma$. It can be seen from Figure 13 that the larger the variance of the added Gaussian noise, the blurrier the image.

Set the variance of Gaussian noise to 0, 1 and 5 for the mnist dataset and fmnist dataset, respectively, and set the variance of Gaussian noise to 0, 1 and 2 for the CICIDS2017 dataset. The experimental results are shown in Table 6 and Figure 14.

The addition of noise will disturb the information in the data set, which will have a negative impact on the performance of federated learning. With the addition of Gaussian noise, the data in the dataset becomes more complex and difficult to process, making it difficult for the client to capture the features of the data during local training. Therefore, adding noise makes the training results of some clients unreliable or inaccurate, thus affecting the update results of the global model of the federated learning algorithm. As can be seen from Figure 14, the performance of all federated learning algorithms declines as the level of noise increases. Although the FedSC algorithm is also affected by noise, compared with other algorithms FedSC algorithm still gives better results. The addition of Gaussian noise can increase the privacy of data, but it will also affect the performance of the model.

4.8.   Comparison of the efficiency of different federal learning algorithms

To compare the efficiency of different FL algorithms, we set P = 100, D = 0.5 C = 1, E = 1, Z = 0 for experiment. In Table 7, the total computation time of 100 communication rounds is shown, and in Table 8, the communication cost per client in each communication round is shown.

We can observe that the time consumed by the FedSC, Fedavg and FedNove algorithms is very close. However, the Scaffold algorithm introduces additional operations during training, resulting in slightly increased time consumption. FedProx directly modifies the objective, which causes additional computation overhead in the gradient descent of each batch and therefore takes the longest time. For the communication cost, the FedSC, FedAvg, FedNove and FedProx algorithms upload only their local model parameters per round, so the communication cost is the same. However, the Scaffold algorithm transmits additional control variables in each round, so its communication cost is twice that of the other algorithms.

5.   Conclusions

Federated learning plays a pivotal role in the field of machine learning, particularly as the importance of data privacy continues to grow. Unfortunately, the accuracy of federated learning is significantly influenced by the distribution of data. When the data exhibits non-IID characteristics, the performance of federated learning deteriorates. This paper introduces a novel federated learning algorithm called FedSC, which outperforms other algorithms in non-IID data scenarios. The FedSC algorithm aims to identify local IID data within non-IID data to mitigate the adverse impact of non-IID data on federated learning. Through experimental comparisons, the FedSC algorithm consistently achieves higher accuracy compared to alternative federated learning algorithms. Moreover, the FedSC algorithm ensures data security by completing the model training without sharing data among participating parties. This contribution further advances the development of federated learning.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Conflict of interest

The authors declare there is no conflicts of interest.