Adaptive rotation attention network for accurate defect detection on magnetic tile surface

1.   Introduction

In industrial production, magnetic tiles are widely used in the rotor or stator of permanent magnet motors ^[1,2,3]. However, mechanical friction and human unconscious collisions will inevitably lead to surface defects of magnetic tiles ^[4], adversely affecting the quality of permanent magnet motors, and even causing disasters and accidents ^[5]. Therefore, it is essential to detect the defects on the magnetic tile surface in the production process.

Recently, defect detection on magnetic tile surfaces mainly depends on experienced workers ^[6,7], but it is time-consuming and has low-accuracy. To address this problem, many methods are proposed for defect detection on the magnetic tile surface, including traditional techniques and machine learning-based approaches ^[8,9]. For the traditional methods, Xie et al. propose a surface defect inspection method on magnetic tiles based on shearlet ^[10]. Yang et al. propose an effective method for defect detection in magnetic tiles using a stationary wavelet transform ^[11]. These traditional methods tend to extract representative features from magnetic tiles to perform defect detection, but they are conducted under specific conditions and show low results. Compared to the traditional methods, machine learning-based approaches have achieved better performance on this task. For example, an unsupervised segmentation method is proposed for defect detection based on attention-enhanced flexible U-Net ^[12]. In ^[13], researchers propose a fusion feature network for detecting surface defect on magnetic tiles using an attention mechanism. Liang et al. propose ^[14] a feature enhancement and loop-shaped fusion network for surface defects detection on magnetic tiles by enhancing shallow features and fusing loop-shaped features.

Although these machine learning-based methods have shown good results in detecting surface defects on magnetic tiles, the practice application is still difficult because of these problems. First, most defects randomly appear on the magnetic tile surface (see Figure 1) ^[15,16], resulting in the difficulty for networks to capture these defects, further degrading the performance of deep learning models. Second, the defects on magnetic tile surfaces vary greatly, such as the long thin cracks and tiny deformation ^[17,18], and they are always overwhelmed by the background, leading to failure detection by networks, as shown in Figure 1(a)–(c).

Figure 1.  Examples of the defect on the magnetic tile surface, where they appears randomly and are tiny, and often overwhelmed by the background.

DownLoad: Full-Size Img PowerPoint

Motivated by these observations, an Adaptive Rotation Attention Network (ARA-Net) is proposed for defect detection on the magnetic tile surface, where the Adaptive Rotation Convolution (ARC) module is devised to capture the random defects on the magnetic tile surface by multi-view learning from rotated feature maps, and the Rotation Region Attention (RAA) module is designed to locate the small defects from the complicated background by focusing more attention on defect features. Our main contributions are summarized as follows.

$1)$ The Adaptive Rotation Convolution (ARC) module, which can learn the multi-view features on the feature map, is devised to capture the random defects on the magnetic tile surface.

$2)$ The Rotation Region Attention (RAA) module, which can focus on the attention of defect features, is designed to locate the small defects from the complicated background.

$3)$ Extensive experiments demonstrate the effectiveness of the proposed ARA-Net, and it outperforms the state-of-the-art approaches.

The rest of this paper is organized as follows: the proposed ARA-Net is described in Section 2, while experiments are discussed in Section 3, and Section 4 shows the conclusions.

2.   Adaptive rotation attention network for defect detection on magnetic tile surface

In this section, an Adaptive Rotation Attention Network (ARA-Net) is proposed for defect detection on the magnetic tile surface, and it consists of two key parts: the Adaptive Rotation Convolution (ARC) module and Rotation Region Attention (RAA) module, as shown in Figure 2, which are shown as follows.

Figure 2.  The framework of the Adaptive Rotation Attention Network (ARA-Net), where the Adaptive Rotation Convolution (ARC) module is devised to capture the random defects on the magnetic tile surface by multi-view learning on the feature map, and then the Rotation Region Attention (RAA) module is developed to locate the small defects from the background by focusing more attention on defect features. Finally, the feature maps are fed into the Feed Forward Networks to perform defect classification on the magnetic tile surface.

DownLoad: Full-Size Img PowerPoint

2.1.   Adaptive rotation convolution module for multi-view features learning

In the production of permanent magnet motors, most of the defects randomly appear on the magnetic tile surface ^[18], leading to network capture failure and degrading the performance of the deep learning model (Figure 1). Correspondingly, multi-view learning and rotated convolution ^[19] can provide rich information about feature maps for network training, thus capturing the object from the complicated background ^[20,21]. Inspired by this, an Adaptive Rotation Convolution (ARC) module is developed to capture the random defects on the magnetic tile surface by multi-view features learning, where the rotation convolution can adaptively obtain feature maps from various angles ^[19,35], as shown in the Adaptive Rotation Convolution Module of Figure 2, defined as

where BR and AR denote the basic rotation and adaptive rotation, respectively, $\alpha$ is the learnable parameter, and Concat represents the concatenation, which are described as follows:

● First, the head convolution is utilized to generate the head feature maps (H).

● Second, the head feature maps (H) are rotated to produce basic multi-view feature maps (M) with four angles: $0^{\circ}$, $90^{\circ}$, $180^{\circ}$ and $270^{\circ}$.

● Third, the basic multi-view feature maps (M) are adaptively rotated to obtain four different-offset feature maps (O), and the offset angles are calculated by linear layer from the head feature maps (H). Specifically, as shown in Figure 3, (a), (c), (e) and (g) denote the rotated feature maps at $0^{\circ}$, $90^{\circ}$, $180^{\circ}$ and $270^{\circ}$ angles, (b), (d), (f) and (h) are rotated from (a), (c), (e), and (g) with any angle, calculated by the linear layer, respectively}.

Figure 3.  The diagram of the adaptive rotation for feature map convolution, where the $0^{\circ}$, $90^{\circ}$, $180^{\circ}$, and $270^{\circ}$ angles are used as the basic shifted feature maps, and then the offset angles between them are adaptively calculated from the Linear layers, which can capture multi-view details from the feature maps.

DownLoad: Full-Size Img PowerPoint

● Fourth, these four different-offset feature maps (O) are added with the basic multi-view feature maps (M) to generate the rotated feature maps (R).

● Finally, the summation results of the rotated feature maps (R) are concatenated to produce the output feature maps.

After these processes, the proposed ARC module can capture multi-view feature maps to locate the random defects on the magnetic tile surface.

2.2.   Rotation region attention module for defect features attention

Some small defects, such as the tiny blowhole and the thin crack, always lead to difficulty in defect detection on the magnetic tile surface ^[22,23]. Correspondingly, the attention mechanism has shown excellent performance in object detection ^[24,25]. Motivated by this, the Rotation Region Attention (RAA) module, is designed to locate the small defects from the complicated background by focusing more attention on defect features, as shown in the Rotation Region Attention Module of Figure 2, defined as

where AT is the self-attention mechanism, and AR denotes the different-offset feature maps (R) obtained from the ARC module, which is shown as follows.

● First, the feature maps (X) are used as the query (Q), Key (K) and value (V), respectively.

● Second, the Query (Q) is flattened and transposed at the space dimension, and then multiplied with the flattened Key (K) to generate the attention weight maps (A).

● Third, the attention weight maps (A) are multiplied with the flattened value (V) to obtain the output feature maps (F).

● Finally, the output feature maps (F) will spatially flatten and add with the different-offset feature maps (R) to produce efficient feature maps.

To this end, the proposed RRA module can help the network focus attention on the small defects on the magnetic tile surface, thereby improving the performance of the proposed ARA-Net.

3.   Experiments, and results analysis

3.1.   Datasets

To validate the performance of ARA-Net for defects detection on the magnetic tile surface, the MTSD3C6K dataset ^[35] is employed to perform experiments, where there are 6450 samples, including 2023 crack images, 2381 normal samples and 2046 blowhole images. In the experiments, this dataset is divided into the training set, validation set, and test set, with the numbers of 3870, 1290 and 1290, respectively.

3.2.   Evaluation metrics

To verify the defect detection performance on the magnetic tile surface, Accuracy, Precision, Recall, F1-score, and Kappa score are used as evaluation metrics:

and

where CAccuracy denotes the class accuracy, computed by

in which $a_i$ is the actual number of samples for each category, and $b_i$ is the number of samples predicted for each class.

Moreover, to validate the complexity of models, the model size, i.e., parameters (Params), is applied to compute the space consumption of models, and the frames per second (FPS) are used to measure the actual inference speed of the model. Furthermore, the confusion matrix is provided to show clear insight into the model performance.

3.3.   Implementation details

The proposed ARA-Net is constructed by Pytorch 1.9.0 ^[26] on a server with one NVIDIA GeForce GTX 3090Ti GPU, the SGD optimizer ^[27] is utilized to obtain the high-performance model, the epoch, batch size, and the initial learning rate are set as 100, 32, and of $1e^{-3}$, respectively, the StepLR is regarded as the learning scheduler, and CrossEntroy is considered as the loss function.

3.4.   Performances on the MTSD3C6K Dataset

To validate the performance of the proposed ARA-net, five classical state-of-the-art deep learning-based networks, i.e., DenseNet-121 ^[28], MobileNetV3 ^[29], EfficientNet-b0 ^[30], GhostNet ^[31], and VGG19 ^[32], and two vision transformers, including Visformer ^[33] and HRNet ^[34], are tested on the MTSD3C6K dataset, where the accuracy, precision, recall, F1-score, and kappa score are used as evaluation metrics. Furthermore, the model efficiency is also explored in this section, which can be seen as follows.

First, these five classical networks are conducted on the MTSD3C6K dataset ^[35]. Then, the vision-based transformer networks are tested, and the results are recorded in Table 1 and the confusion matrix in Figure 5. We can note that the proposed ARA-Net can achieve the best performance on defect detection, with the 97.05, 97.07, 97.12, 97.08%, and 0.858 of accuracy, precision, recall, F1-score, and kappa, respectively, showing that it can be used as a valuable tool for defect detection for magnetic tiles.

Table 1.  The quantitative comparisons on the MTSD3C6K dataset.

Model	Accuracy (%)	Precision (%)	Recall (%)	F1-score (%)	Kappa	Params (M)	FPS (frames/s)
DenseNet-121	88.94	98.32	90.12	94.03	0.663	6.867	35.4
MobileNetV3	89.82	96.73	92.72	94.68	0.661	0.561	115.4
EfficientNet-b0	92.03	95.87	96.86	96.36	0.701	3.969	74.1
Visformer	92.92	97.27	94.22	95.72	0.758	39.186	100.2
HRNet	93.81	97.83	94.78	96.28	0.788	11.143	59.1
GhostNet	93.81	98.89	94.81	96.81	0.793	3.899	56.4
VGG19	95.13	98.92	95.33	97.09	0.836	128.783	196.7
ARANet	97.05	97.07	97.12	97.08	0.858	0.133	37.3

---

 | Show Table

DownLoad: CSV

In addition, two metrics, i.e., Params and FPS, are utilized to evaluate the model efficiency, and the results are listed in Table 1. It can be seen that the proposed ARC-Net can achieve fast calculation speed, achieving an FPS of 37.3 with small capacity parameters (Params) of 0.133 M. Furthermore, compared with the second-best VGG19 method, although the proposed ARA-Net is slower than the VGG19, it can obtain significant improvements on Params, demonstrating the ARA-Net can be used as an effective tool for defect detection on the magnetic tile surface.

3.5.   Ablation studies

In this section, we perform a series of ablation experiments to validate the effect of the Adaptive Rotation Convolution (ARC) module, the Rotation Region Attention (RAA) module, learning rate, training epoch, and network optimizer.

3.5.1.   Impact of adaptive rotation convolution module

In section 2.1, the Adaptive Rotation Convolution (ARC) module is devised to capture the random defects on the magnetic tile surface. To verify the effectiveness of the proposed ARC module, several ablation studies are performed on the MTSD3C6K dataset, and the results are listed in Table 2. No.1 is the performance of the baseline without ARC and RRA modules, and it can achieve the performance of 92.47% and 0.787 on accuracy and kappa score, respectively. After adding the ARC module to the baseline, it can obtain better results on defect detection, with 94.51% accuracy and 0.812 kappa score, demonstrating it is beneficial for defect detection. Moreover, from the visualized CAM feature maps in Figure 4, we can observe that these results (A and B) show that the baseline+ARC can pay more attention to the random defects while some defects are ignored by the baseline. These excellent results show that the proposed ARC module can assist the network in capturing more random defects on the magnetic tile surface by multi-view learning.

Table 2.  Ablation studies of proposed modules for defect detection on the magnetic tile surface.

No.	Baseline	ARC	RAA	Accuracy (%)	Kappa
1	$\checkmark$			92.47	0.787
2	$\checkmark$	$\checkmark$		94.51	0.812
3	$\checkmark$		$\checkmark$	94.77	0.826
4	$\checkmark$	$\checkmark$	$\checkmark$	97.05	0.858

---

 | Show Table

DownLoad: CSV

Figure 4.  Examples of class activation mapping (CAM) for defect detection on the magnetic tile. The row denotes the results of the ablation studies of the proposed modules: A. raw images, B. baseline, C. baseline + ARC, D. baseline + ARC + RAA. Besides, (b), (d)–(h) represent the random defects while (a), (c), (f) and (g) denote the tiny defects hidden by background.

DownLoad: Full-Size Img PowerPoint

3.5.2.   Impact of rotation region attention module

Similarly, the proposed Rotation Region Attention (RRA) module, described in Section 2.2, is employed to focus on the defects, especially the tiny defects on the magnetic tile surface. To validate the impact of the proposed RRA module, many experiments are carried out on the magnetic tile surface, and the results are given in Table 2. It can be seen that the performance of the network is increased after adding the RRA module, achieving improvements of 2.28 and 3.87% on accuracy and kappa, respectively, compared to the baseline + ARC, indicating that it is effective for defect detection. In addition, the visualized CAM feature maps are shown in Figure 4. It can be found from C of Figure 4 that adding the ARC module can help the network to expand the perception field of vision to locate random defects on the input images, but shows low sensitivity on the small defects. Furthermore, the RRA module can locate small defects from the magnetic tile surface. Specifically, as can be seen from (a), (c), (f) and (g) in Figure 4, adding the RAA module can further help the network focus on the attention of small defects on the magnetic tile, compared D with C. This demonstrates that the RAA module can weaken the disturbance of the background when detecting defects on the magnetic tile surface.

Moreover, we also have performed some experiments to validate the effectiveness of the proposed modules by integrating separately single modules into the network. The results are listed in the second row and third one of Table 2. These results also demonstrate the proposed modules can improve the performance of the network for defect detection on the magnetic tile surface.

3.5.3.   Impact of learning rate

To explore the effect of the learning rate, four different learning rates, i.e., 0.01, 0.001, 0.0005, and 0.0001 are used in the experiments on the magnetic tile defect dataset, and the results are given in Table 3. It can be seen that the proposed ARA-Net can achieve the best performance when the learning rate is set to 0.001. Also, we can observe from Figure 6 that the training and validation accuracies are the best while the losses are the lower when the learning rate is equal to 0.001. Therefore, the learning rate is set to 0.001 in the proposed ARA-Net during experiments.

Table 3.  The hyper-parameter tuning results tested on the MTSD3C6K dataset.

Name	Value	Accuracy (%)	Precision (%)	Recall (%)	F1-score (%)	Kappa
Lr	0.01	95.22	96.13	94.89	95.51	0.813
	0.001	97.05	97.07	97.12	97.08	0.858
	0.0005	92.38	93.88	92.41	93.14	0.746
	0.0001	83.97	84.56	82.91	83.73	0.633
Epoch	30	94.92	95.47	95.69	95.57	0.813
	50	95.81	96.93	96.14	96.53	0.828
	80	95.93	96.14	96.22	96.18	0.837
	100	97.05	97.07	97.12	97.08	0.858
Optimizer	Adagrad	91.15	92.08	93.11	92.59	0.686
	RMSprop	92.03	93.34	94.77	94.05	0.737
	Adam	93.45	94.61	95.13	94.87	0.769
	SGD	97.05	97.07	97.12	97.08	0.858

---

 | Show Table

DownLoad: CSV

Figure 5.  The evolutions of training and validation accuracies with epoch and the confusion matrix on the magnetic tile defect.

DownLoad: Full-Size Img PowerPoint

Figure 6.  The accuracy and loss of different learning rates during training and validation on the magnetic tile defect.

DownLoad: Full-Size Img PowerPoint

3.5.4.   Impact of epoch

To verify the effect of the epoch, four different epochs, i.e., 30, 50, 80, and 100 are applied to the experiments while other parameters are the same, and the results are shown in Table 3. It can be noted that the proposed ARA-Net shows the best performance when the epoch is set to 100, achieving an accuracy of 97.05%, a precision of 97.07%, a recall of 97.12%, and the F1-score of 97.08% on the magnetic tile defect dataset. Besides, it can be found from Figures 5 and 7 that both accuracy and loss tend to be smooth with only small fluctuations after about 40 epochs, i.e., the overfitting. Similarly, the difference is small between these epochs, i.e., 30, 50, 80, and 100, but the accuracy is the highest when the epoch is set to 100. Therefore, the epoch is set to 100 in the experiments.

Figure 7.  The accuracy and loss of different epochs during training and validation on the magnetic tile defect.

DownLoad: Full-Size Img PowerPoint

3.5.5.   Impact of optimizer

To validate the effect of the optimizer, four different optimizers, i.e., Adagrad, RMSprop, Adam, and SGD are employed to optimize the proposed ARA-Net for magnetic tile defect detection, and the results are listed in Table 3. We can note from Table 3 that the proposed ARA-Net with the SGD optimizer performs better than others. Similarly, Figure 8 shows that the accuracy and loss curves of the SGD optimizer are better and smoother than those of others. For this, the SGD optimizer is used to obtain the high-performance model in the experiments.

Figure 8.  The accuracy and loss of different optimizers during training and validation on the magnetic tile defect.

DownLoad: Full-Size Img PowerPoint

4.   Conclusions

In this article, an Adaptive Rotation Attention Network (ARA-Net) is proposed for Defect Detection on the magnetic tile surface, where the Adaptive Rotation Convolution (ARC) module is devised to capture the random defects on the magnetic tile surface, and the Rotation Region Attention (RAA) module is designed to locate the small defects from the complicated backgrounds. Experiments conducted on the benchmark dataset demonstrate the effectiveness of the proposed ARA-Net, further assisting in permanent magnet motor production. However, the proposed ARA-Net is conducted on a limited magnetic tile defect dataset. In future works, it is essential to establish a larger magnetic tile dataset to promote its practical application, and it will be further validated in the actual industrial environment.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Conflict of interest

No potential conflict of interest.

Acknowledgements

This work was supported in part by the Natural Science Foundation of Hunan Province, China under Grant 2023JJ50066; in part by Guangdong Provincial Department of Education features innovative projects 2022KTSCX356; in part by Guangdong Provincial Department of Education Intelligent Robot Industry-Education Integration Innovation Platform for Higher Vocational Colleges 2020CJPT034; in part by Chenzhou Science and Technology Development Plan Project ZDYF2020161; in part by the Scientific Research Fund of Hunan Provincial Education Department 22B0812; in part by the 2021 Hunan Colleges and Universities Innovation and Entrepreneurship School-Enterprise Cooperation Base 74th; in part by the Innovation and Entrepreneurship Education Center in Ordinary Universities in 2022 70th; in part by the Chenzhou Low Carbon Intelligent Manufacturing Technology Research; in part by the Applied Characteristic Disciplines of Electronic Science and Technology of Xiangnan University XNXY20221210.