Semi-automatic fingerprint image restoration algorithm using a partial differential equation

1.   Introduction

Biometrics refers to an automated method of identifying individuals by analyzing their distinct behavioral and biological traits ^[1]. In the past decade, humans have developed various biometric techniques, including fingerprint recognition, face and facial feature analysis, personal ID numbers, signatures and more. In this era of abundant information, there has always been an urgent need for a more accurate method of identity verification. Fingerprint usage stands out as one of the leading and widely adopted verification methods due to its low cost, high resistance to spoofing and the availability of extensive data. Numerous studies have proposed more precise and efficient fingerprint recognition algorithms. However, a significant challenge in fingerprint recognition lies in handling poor and low-quality fingerprints, which may exhibit issues such as missing data, unclear boundaries and low-contrast images ^[2,3,4]. Therefore, when we recognize the information of the fingerprint image, it can be recognized using a model such as a transform-minutiae fusion-based model in ^[5]. If the image is damaged, the damaged fingerprint image can be restored using the proposed method.

Including fingerprint restoration, image inpainting has been a popular topic in image processing due to its applications in real life. There are several algorithms for inpainting, and the use of partial differential equations is one of the major areas. Halim and Kumar ^[6] proposed a fourth-order anisotropic partial differential equation (PDE) for grayscale image inpainting. A modified Cahn-Hilliard (CH) equation with a fidelity term is used for the proposed method, and the authors showed that the numerical scheme is unconditionally stable, consistent and convergent. The authors ^[7] proposed a coupled system with a nonlinear time-delay structure tensor. Through several experiments, it was confirmed that the coupled anisotropic nonlinear diffusion system is effective in image enhancement. Instead of attempting to directly authenticate the damaged fingerprint, another approach is to restore the damaged image as a preprocessing step. Fingerprint restoration falls under the category of image restoration, and specific methods suitable for restoring fingerprint images have been proposed. Shams et al. ^[8] proposed a fingerprint enhancement filter with normalization; segmentation; coherence diffusion filter; estimating orientation and frequency; log-Gabor filter and binarization steps. One major approach to fingerprint restoration is adaptive filtering, which involves progressive enhancement and feedback. In ^[9], Tu et al. illustrated a new fingerprint representation method that fits ridges using cubic Bézier curves and curve fitting after Gabor enhancement and thinning the ridges as preprocessing. Claesson et al. ^[10] proposed an improved adaptive fingerprint enhancement technique using contextual filtering. They achieved enhanced results by updating four processing blocks: Preprocessing, global analysis, local analysis and matched filtering. In their study, they compared their method with the NIST fingerprint matching method. Sutthiwichaiporn and Areekul ^[11] proposed a method that applies different schemes for regions with high and low quality. They utilized a Gaussian filter for the high-quality regions and iteratively propagated good spectra from the enhanced ridges to the low-quality regions. In their evaluation, out of $15$ fingerprint verification tests, their method demonstrated the best average equal error rate in $8$ tests. Neural network-based fingerprint restoration is also a widely used method. Convolutional neural networks (CNNs) are deep learning models that can learn filters and characteristics to differentiate various input images. Joshi et al. ^[12] proposed a context-aware fingerprint restoration model, and Figure 1 illustrates an example of this fingerprint restoration.

Figure 1.  Example of restoration fingerprint image, (a) Original image, (b) Restored image using CA-GAN ^[12].

DownLoad: Full-Size Img PowerPoint

Gao et al. ^[13] proposed a fingerprint preprocessing method using CNN for fingerprint recognition. Their approach was applied to grayscale noisy fingerprints, and the enhanced fingerprints retained most of the characteristics of the original fingerprint. Another machine learning architecture used for fingerprint restoration is the generative adversarial network (GAN), which consists of two neural networks engaged in a zero-sum game. Zhang et al. ^[14] introduced a lightweight and fully convolutional GAN architecture called FCGAN. Experimental results demonstrated over $99%$ training accuracy when using FCGAN-augmented samples, whereas classical augmentation techniques achieved $96.34%$ accuracy. In ^[15], the authors presented a GAN-type fingerprint enhancement model for restoring poor ridge structure.

In this paper, we present an automatic fingerprint image restoration technique using a partial differential equation. Many of the state-of-the-art studies have focused on image quality improvement, such as noise removal, while we develop the efficient algorithm for restoration in the case that some parts of the image are completely damaged. Specifically, we focus on the CH equation with a source term, which effectively models the phase separation in diblock copolymers. The use of diblock copolymers in this context has been studied by Ohta and Kawasaki ^[16]. Previous work by Li et al. ^[17] has explored the idea of employing the nonlocal CH equation and diblock copolymers for fingerprint restoration. However, a drawback of their approach is that the parameter values of the CH equation with a source term had to be manually adjusted for different fingerprints. This required a trial and error process for successful restoration, resulting in slow and less intuitive restoration procedures. To resolve this limitation, we propose an upgraded and improved fingerprint restoration method by introducing a step that solves a partial differential equation. By specifying the discrete boundary and wave period, the rest of the restoration process becomes automatic, as we can fix the parameter values for the CH equation with a source term. This enhancement improves the efficiency, intuitiveness, and convenience of the restoration process.

This paper is structured as follows. In Section 2, we present a computational approach for implementing the fingerprint restoration method. Section 3 consists of several numerical experiments conducted to demonstrate the effectiveness of the proposed method. Lastly, in Section 4, we offer concluding remarks.

2.   Automatic fingerprint restoration method

In this section, we present the proposed automatic fingerprint restoration method. Let us denote by $f(x, y)$ a given damaged fingerprint image on an entire domain $\Omega$ as shown in Figure 2(a). Here, $\Omega_{in} \subset\Omega$ is a local damaged fingerprint domain and $\partial \Omega_{in}$ is the boundary of $\Omega_{in}$.

Figure 2.  (a) Damaged fingerprint image $\Omega_{in}$ and (b) discretized $\Omega_{in}^{h}$.

DownLoad: Full-Size Img PowerPoint

Let us consider the following PDE ^[16,18,19]:

which is obtained by adding a source term to the original CH equation ^[20,21]. Here, $\phi(x, y, t)$ is a phase-field function at space $(x, y)$ and time $t$, $F(\phi) = (\phi^2-1)^2/4$, $\epsilon$ is an interfacial transition layer thickness related parameter, $\alpha$ represents a long-range repulsive interaction parameter, and ${\bar \phi}$ is the spatial average value of the initial $\phi(x, y, 0)$ values, which are defined as follows:

where $f_{\max}$ is the maximum and $f_{\min}$ is the minimum of $f(x, y)$. From this definition, $\bar\phi$ is zero.

For the sake of completeness of the description of the proposed numerical algorithm, we briefly review the explicit numerical solver for the CH equation with the source term ^[22]. Let $\Omega = (0, a)\times(0, b)$ and $\Omega^h = \{(x_i, y_j)| x_{i} = (i-1/2)h, \; y_{j} = (j-1/2)h, \; i = 1, \ldots, N_x, \; j = 1, \ldots, N_y \}$ be its discrete domain, where $N_x$ and $N_y$ are the numbers of the pixels in $x$- and $y$-directions, respectively; and $h = a/N_x = b/N_y$ is the spatial grid size. Let $\Omega_{in}^h$ and $\partial \Omega_{in}^h$ be the discrete domain and boundary, respectively, as shown in Figure 2(b). Open and filled circles represent $\Omega_{in}^h$ and $\partial\Omega_{in}^h$, respectively. Let $\phi_{ij}^n = \phi(x_i, y_j, n \Delta t)$, where $\Delta t$ is the time step. We consider a stable numerical method ^[23,24]:

where $\Delta_d \phi_{ij} = (\phi_{i-1, j}+\phi_{i+i, j}-4\phi_{ij}+\phi_{i, j-1}+\phi_{i, j+1})/h^2$. If use a Saul'yev-type method, we have

where $r = 1/ \Delta t +4/h^2 +10\epsilon^2/h^4+\alpha$. Here, we use the Dirichlet boundary condition. To compute the solution on the boundary points in $\Omega_{in}^h$, we use the value of black-filled circles near the damaged area, which contains the information of the original lamella-type patterns. More details can be found in ^[22].

We present an automatic fingerprint restoration algorithm for solving Eqs (2.1) and (2.2) under the Dirichlet boundary condition. In ^[17], parameters such as the interaction parameter $\alpha$, model parameter $\epsilon$ and other parameters $h$ and $\Delta t$ were manually selected to restore the damaged fingerprint. However, our proposed algorithm automatically determines the appropriate parameter values by analyzing the fingerprint ridges around the damaged area.

The main idea of our proposed algorithm is to find an appropriate scaled spatial step size $\bar{h}$, which we call a scaling factor, using the information very close to the damaged area. After we fix values of $\alpha$, $\epsilon$, and $\Delta t$ closely related to the period of given patterns, we can resize the damaged fingerprint image with $\bar{h}$. For the automatic restoration method, we propose an algorithm for estimating the period of fingerprint patterns in two steps: First, using the governing equation, we find an equilibrium wave with a random perturbation, and call it a reference; second, we estimate the period of the target patterns using the reference.

Let $K$ be the wavelength of the equilibrium wave, $L$ be the length of the random perturbation, and $H$ be the spatial grid size at equilibrium. We set a random perturbation in $x$-direction with $0.2$ amplitude, $0$ average, and the total wave length $L = 100$, and then solve the governing equation (2.4) with $H = 1$ to obtain the equilibrium wave. For estimating the target wave of the damaged fingerprint image, a schematic of the algorithm for estimating the period is illustrated in Figure 3. We take a line perpendicular to the pattern around the damaged area, and then interpolate data along the line. The interpolated data is then rescaled from $-1$ to $1$. Using the interpolation method, we obtain a set of points denoted as $P = \left\{p_i = (X_i, Y_i)\, \vert \, \phi(X_i, Y_i) = 0, \; i = 1, \dots, m\right\}$, where $m$ is the number of points defined as $\vert P \vert$. We define the wavelength as $k = m-1$, the length $(X_m-X_1)$ of the red line as $l$, and the spatial grid size and scaling factor as $\bar{h}$. Therefore, we can obtain the suitable scaling factor as follows:

Figure 3.  Schematic illustration for estimating the period of the fingerprint patterns. The original image was reprinted from ^[9].

DownLoad: Full-Size Img PowerPoint

3.   Computational tests

It should be noted that we stop the evolution and consider the numerical results as equilibrium solutions when the maximum error $||\phi^{n+1}-\phi^{n} ||_{\infty}$ is smaller than the tolerance $tol$. Unless otherwise stated, we will use the Dirichlet boundary condition for the boundaries.

3.1.   Fingerprint restoration with different $\alpha$ values

We used the following parameters: $h = 1$, $\Delta t = 0.2h^2$, $\epsilon = 1.1$, and $tol = 1.0e$-$7$, with a maximum iteration of $10000$. Figure 4 shows an intact fingerprint, a damaged fingerprint, and restorations of a damaged fingerprint with $\alpha = 0.01, \; 0.1$, and $0.5$. The restoration image appears coarse when $\alpha$ is small and fine when $\alpha$ is large. The value of $\alpha$ is the parameter that affects the thickness of the fingerprint patterns. In this test, it appears that the restored fingerprint image is the most appropriate when $\alpha = 0.1$ among the three different $\alpha$ values. The best $\alpha$ value is found through trial and error so that the pattern formed by the ridges and valleys around the damaged part of the given image could be smoothly connected to the restored pattern. Therefore, instead of trial and error manually taking parameters, our proposed method is to automatically fit the significant parameter, $\alpha$, in fingerprint restoration by estimating the period of fingerprint patterns near the damaged area.

Figure 4.  Results of fingerprint restoration with different $\alpha$ values. (a) Intact fingerprint, (b) damaged fingerprint, and restoration with (c) $\alpha = 0.01$, (d) $\alpha = 0.1$, (e) $\alpha = 0.5$.

DownLoad: Full-Size Img PowerPoint

3.2.   Fingerprint restoration using semi-automatic algorithm

In this section, we apply the semi-automatic algorithm to restore damaged fingerprint images. Using the proposed algorithm, we can obtain a suitable spatial grid size $h$ based on the information near the damaged area. Specifically, because there is a close relation among $h$, $\epsilon$ and $\alpha$, our goal is to obtain the value of $h$ given $\epsilon$ and $\alpha$. In our experiments, we set $\Delta t = 0.1$, $\epsilon = 1.2$, $\alpha = 0.1$, and $tol = 1.0e$-$4$.

Figure 5 illustrates the process of restoring a damaged fingerprint image. We use the same intact image as shown in Figure 4(a). In Figure 5(a), the damaged area is highlighted by a red circle. To estimate the period of the pattern, we take a line near the damaged area and compare it with the equilibrium wave obtained from random perturbation. The purpose of finding the equilibrium wave is to use it as a reference for estimating the period. In this test, the scaled spatial grid size is $h = 0.9741$ and the scaled domain is $\bar{\Omega} = (0, ah)\times(0, bh)$. Using these parameters, we compute Eq (2.4), and the final time is reached at $t = 7585\Delta t$. The temporal evolution is shown in Figure 5(b)–5(d).

Figure 5.  From top to bottom, fingerprint restoration and the profiles of the numerical solution with a scaling factor $h = 0.9741$: (a) damaged image and (b)–(d) temporal evolution.

DownLoad: Full-Size Img PowerPoint

The original CH equation does not satisfy the maximum principle, which means its solution $\phi$ may become larger than $1$ or smaller than $-1$ at some point $(x, y, t)$. However, we use the CH equation with an added source term that is the $\alpha$ term on the right-hand side in Eq (2.1). It is observed that the $\alpha$ term stabilizes the numerical solution inside the damaged area between $-1$ and $1$ from the bottom row in Figures 5 and 6. On the other hand, When $\alpha = 0$, some solutions could be smaller than $-1$ as shown in Figure 6.

Figure 6.  Maximums and minimums of numerical solution inside the damaged area (red circle in Figure 5) according to $\alpha$.

DownLoad: Full-Size Img PowerPoint

Figure 7 shows the temporal evolution with different radii of the damaged area in the same image. From top to bottom in Figure 7, the radius of damaged area is $20$, $40$, and $60$, respectively. We fix the final time $T = 7000\Delta t$ to compare the results and their computational costs according to the radii of the damaged area. We finally compare the CPU times (in seconds) of the computations. Here, we use MATLAB R2022b on a computer with an Intel Core i9-12900K CPU at $3.19$ GHz with $16$ GB RAM.

Figure 7.  From top to bottom, restoration results of the fingerprint images with a circular damaged area with each radius of (a) 20, (b) 40 and (c) 60, respectively. Each time is denoted below each figure.

DownLoad: Full-Size Img PowerPoint

Total CPU time(s) of numerical tests of Figure 7 (see Table 1).

Table 1.  Total CPU time(s) of numerical tests of Figure 7.

Radius	Number of Damaged pixels	CPU time (s)
20	1264	6.48
40	5024	21.51
60	11302	46.58

---

 | Show Table

DownLoad: CSV

Figure 8 shows the temporal evolution with the same $\epsilon$ and $\alpha$ but different $h$ values. We use a different intact image, and Figure 8(a), (b) illustrate the damaged area of the given image represented as a red circle. We solve the discrete governing equation with the initial image, and Figure 8(c), (d) show the temporal evolution until the final time. In this test, the scaled spatial grid size is $h = 1.2565$.

Figure 8.  Fingerprint restoration with a scaling factor $h = 1.2565$: (a) intact image ^[9] and (b) damaged image, and (c)–(d) temporal evolution.

DownLoad: Full-Size Img PowerPoint

Figure 9 displays the temporal evolution of damaged fingerprint images whose numbers of damaged pixels are the same, but each location is different. According to the proposed method, the scaling factors are $h = 1.2710$, $1.3199$, $1.2648$, $1.2382$ from top to bottom.

Figure 9.  Temporal evolution of fingerprint images damaged at different locations. Each scaling factor $h$ is (a) $1.2710$, (b) $1.3199$, (c) $1.2648$ and (d) $1.2382$ from top to bottom. Each time is denoted below each figure.

DownLoad: Full-Size Img PowerPoint

4.   Conclusions

In summary, this research introduced a new approach for restoring damaged fingerprint images using a semi-automatic method based on a partial differential equation derived from the Cahn-Hilliard (CH) equation with a source term. The algorithm reduces user involvement by automatically calculating the required model and numerical parameters, resulting in a streamlined restoration process. Computational simulations on damaged fingerprint samples demonstrated the algorithm's exceptional ability to accurately restore fingerprints. The proposed technique is promising for improving the reliability and security of fingerprint-based biometric authentication systems.

Use of AI tools declaration

The authors have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

The authors express their gratitude to the reviewers for their valuable and constructive input during the revision of this article.

The first author (C. Lee) was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2022R1C1C2003896).

Conflict of interest

The authors declare there is no conflicts of interest.