Improved FCM algorithm for fisheye image cluster analysis for tree height calculation

1.   Introduction

The height of standing trees is one of the basic indicators of forest inventory and forestry development, which is widely used to predict important tree properties, but it cannot be directly measured in a non-destructive manner ^[1]. The height of standing trees precision measurement is not only used in forest resource survey research ^[2], but also can be used to determine which roads are most vulnerable to damage by fallen trees ^[3]. The study of tree height is also of great significance for urban road planning and control of air pollution ^[4]. In large-scale forest volume or biomass estimation and carbon accounting, tree height can estimate forest volume and large-scale calculation of forest biomass ^[5]. However, the height of standing trees is not only hard to measure directly but also the environmental factors increases the measurement difficulty.

The current tool used to measure the height of standing trees is the ultrasonic rangefinder (Vertex Ⅲ & Transponder T3). Its advantages lie in its portability and real-time access to data, but it requires the operator to find the highest point of trees, so it is influenced by human factors and wind speed that makes a big difference in each measurement. Therefore, the measurement of the height of standing trees is still a problem that experts and scholars are trying to improve.

Cabo et al. (2018) calculated the height of the tree by creating a highly standardized point cloud version, denoising and clustering the points of each tree, and total tree height (TH) RMSE ranged from 0.3 to 0.7 m ^[6]. Wang et al. ^[1] analyzed the reliability and robustness of the airborne laser scanning (ALS) and terrestrial laser scanning (TLS) tree height observations. The statistical results show that the TLS tree height is below 15–20 m, it is reliable in measurement. However, due to the limited visibility of the upper part of the canopy, the higher the tree, the lower the accuracy of the tree height measurement based on TLS, and both ALS and TLS are expensive and inconvenient to carry. Using drone equipment to measure is more expensive and has poor endurance. From the point of view of economy and accuracy, the measuring method based on a digital image has obvious advantages. With the development of photography and computer vision technology, photogrammetry technology has been greatly developed. The monocular vision measurement of the ordinary camera has the advantages of a simple camera calibration process, and fewer parameters required for calculation. However, the ordinary camera has a small angle of view, which requires a long shooting distance when measuring objects in a large scene. When horizontal shooting is not used, it is required to measure the included angle between the camera projection and the optical axis and the vertical elevation angle of the camera. The accuracy requirements are high and the measurement error is large, and the angle information is difficult to measure ^[7]. One of the effective methods to solve the problem of perspective is to use binocular vision measurement to extract and match the feature points of two images collected from the measured tree, and reconstruct the three-dimensional pixel points. However, there are some problems such as complex measurement steps, difficult matching of feature points, various algorithms required, and lengthy operation ^[8]. This defect of ordinary cameras is more obvious when collecting images of high trees. The fisheye lens is a super wide-angle lens, which provides a very wide viewing angle, which can produce wide-angle panoramic or hemispherical images ^[9], and can shoot objects in large scenes at a very close distance. Fisheye cameras have been increasingly used in photogrammetric tasks, such as large viewing angle monitoring systems ^[10], target detection, and positioning ^[11] digital image measurement ^[12]. But for fisheye cameras, the smaller the focal length, the more serious the distortion, and excessive distortion will cause image distortion, therefore, it is important to choose a suitable focal length.

Usually, the measurement method of vision-based technology should identify the object being measured in the image, then obtain coordinates of the extreme points through image processing technology, and brings them into the mathematical model to calculate the measured value of the object. In this process, the extraction of extreme points is usually done using clustering algorithms. However, the accuracy of the image clustering algorithm used depends on the compression of the clustered samples, and it takes a long time to execute a cluster (the time consumption here is the relevant quantization interval) ^[13], the clustering effect of different clustering algorithms in tree segmentation is also unstable. The Fuzzy C-Means (FCM) algorithm is a classic algorithm in fuzzy cluster analysis ^[14]. The FCM algorithm was first proposed by Dunn. It is an unsupervised algorithm that does not require manual intervention ^[15]. It uses the minimized objective function to achieve the division of the target area. FCM has the advantages of being simple, fast, and easy to implement, and has been applied in many fields ADDIN EN.CITE.DATA ^[16,17,18]. However, FCM also has some problems, mainly including two aspects. On the one hand, it is sensitive to noise in image segmentation, and on the other hand, it is sensitive to the initial cluster center.

The FCM algorithm is sensitive to noise because of the lack of spatial information of pixels in the local image segmentation process. To improve the ability of the FCM algorithm to suppress noise in image segmentation, people have proposed a large number of improved clustering algorithms that incorporate local spatial information into the objective function in recent years ADDIN EN.CITE.DATA ^[19,20,21]. These improved algorithms are roughly divided into two types. The first is to use the neighborhood information of the central pixel of a fixed size window to improve the image segmentation effect. For example, in 2010, Krinidis introduced the concept of fuzzy factor and applied it to FCM to determine the weight, and proposed a fuzzy local information C-means clustering algorithm ^[22]. The advantage of these algorithms is that in addition to the FCM algorithm and the fuzzy local information C-means clustering algorithm, the neighborhood information can be calculated in advance, which reduces the computational complexity. However, the neighborhood window of fixed size and shape cannot meet the robustness requirements of image segmentation. Therefore, the second improved FCM algorithm that uses adaptive neighborhood information instead of fixed-size windows is proposed. For example, in 2016, Bai et al. proposed an improved FCM segmentation method based on spatial information ^[23], which adds to the segmentation target Non-local spatial information, while using Markov random field to refine the local spatial constraints. In 2017, Zhang et al. proposed an adaptive FCM method that combines local spatial and gray information constraints ^[24], which uses The fuzzy local similarity measurement method based on the pixel space attraction model does not require any experimentally set parameters and adaptively determines the weighting factor of the adjacent pixel effect. Since the adaptive neighborhood information is consistent with the structural information of the real image, the second improved algorithm is more robust to noisy images and the segmentation effect is better than the first improved algorithm.

Although the above-mentioned improved FCM algorithm considers the neighborhood information of the image, it ignores the neighborhood information that helps to improve the classification effect. Hidden Markov Random Fields (HMRF) is a commonly used algorithm to solve this problem ^[25]. Since HMRF considers the previous state of the current membership degree, it obtains better results than FCM for image segmentation. Based on this theory, in 2014 Zhang et al. merged the local spatial membership information into the FCM objective function ^[26], and got a better image segmentation effect. Although these clustering algorithms based on Hidden Markov Random Fields effectively improve the effect of image segmentation, the calculation is more complicated because the neighborhood information provided by the original image and the membership degree of the previous state is calculated for each iteration. The current FCM algorithm still has shortcomings, such as high computational complexity and sensitivity to the initial cluster center when using the FCM algorithm for color image segmentation.

In this paper, two improvements are made to the FCM algorithm. One is for the high computational complexity, and an improved watershed algorithm is proposed to perform super-pixel processing on the image. The second is to improve the salp swarm algorithm to optimize the initial clustering center, the improved algorithm is applied to the non-contact standing trees height measurement. In this study, mobile phones equipped with fish-eye lenses have been used to capture images to shorten the shooting distance and improve portability. An effective measurement mathematical model is constructed to reduce the impact of fisheye lens distortion. An improved hybrid algorithm is used to identify feature points to improve accuracy and efficiency.

2.   Improved algorithms

2.1.   Improved watershed algorithm to reduce calculation time of FCM algorithm

Bezdek presented the FCM algorithm based on the fuzzy set theory. Regarding the FCM algorithm as an optimization problem that minimizes the objective function, its basic structure is as follows: let the sample data set be $X = \left\{ {{x_1}, {x_2}, ..., {x_n}} \right\}$, $\forall {x_i} = ({x_{i1}}, {x_{i2}}, ..., {x_{id}}) \in {R^d}$, ${x_i}$ is a $d$ dimensional vector, that is, each data element contains $d$ attributes, need to divide the data set $X$ into $c$ categories ($2 \leqslant c \leqslant n$). The objective function of FCM algorithm can be expressed as:

In the formula, $U = {[{u_{ij}}]_{c \times n}}$ represents the membership matrix, ${u_{ij}}$ represents the membership degree of sample $j$ data belonging to category $i$, $V = {[{v_i}]_{c \times d}}$ represents the cluster center matrix, ${v_i}$ represents the center of the $i$ category. $m \in [1, + \infty)$ is the fuzzy index, Generally, $m = 2$, ${d_{ij}} = \left\| {{x_j} - {v_i}} \right\|$ represents the Euclidean distance from sample ${x_j}$ to the center ${v_i}$ of class $i$. The iterative update formula for solving fuzzy membership degree ${u_{ij}}$ and cluster center ${v_i}$ using Lagrange Multiplier Method is:

${u_{ij}}$ and ${v_i}$ are updated through repeated iterative calculations of Eqs (2) and (3) until the objective function ${J_m} = (U, V)$ converges.

The FCM algorithm actually uses the gradient descent method to find the optimal solution along the direction in which the objective function $J$ gradually decreases. In essence, it is a local optimization search algorithm. The algorithm is simple, fast and easy to implement, but there are problems of high computational complexity and sensitivity to initial clustering centers in image segmentation. This article improves the algorithm from these two aspects.

The traditional FCM algorithm needs to calculate the distance between each pixel and the cluster center, which leads to high computational complexity and time-consuming when processing high-resolution images. Compared with pixel clustering, the enhanced FCM algorithm was proposed by ^[27] solves this problem by clustering gray levels. By applying the histogram to the objective function, the repeated calculation of distance is eliminated, thereby effectively reducing the amount of calculation and reducing the calculation time. However, for color image segmentation, it is difficult to apply this idea, because the number of different colors is usually close to the number of pixels in a color image. To solve this problem, this article calculates the histogram of the color image based on the corresponding super-pixel image. We will use the average value of all pixels in a region to replace these pixels to reduce the number of different colors in the original color image. There are only a few different color characteristics in super-pixel images, and the histogram of super-pixel images is easy to calculate. Because super-pixel images can provide better local spatial information, super-pixel methods, such as mean-shift ^[28], simple linear iterative clustering (SLIC) ^[29], watershed transform (WT), are generally considered to be used to improve clustering. The pre-segmentation algorithm that produces the segmentation result of the class algorithm, compared with SLIC, the irregular super-pixel area produced by mean-shift and WT is better than the polygonal area obtained by SLIC. Although mean-shift can provide better super-pixel results, it is very sensitive to parameter values, and mean-shift has a higher computational complexity than WT. Therefore, there is a need for a fast super-pixel algorithm that can provide better pre-segmentation results and requires less time than mean-shift. In this paper, a new and improved WT is used to generate super-pixel images ^[30]. To improve the shortcomings of WT, a morphological gradient reconstruction (MGR) algorithm is proposed to improve WT. MGR is an algorithm to overcome over-segmentation, which can eliminate noise and useless gradient details while preserving the details of the target contour. The expression of MGR is:

Among them, ${R^\varepsilon }$ and ${R^\delta }$ represent morphological corrosion and expansion reconstruction respectively, $f$ is the original image(the reference image), $g$ is the gradient image, $\varepsilon$ is the corrosion operation, and $\delta$ is the expansion operation. Corrosion reconstruction requires $g \geqslant f$, expansion reconstruction requires $g \leqslant f$, $\varepsilon _g^{(1)}(g) = \varepsilon (g) \vee f$, $\varepsilon _g^{(i)}(f) = \varepsilon ({\varepsilon ^{(i - 1)}}(g)) \vee f$, $\varepsilon _f^{(1)}(g) = \delta (g) \wedge f$, $\delta _g^{(i)}(f) = \delta ({\delta ^{(i - 1)}}(g)) \wedge f$. Among them, $\wedge$ represents the maximum value, and $\vee$ represents the minimum value. Morphological opening and closing operations are morphological operations just like corrosion expansion. Because morphological opening and closing operations have stronger denoising and feature extraction capabilities than corrosion expansion, they are more popular than corrosion expansion. The open reconstruction of morphology is represented by ${R^o}$, and the closed reconstruction is represented by ${R^c}$. The expression is:

In the corrosion reconstruction $g = {\varepsilon _B}(f)$, in the expansion reconstruction $g = {\varepsilon _B}(f)$, $B$ is a structural element, over-segmentation can be achieved by morphological opening and closing reconstruction to eliminate the regional minimum in the gradient image. The structural element is defined as a disk, $r$ is its radius, the larger the $r$, the fewer the partitions. Also, the size of the structure element will affect the accuracy of the segmentation. If the structure element is too large, the segmentation will be insufficient. If the structure element is too small, it will cause over-segmentation. Therefore, it is not easy to use MGR to obtain super-pixel images with few segmented areas and clear outlines. In order to obtain super-pixel images with good effects, a suitable area measurement standard is required. However, it is difficult to choose a suitable area measurement standard for different images. To solve this problem, different structural elements are used to reconstruct $g$, and the reconstructed $g$ is merged to eliminate the dependence of the segmentation results on structural elements. Therefore, a modified morphological gradient reconstruction (MMGR) operation is proposed, denoted by ${R^{MC}}$, which is expressed as:

${r_1}$ is the minimum value of $r$, ${r_2}$ is the maximum value of $r$, ${r_1} \leqslant r \leqslant {r_2}, {r_1}, {r_2} \in {N^ + }, g \leqslant f$. ${R^{MC}}$ uses different structural elements to reconstruct $g$. By calculating the maximum value of reconstructed $g$, a gradient image that can retain important edge details and eliminate most of the useless local minima at the same time is obtained.

MMGR has two parameters ${r_1}$ and ${r_2}$, ${r_1}$ controls the size of the smallest area, and ${r_2}$ controls the size of the largest area. In the process of processing the image, ${r_1}$ is too small to cause many small areas, and vice versa, the boundary accuracy is low. As shown in Figure 1, when ${r_1} = 1$, the image has higher contour accuracy, but there are some small areas. When ${r_1} = 3$ or ${r_1} = 5$, the image has higher contour accuracy and small areas are eliminated. As ${r_1}$ increases, the image will be distorted gradually, when ${r_1} = 10$, the image is distorted. ${r_2}$ controls the size of the largest area, the larger ${r_2}$ the better the super-pixel image, as shown in Figure 2. However, when ${r_2}$ is greater than the threshold, the super-pixel image remains unchanged, the threshold in Figure 2 is 15. The super-pixel image is converged by increasing the value of ${r_2}$. When ${r_2}$ is greater than the threshold, MMGR is not sensitive to changes in ${r_2}$. Table 1 shows the comparison of the number of super-pixel regions between MGR-WT and MMGR-WT.

It can be seen from Table 1 that ${r_2}$ can be a variable, but it is difficult to set a different ${r_2}$ value for each image, in practical applications, ${r_2}$ is adaptive, the improved morphological gradient reconstruction does not need it, so the minimum error threshold $\eta$ is used instead of ${r_2}$, and its expression is:

where ${r_2}$ can be replaced by $\eta$, because ${r_2}$ has a different value for each image in the BSDS (Berkeley Segmentation Dataset), but a fixed $\eta$ value can be used for all images in the data set. If $\eta$ is too large, ${r_2}$ will be small and the error will be large, on the contrary, the error is small and ${r_2}$ is large, which will lead to a large computational burden for the improved morphological gradient reconstruction. Therefore, it is important to choose an appropriate $\eta$ value for the data set. The improved morphological gradient reconstruction of 10 images in the Berkeley image segmentation standard segmentation database can obtain different ${r_2}$ values according to a fixed $\eta$ value ^[31], as shown in Table 2.

Table 2 shows that when $\eta$ is continuously reduced, when $\eta$ is less than or equal to ${10^{ - 4}}$, ${r_2}$ remains unchanged. Through experiments, it is suggested that the selection range of ${r_1}$ in practical applications is $3 \leqslant {r_1} \leqslant 5$, and $\eta = {10^{ - 4}}$. At this time, it can reduce the error while preventing the calculation burden of morphological gradient reconstruction. In this paper, select ${r_1} = 3$, $\eta = {10^{ - 4}}$.

The steps of using the improved watershed algorithm to preprocess the fisheye image of trees are to initialize the required parameters, and then perform Gaussian filtering on the image to reduce the interference of noise on the image, and then use the improved morphological gradient reconstruction to process the Gaussian filtering. Then, perform a watershed operation on the tree fisheye image to obtain the super-pixel image, and finally extract the color image histogram of the super-pixel image. The image preprocessing steps are shown in Figure 3.

Based on MMGR-WT super-pixel images, the objective function of the improved FCM algorithm is proposed:

In this equation, $l$ is the color level, ${\rm{1}} \leqslant l \leqslant q$, $q$ is the number of regions of the image, $l \in {N^ + }$, $q \in {N^ + }$, ${S_l}$ is the number of pixels of ${R_l}$ in the $l$, and ${x_p}$ is the color pixel in the $l$ of the MMGR-WT processed super-pixel image. Different from the traditional FCM objective function, the new objective function introduces histogram information, because each color pixel in the original image is replaced by the average value of the color pixel in the corresponding area of the super-pixel image, so the number of regions in the super-pixel image Same as the number of gradations. Using the Lagrangian multiplier method, the optimization problem can be transformed into an unconstrained optimization problem of minimizing the objective function, and the solutions corresponding to ${v_k}$ and ${u_{kl}}$ are obtained as:

2.2.   Optimizing the initial clustering centers of Fuzzy c-means algorithm based on salp swarm algorithm

The salp swarm algorithm (SSA) is used to simulate the process of salvia squirts cruising and foraging in the ocean. Individuals are divided into leaders and followers. The leader is at the front end of the salvia chain, the rest are considered followers. SSA is a meta-heuristic algorithm, which has a better global optimization effect than some traditional optimization algorithms. Although the SSA has obvious global optimization effects, it is easy to fall into local optimization in terms of local optimization problems. In this paper, adaptive differential evolution (ADE) with strong local search capability is integrated with the SSA algorithm to improve the accuracy and stability of the algorithm.

Differential evolution (DE) is a simple and powerful algorithm, there are three main operators in the algorithm, "mutation", "crossover" and "selection".

a) Mutation operator

The mutation operation of DE algorithm is defined as:

$m_i^{t + 1}$ is the mutant individual in the $t + 1$ iteration, $x_{{r_1}}^t$, $x_{{r_2}}^t$, and $x_{{r_3}}^t$respectively represent three different individuals in the population(${r_1} \ne {r_2} \ne {r_3}$), $SF$ represents the scale factor, which is a constant.

b) Crossover operator

In the mutation process, in order to increase the diversity of the population, select test individuals $c_i^{t + 1}$ from existing individuals $x_i^t$ or mutant individuals $m_i^{t + 1}$. The crossover operation of the $DE$ algorithm is mathematically defined as:

$Rand$ is a random number between $[0, 1]$, and $CR$ represents the probability of crossover.

c) Selection operator

In the selection process, the test individual $c_i^{t + 1}$ and the current individual $x_i^t$ are compared in order to obtain the $t + 1$ generation individual. For the minimization problem, the selection operation is defined as:

In the above formula, $f$ is the fitness function.

According to the above expression, $SF$ and $CR$ are two important parameters in $DE$, and the choice of their values will affect the optimization effect. However, in the $DE$ algorithm, its values are all constants, which cannot be well adapted to various problems, especially complex high-dimensional problems. Therefore, adaptive control parameters are introduced ^[32], and the improved algorithm is called adaptive differential evolution algorithm (ADE). The adaptive control parameters $SF$ and $CR$ are respectively expressed as:

In the formula, $ran{d_1}$, $ran{d_2}$, $ran{d_3}$, $ran{d_4}$ are random numbers between $[0, 1]$, ${\tau _1}$ and ${\tau _2}$ represent the probability of conversion, $S{F_l}$ and $S{F_u}$ are boundary scaling factors. In this design, let ${\tau _1} = {\tau _2} = 0.1$, The initial value of $SF$ is set to 0.5, and the initial value of $CR$ is 0.9.

Combining the SSA with the adaptive differential evolution algorithm to form the salp swarm algorithm adaptive differential evolution (SSA-ADE), the purpose is to improve the search accuracy, avoid the population falling into a local extreme value, and maintain the population diversity in the later iterations. In each population iteration, calculate the average fitness value of $f$, and the individual fitness value of $fi$, when $fi < f$, $ADE$ is used for optimization, and when $fi \geqslant f$, $SSA$ is used for optimization. The specific steps of the SSA-ADE algorithm are shown in Figure 4:

3.   Materials and methods

3.1.   Fish-eye camera imaging model

The fisheye lens imaging model can determine the coordinate transformation relationship between the world coordinate system and the image pixel coordinate system, and project the target points in the world coordinate system into the image pixel coordinate system. The fisheye lens imaging model is shown in Figure 5.

The coordinate system ${O_{\rm{w}}}{X_w}{Y_w}{Z_w}$ in Figure 5 is the world coordinate system, which is a reference coordinate system selected in an appropriate space, and $P\left({{x_w}, {y_w}, {z_w}} \right)$ represents the coordinates of the target point $P$ in the world coordinate system. The coordinate system $OXYZ$ is the reference coordinate system of the fisheye lens. The coordinate system ${O_{\rm{c}}}{X_c}{Y_c}{Z_c}$ is the camera coordinate system, the origin $O_{\rm{c}}$ is located at the optical center of the camera, the coordinate axes $X_c$ and $Y_c$ are parallel to the $X$ axis and the $Y$ axis, respectively, the $Z_c$ axis is the main optical axis of the camera, and the point $O_{\rm{c}}$ coincides with $O$, $P'\left({{x_{\rm{c}}}, {y_c}, {z_c}} \right)$ is the imaging point corresponding to point $P$ in the camera coordinate system, $r'$ is the distance from point $P'$ to the optical axis. The coordinate system $ouv$ is the image pixel coordinate system, the origin $o$ is the optical center point, the coordinate axes $u$ and $v$ are parallel to the $X_c$ axis and the $Y_c$ axis, respectively, and $P''\left({{\rm{u, v}}} \right)$ is the midpoint of the image pixel coordinate system $P$. The imaging point, $r''$ is the distance from point $P''$ to the optical axis.

According to the isometric projection theorem:

where $r'$ is the distance from the point $P'$ to the optical axis, $f$ is the object square focal length of the optical system, $w$ is the incident Angle of the point $P$ relative to the optical axis, and $L$ is the horizontal distance between the point in the world coordinate system and the center of the fisheye lens. Due to the distortion of the fisheye lens, in order to ensure the uniformity of the image, the distortion coefficient is introduced $\lambda$ to obtain:

The camera plane center point is ${O_c}({x_0}, {y_0})$, the coordinates of $P'$ point are ${\rm{(}}{x_c}, {y_c})$, and the coordinates of $P$ point are $({x_w}, {y_w}, {z_w})$. Let the distortion coefficient components of $X_c$ and $Y_c$ axes be $\lambda _x$ and $\lambda_ y$, then:

where $\theta$ is the azimuth of point $P$, and also the azimuth of point $P'$ in the camera coordinate system.The coordinates of the center point $o$ in the image pixel coordinate system are $\left({u_0, v_0} \right)$, $P'$ is obtained by equidistant projection $P'$, and the relationship between the camera coordinate system and the corresponding points in the image pixel coordinate system is:

where, $m_x$ and $m_y$ are the amplification factors. ${k_x} = {\lambda _x}{m_x}f, {k_y} = {\lambda _y}{m_y}f$

From Eqs (15)–(20):

where $k_x$ and $k_y$ are distortion coefficients, Eq (22) is an equidistant projection model with distortion coefficients introduced to establish the conversion relationship between the corresponding points in the world coordinate system and the image pixel coordinate system. The parameters needed for this model are optical center point $o\left({u_0, v_0} \right)$, distortion coefficient $k_x$ and $k_y$, and horizontal distance $L$. According to Figure 5.

where $l$ is the virtual imaging distance of the fisheye lens, and $h$ is the horizontal distance from the point in the world coordinate system to the top of the fisheye lens.

3.2.   Establishment of measurement model

Based on the equal-distance projection model of fisheye lens with distortion coefficient, the measurement system model is constructed. The measurement system model consists of a fish-eye lens, a rangefinder and a smart phone, as shown in Figure 6.

3.2.1.   Mathematical model of measurement system

In Figure 6, the reference points in plane $A'$ and plane $B\prime$ are located in the image pixel coordinate system. Let the $A'$ coordinate be $\left({u_A', v_A'} \right)$ and $B'$ coordinate be $\left({u_B', v_B'} \right)$.$A$ and $B$ are two points in the measured object, which are in the world coordinate system. $A'$ and $A$ exist corresponding relations.$B'$ and $B$ exist corresponding relations. The relationship between the corresponding points in the world coordinate system and the image pixel coordinate system is as follows:

Equations (19)-(21) can be obtained:

When $z_w = 0$, according to Eq (25), the value of coordinate $A\left({x_A, y_A} \right)$, $B\left({x_B, y_B} \right)$ can be obtained. Calculate the distance $H$ between $AB$, $H$ is the result obtained by the measurement system model.

Where $H$ is the final calculated value which is used to calculate tree height, and several important parameters are required to calculation $H$. These parameters include the shooting distance $L$, the distortion coefficient $kx$ and $ky$, and the extreme points $A$ and $B$ of the tree. The method of obtaining these parameters will be described in detail later in the paper.

3.2.2.   Optical center point acquisition

To obtain the optical center point, Scaramuzza fish-eye lens calibration model was introduced. In Scaramuzza model, the relationship between the world coordinate system and the corresponding points in the camera coordinate system is as follows:

where $R$ is the rotation matrix and $T$ is the Transfer matrix. The relationship between the points in the camera coordinate system and the ideal image pixel points is:

where $\left({u'', v''} \right)$ is an ideal image pixel point without distortion, $\mu$ is a scalar factor, $a_0, a_2, a_3, a_4$ are polynomial coefficient of Scaramuzza model, and $p$ is a function of $u'', v''$.

The relationship between the point in the pixel coordinate system of the actual image and the pixel point of the ideal image is:

Where $\left({u_0, v_0} \right)$ is the optical center point and $\left({u, v} \right)$ is the point in the pixel coordinate system of the actual image where distortion occurs.

3.2.3.   Acquisition of virtual imaging distance

The world coordinate system is a reference coordinate system that can be artificially selected. According to Eq (27), when $z_w = 0$, $z_c = t3$, as shown in Figure 5, the distance between the world coordinate system and the camera coordinate system:

According to Eqs (23) and (31):

3.2.4.   Acquired distortion coefficient

Distortion coefficient $kx$ and $ky$ are introduced to determine the corresponding relationship between the calibration plate plane and the imaging plane, and the calibration distortion coefficient model is established. The calibration distortion coefficient model is shown in Figure 7, in which the calibration plate plane is located in the world coordinate system and the imaging plane is located in the image pixel coordinate system.

$o$ is the optical center point, the coordinate is $(u_0, v_0)$, $r''$ is the distance between the point $P''$ and the optical center point $o$, $r_1''$ is the component of $r''$ in the direction of $u$ axis, and $r_2''$ is the component of $r''$ in the direction of $v$ axis. $ow$ is the intersection point between the optical axis and the calibration plate plane, the coordinate is $(x_{ow}, y_{ow})$, $r$ is the distance from the point $P$ to $ow$, $r_1$ is the component of $r$ in the direction of $X_w$ axis, and $r_2$ is the component of $r$ in the direction of $Y_w$ axis. $w$is the incident Angle of point $P$ relative to the optical axis.

According to Eqs (27), (28) and (30):

$ow$ can be obtained from the above equation, and it can be obtained from Figure 7:

According to Eqs (33)-(37), the distortion coefficient can be calculated as follows:

3.3.   Extract image extreme points

In this paper, tree extreme points are defined as the highest and lowest points of a single tree. The tree extreme points in the fisheye image are extracted, and their extreme point coordinates are obtained and then substituted into the tree height estimation model proposed in this paper to obtain the actual height of the tree. The tree image is segmented by the improved tree image segmentation algorithm, and the tree extreme points can be obtained.

4.   Experimental results and discussion

4.1.   Improved FCM processing image

To prove the effectiveness of MMGR. In the experiment, SLIC, mean-shift, and MMGR-WT were used to process images to obtain super-pixel images. It can be seen from Figure 8 that the super-pixel images generated by SLIC contain a large number of regions with similar shapes and sizes, while mean-shift and MMGR. The super-pixel image generated by WT contains a large number of regions with different shapes and sizes. It can be seen that mean-shift and MMGR-WT provide better visual effects for real image requirements.

Additionally, the execution time of the three algorithms is compared as shown in Table 3, Table 3 shows the execution time of the three algorithms after performing super-pixel processing on the fisheye image of tree. It can be seen that the execution speed of MMGR-WT is faster than the mean-shift, so MMGR-WT is more suitable for subsequent image segmentation. However, despite the effective reduction of the calculation time, the processing time for images in practical applications is still very long. The average size of the images in this experiment is 1500 KB. Compressing the image to 200 KB can shorten the calculation time of the algorithm to a quarter of the original calculation time (TinyPNG). When the algorithm is applied in practice, it is an effective way to preprocess the image first. It should be noted that the method of compressing the image should be selected without changing the resolution, and reducing the original image resolution will affect the segmentation effect of the algorithm.

To obtain better local spatial information for fuzzy clustering, the adaptive local spatial information is integrated into the objective function of FCM, and the MMGR-WT algorithm is used for image pre-segmentation.

The enhanced algorithm has a reference value in the realization of image segmentation because the gray image only contains 256 gray levels, which is usually much smaller than the number of pixels in the image. However, the number of different colors in a color image is far greater than 256, so the quantization technology is introduced, that is, a clustering algorithm is performed on each channel of the color image to reduce the number of colors in the image. Traditional color quantization ignores local spatial information and only reduces the number of colors, so the quantized image is not much different from the original image in terms of color; super-pixel images have image space information while reducing the number of different colors. So the super-pixel image is better than the image quantified by the clustering algorithm. Application quantizes the color image and calculates its histogram. As shown in Figure 9, the number of pixels in the quantized histogram is greatly reduced compared to the original image. The histogram in Figure 9(d) only shows a few different colors. According to Figure 9(d), the enhanced FCM algorithm can be easily extended to color image segmentation.

4.2.   Improved SSA clustering effect comparison

To verify the effect of the proposed improved SSA to optimize the initial clustering center of the FCM algorithm (SSA-ADE-FCM), the experiment compares the FCM and SSA-FCM algorithms. Select the Iris, Breast cancer wisconsin, and UCI data sets in the A database for experiments. The composition of the data set is shown in Table 5. The population size of the algorithm is set to 30, the number of iterations is set to 500, and the parameter settings are shown in Table 4.

${c_1}$ is the balance coefficient, ${c_2}$ and ${c_3}$ are random numbers, $CR$ is the cross probability, $SF$ is the scale factor, ${\tau _1}$ and ${\tau _2}$ are conversion probabilities, $S{F_u}$ and $S{F_l}$ are scaling factors.

To compare the clustering effect, the real distribution results of one-dimensional and three-dimensional sample points of the Iris data set are selected. In this comparison experiment, FCM and SSA-FCM are selected. The clustering effect diagram of the algorithm in this paper is shown in Figure 10.

It can be seen from Figure 10 that the FCM algorithm has repeated classification regions and the classification effect is poor. Compared with the FCM algorithm, the classification effect of the SSA-FCM algorithm is improved, but there is a problem of inaccurate partial classification. Compared with the previous two algorithms, the algorithm in this paper has a better classification effect and a higher classification accuracy rate, which shows that the algorithm proposed in this paper improves the FCM algorithm.

To further test the clustering effect of the SSA-ADE algorithm, the three algorithms are tested in terms of accuracy, and each algorithm is performed 20 experiments to take the average value. The results are shown in Table 6.

It can be seen from Table 6 that in different data sets, the algorithm proposed in this paper is better than the FCM and SSA-FCM algorithm in accuracy, indicating that the SSA-ADE proposed in this paper has a good effect in optimizing the initial cluster centers and improves the problem that FCM is sensitive to the initial cluster centers is discussed.

4.3.   Model verification

The experimental equipment includes a smartphone, a fisheye lens, a black and white checkerboard calibration board, and a laser rangefinder. The effective pixels of the fisheye image are 3156 × 3156, and the checkerboard interval is 50 mm. Select 10 fisheye images parallel to the calibration plate but at different distances. The experiment was carried out on the windows10 system platform, the processing software is matlab2018b.

Use the Scaramuzza model to find the optical center point, and directly call the matlab2018b fisheye lens calibration toolbox to process the fisheye image. Figure 11 is a checkerboard picture and a checkerboard corner extraction diagram. Obtain the optical center point (1578.50, 1579.60). According to Eqs (31), (32) and (35), it is calculated that the imaginary distance $l$ is close to a value. In the subsequent calculations, the imaginary distance $l$ is replaced by the average value of ten calculations, $\bar l = 15.6246mm$.

There are 196 corner points in each checkerboard picture. Find the distortion coefficients corresponding to all the corner points in each group of checkerboards, and find the average value $k\bar x = 20.150$, $k\bar y = 20.032$. Figure 12 shows the calibration results of $k_x$ and $k_y$.

To verify the accuracy of the model, five sets of checkerboard pictures at different distances were taken, as shown in Figure 13. Take three sets of distances on the checkerboard to check the accuracy of the measurement model, and analyze the error between the calculated value and the true distance. Figure 14 shows the fluctuation of the measurement distance between $AB$, $AC$ and $AD$, the maximum measurement error is 8.283 mm, the maximum relative error is 1.274%, the minimum relative error is 0.323%, and the average relative error is 0.823%. The effectiveness and accuracy of the model.

4.4.   Tree segmentation comparison experiment

In this section, to verify whether the algorithm in this paper can effectively segment the extreme points of the tree in the fisheye image, the experiment compares the traditional FCM algorithm, the SSA-ADE-FCM without super-pixel processing, EnFCM ^[24], MRFCM ^[24], and the SSA-ADE-FCM with super pixel processing (SP-SSA-ADE-FCM). We evaluate the performance of the proposed algorithm on tree fisheye image segmentation. The evaluated data set is 351 fisheye images of trees taken randomly. We will discuss the results qualitatively and quantitatively. When evaluating the performance of each method quantitatively, we use Jaccard similarity (JS) and Dice ratio (DR) as the index of segmentation accuracy ^[24]. The equations of JS and DR are shown as follows, respectively.

Where S1 and S2 are two segmentation results of the same image. To evaluate the accuracy of segmentation, we compute the region S1 by the algorithm and the corresponding region S2. The closer the values of JS and DR to 1, the better the segmentation and bias correction. Table 7 shows the JS and DR values for these methods. It can be concluded that our method is more accurate in segmenting fisheye images of trees.

The comparison result is shown in Figure 15.

The traditional FCM algorithm, the improved SSA algorithm without super-pixel processing, EnFCM, MRFCM algorithm can distinguish the highest point of the tree, but can not distinguish the lowest point of the tree, that is, the ground and the tree can not be distinguished by the segmentation result of the fisheye image of the tree. Using the algorithm proposed in this paper to segment the fisheye image of the tree, the highest point and the lowest point of the tree can be clearly distinguished. Therefore, the improved FCM algorithm proposed in this paper has a better effect on the extraction of extreme points in the fisheye image of trees.

4.5.   Tree height calculation

The tree height measurement process is shown in Figure 16. Tree height measurement consists of five steps. Tree height measurement steps include set up measuring equipment, collect tree images, extract the image coordinates of tree extremum points, construct the tree height measurement model, and calculate tree height. The extreme point extraction is obtained by the SP-SSA-ADE-FCM program. It is enough to establish a tree height measurement model for the built measuring equipment because the measurement model of the same measurement equipment is the same. In the last step, the image coordinates of the tree extremal points are brought into the tree height measurement model to complete the tree height measurement.

To verify the accuracy of the proposed tree height measurement method in tree height measurement, tree samples were obtained from different locations in the study area, and 50 trees were measured. The tree fisheye image can be a single tree or a row of trees. The highest point and lowest point of all trees are extracted using the extreme point extraction method in Section 4 and brought into the tree height estimation model to calculate the tree height. The actual height of the trees is measured by a total station, and the average value of each tree is measured 5 times as the actual height of the tree. The method in this paper is compared with the method of Transponder T3, and the tree height is as low as 50 trees. The trees are numbered, the tree height measurement results are shown in Figure 17, and the measurement error analysis is shown in Figure 18.

4.6.   Discussion

The experimental results show that the relative error of the method in this paper is lower than Transponder T3. The method in this paper calculates the highest relative error of 3.05%, the lowest relative error of 0.5%, and the average relative error of 1.45%, the measurement error interval is -0.196 m ~ 0.195 m. Using Transponder T3 to measure tree height, the highest relative error is 6.13%, the lowest relative error is 0.35%, and the average relative error is 3.34%. Compared with the Transponder T3 method of measuring tree height, the method in this paper has higher measurement accuracy. The measurement equipment is compact and has the advantage of being convenient to carry. The method is less affected by wind and can be used in complex environments. At the same time, its portability determines that it is suitable for fieldwork.

5.   Conclusions

The need for monitoring forests has become one of the hotspots in scientific communities. Forest ecosystem plays an irreplaceable role in global carbon balance and mitigating global climate change, therefore accuracy calculation of tree height which is considered as an important indicator of forest carbon stork is a focus of forest research. In this study a new method of measuring tree height has been proposed which has the following advantages:

(Ⅰ) The measuring equipment is easy to get which consists of a cell phone and an adaptive fisheyes lens, that will be continuous improved with the rapid development of electronic technique and manufacturing capacity.

(Ⅱ) This method is open research. The specific operation steps, camera calibration methods, and image processing methods have been provided in this paper, in the future, the method can be optimized according to the actual situation.

(Ⅲ) The accuracy of this method is higher than the currently widely used Transponder T3 which avoids the interference of wind speed and reduces the error of human operation. In addition, the permanent storage of images can be used for comparison.

Future work can use more advanced segmentation algorithms, then optimize the program and make it into a mobile APP that can real-time measurement. As smartphones continue to improve and become ubiquitous, it is promising to implement this approach in future smartphone deployments.

Acknowledgments

Conceptualization, Jiayin Song. and Xiaopeng Zhang.; methodology, Jiayin Song.; software, Zhixiang Chi.; validation, Yue Zhao., Jiayin Song. and Qiang Ma; formal analysis, Yue Zhao.; investigation, Yue Zhao.; resources, Tianrui Yin.; data curation, Yue Zhao.; writing—original draft preparation, Jiayin Song.; writing—review and editing, Jiayin Song.; visualization, Yue Zhao.; supervision, Jiayin Song.; project administration, Jiayin Song.; funding acquisition, Jiayin Song. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by "Jiayin Song, grant number 2572017CB13" and "The APC was funded by Jiayin Song".

Conflicts of interest

There is no conflict of interest to declare.