Multiview intelligent networking based on the genetic evolution algorithm for precise 3D measurements

1.   Introduction

3D vision measurement technology uses the type of image information obtained by a visual sensor as the carrier. The geometric information of the object to be measured in the space is calculated, and then the target object is reconstructed, and its 3D morphology information is restored ^[1,2,3,4]. High-precision 3D scanners have been widely used for large and complex structures based on the need for fast and noncontact measurements. However, the measurements can only be performed offline and require a long time. To enable online panoramic 3D measurements of small industrial structures, a high-speed 3D profilometer has been developed. The ultrafast industrial speed can reach 64 kHz, and materials with different reflective intensities are simultaneously measured with high speed, precision and stability. However, the measurements are expensive and have a limited application range. It is difficult to measure large complex structures in three dimensions ^[5]. Although visual inspection systems provide excellent accuracy within their field of view, they are not reliable in verifying the spacing between two entities located in different fields of view. In addition, a visual inspection system requires the installation of multiple cameras on fixtures dedicated to an application, project or component. This lack of flexibility is a major disadvantage, because if certain parameters of the tested component change, the position of the camera may become askew, and the fixtures may need to be replaced. Therefore, a fixed, rigid, permanent, expensive system suitable for small variations and mass production is well suited to the emerging needs of intelligent manufacturing. To date, there is no perfect solution for achieving flexible online 3D inspection of industrial components.

Multivision networking measurement technology is an intelligent networked measurement method that has been developed in recent years. Some of the problems described above were solved, and it is one of the important development directions for modern industrial measurement ^[6,7,8]. The method can be used to accurately measure the 3D topography of industrial parts with multiple sizes at a low cost. Using this method, the visual sensors are placed at different measuring positions, and the object to be measured is photographed from different angles. Multiple viewpoint positions are required for the formation of a camera imaging network. In the actual measurement environment, the surface topography curvature of the part is not singular, which makes the best measurement position and azimuth angle of each visual sensor different. Based on the above issues, a visual measurement network planning problem arises. However, the determination of the position points in the current visual measurement network is mainly restricted by the measurement accuracy ^[9,10,11], and this is somewhat incomplete. Therefore, it is necessary to extend stereo vision measurement technology to the topography measurement engineering practice through intelligent algorithms.

Scholars have conducted a substantial amount of research on camera pose optimization and camera distribution methods for multivision 3D measurement technology. In the initial research on multivision measurement, optimizations were mainly based on full coverage regarding imaging with certain constraints ^[12,13]. For example, in reference ^[14], the visibility constraints and coverage are considered, and the measurement cameras are classified and screened. Then, the penalty function is used to continuously optimize the screening, and the optimal visual measurement network is obtained. However, this measurement network does not take into account the influence of the shielding area and observation accuracy, so it does not have complete universality. In ^[15], in the process of multivisual networking, only the attitude of each node is considered, and the constraints between the nodes are not considered. Therefore, the obtained network has serious redundancies. Chen et al. proposed a photogrammetric network planning method based on a genetic algorithm (GA), which in turn was based on the high measurement accuracy requirements of a large trough concentrator ^[16]. This method used the 3D reconstruction uncertainty of the measurement target point as the network planning evaluation standard and combined the constraints of the actual measurement environment to conduct a simulation analysis to obtain the optimized simulation measurement network. The simulation results are verified by 3D measurement experiments.

The various visual measurement methods described above attempt to predict the visibility of the target points and the feasibility of the camera station to optimize the initial imaging network without using CAD models ^[17]. There are known disadvantages; all of these technologies need a good initial network expert (i.e., prior knowledge) ^[18]. Some network design strategies are suitable for coordination and are used for specific object features or texture changes of the 3D reconstructions from multiple views. Their method considers the difference in the image texture in the overlapping area, and by adjusting the feature vectors of the 3D points perpendicular to the camera view direction, the feature values are minimized to ensure the accuracy of the 3D reconstructions. However, the constraints related to the measurement range are not considered, and the self-occlusion area is not applicable.

There are many constraints within the measurement range, such as the image size and resolution constraints, number and distribution of the feature points, camera field of view, depth of field, the imaging and size overlapping area of the target working area ^{[19,20,21,22,23,24]}. These range constraints are related to another set of visibility constraints, including the incidence angle, occlusion and self-occlusion area. The design of a multiview visual network is characterized by the use of multiple parameters, multiple constraints, a large amount of computation and so on. These characteristics determine that the search for an absolute exact optimal solution requires a very large amount of computation. Theoretically, this is a complex optimization strategy problem, which is difficult to describe theoretically and requires a highly technical implementation. Therefore, the efficiency of the algorithm must be considered in the implementation. If the above factors are considered in the planning of a visual measurement network, and intelligent algorithms extend stereo vision measurement technology to the topography measurement engineering practice. Then, in the industrial vision measurement field, the accuracy and reliability of the measurements will make a qualitative leap.

The feasibility of a GA solution in the search space has broad possibilities, and the population search process has significant advantages. Moreover, the algorithm can search for the global optimal solution in parallel, and it can be selected and applied in combination with other intelligent algorithms for different engineering problems. In this paper, the multiview stereovision 3D measurement principle and the multivision networking constraints are analyzed, and the parameters to be optimized are obtained. To seek an accurate position and pose of the camera in the vision measurement network, it is proposed that the genetic evolutionary algorithm (GEA) is integrated into the network design. In this algorithm, the genetic operation step is improved, and the fitness function is dynamically calibrated. Then, the centering networking method is designed for the workpiece structure, resulting in easy centering. The spherical model is adopted, and the optimal observation distance is taken as the spherical radius. The required constraint conditions are considered, and the optimal centering camera network is iteratively selected by GEA to implement precise 3D measurements.

2.   The 3D measurement principle based on multiview stereo vision

To achieve full measurement coverage, numerous measurement modules must be arranged in space to form a multiview stereo vision measurement network. Based on the binocular vision 3D measurement principle, the multivision 3D measurement model is further derived. The 3D measurement principle is shown in Figure 1.

The cameras of each measuring viewpoint are set as ${O}_{1}, {O}_{2}, \cdot \cdot \cdot, {O}_{\mathrm{n}}$. For the convenience of calculation, the camera coordinate system, (${O}_{c1}-{X}_{c1}{Y}_{c1}{Z}_{c1}$), of Camera 1 overlaps with the world coordinate system, (${O}_{w1}-{X}_{w1}{Y}_{w1}{Z}_{w1}$) analogously with the 3D measurement principle of binocular stereovision. The characteristic points on the object to be measured are ${P}_{i}(i = \mathrm{1, 2}, 3...)$. The coordinate of ${P}_{i}$ in the coordinate system of Camera 1 is $({x}_{i1}, {y}_{i1}, {z}_{i1})$, and the coordinate system of ${P}_{i}$ in the coordinate system of Camera 2 is $({x}_{i2}, {y}_{i2}, {z}_{i2})$. The rotation matrix between the coordinate systems of Cameras 1 and 2 is ${R}_{21}$, and the translation matrix is ${t}_{21}$. Therefore, the transformation formula for the coordinate system of Cameras 1 and 2 is shown in Eq (1).

By analogy, the coordinates of ${P}_{i}$ in the coordinate system of Camera n-1 are $({x}_{i(n-1)}, {y}_{i(n-1)}, {z}_{i(n-1)})$, and the coordinates of ${P}_{i}$ in the coordinate system of Camera n are $({x}_{in}, {y}_{in}, {z}_{in})$. The transformation relation between the coordinate systems of Cameras n-1 and n is shown in Eq (2).

The coordinate transformation matrix of the camera is obtained from Eqs (1) and (2), as shown in Eq (3).

According to the above derivation, any camera coordinate system in the measurement network of a multiview stereo vision can be expressed as ${O}_{cj}-{X}_{cj}{Y}_{cj}{Z}_{cj}(2\le j\le n)$, and the spatial 3D coordinates of its characteristic point ${P}_{i}$ are shown in Eq (4).

The 3D information of any object to be measured in space can be handled through integrating the binocular and multiview stereo vision 3D measurement principle.

3.   Multiview intelligent networking method based on GEA

Visual networking has many constraints, a large amount of computation, complex parameters and requires interdisciplinary knowledge. Based on these characteristics, the stability and operational efficiency of algorithm technology must be considered in networking design implementation, however, it the method of finding the global optimal solution is difficult to use in traditional mathematical modeling. This paper improves on GA to solve the visual networking design problem.

3.1.   Implementation process of GEA

According to Darwinian evolution theory, a genetic algorithm (GA) can automatically find a solution model for the theoretical global optimal solution ^[25]. Genetic operations include three methods: selection, crossover and mutation. GEA is a new calculation model that evolved based on the GA mechanism. In addition to having all the characteristics of GA, GEA has different improvement methods for different genetic operation problems, and can also conduct individual optimizations for multiple objectives ^[26]. The specific implementation process is as follows:

Step 1. Input the total number of individuals N, the maximum number G of iterations, the cross probability $C\in \left[0.4, \mathrm{ }\mathrm{ }\mathrm{ }\mathrm{ }\mathrm{ }\mathrm{ }0.99\right]$, the mutation probability $M\in 0, \mathrm{ }\mathrm{ }\mathrm{ }\mathrm{ }\mathrm{ }\mathrm{ }0.05$ and the intensity factor α. The parameters of fitness function threshold Fit and the discrete precision epsDiscrete are set according to the measurement accuracy requirements. The N algorithm individuals x_i (i = 1, 2, …, N) are randomly initialized.

Step 2. Since each function to be solved has an interval constraint, the operation process uses binary and is coded according to the iterative precision. N coded individuals are uniformly and randomly generated, and the fitness function is calculated for each individual.

Step 3. Implement genetic manipulation, selection, crossover and mutation.

Step 4. Record the initial generated optimal individual ${x}_{op}$ and its fitness function $f\left({x}_{op}\right)$.

Step 5. When the number of iterations k < G, consider whether a large mutation is needed. The mutation operation in the genetic operation is used to avoid the "precocity" of the algorithm. However, since, in the operation, the mutation probability is usually small, it is necessary to iterate a large progeny group before the mutation of new individuals can occur, which greatly increases the amount of computation.

Define:

where ${f}_{ave}$ is the average fitness value, ${f}_{min}$ is the minimum fitness value and ${f}_{max}$ is the maximum fitness value. If Eq (5) is satisfied, there is no need for large variations. If Eq (5) is not satisfied, all the individuals, except for the best ones, need to conduct large genetic mutation operations.

Step 6. The fitness function after large mutations is dynamically calibrated. In the algorithm operation process, the fitness function of each individual may be slightly different, resulting in the selection operation weakening or even disappearing, which is not conducive to the stability of the algorithm. In this paper, the fitness function is calibrated dynamically to ensure the relative difference of the individual fitness function.

The formula $f\left({x}_{i}\right) = f\left({x}_{i}\right)-{fk}_{min}$ is used to calculate the individuals of the optimal fitness function and replace the worst individual, where Q is the initial adjustment value, $q\in \left[0.9, \mathrm{ }\mathrm{ }\mathrm{ }\mathrm{ }\mathrm{ }\mathrm{ }0.999\right]$ represents the constringency coefficient, and ${\mathrm{f}}_{\mathrm{a}\mathrm{v}\mathrm{e}}$, ${f}_{min}$ and ${f}_{max}$ are calculated again.

Step 7. Equation (6) is used to calculate the selection probability and cumulative probability:

Step 8. N offspring are produced. When ${P}_{i-1} < r < {P}_{i}$, for each offspring i; based on the idea of "survival of the fittest" in GEA, the roulette algorithm is used to select the paternal line in the GA, and then the maternal line is randomly selected.

The calculation of adaptive genetic crossover probability is shown in Eq (7):

If $r < prC$, a cross operation is performed. The probability of adaptive genetic mutation is calculated as shown in Eq (8):

If $r < prM$, perform the mutation operation. All the individuals, except the optimal one, are replaced by the next generation.

Step 9. Update the iterative optimal individual ${x}_{opt}$, and record the iterative number k of offspring. When ${f}_{ave}/f\left({x}_{opt}\right)\ge thFitness$, the fitness function exceeds the threshold, the algorithm converges and the global optimal solution is obtained. At the end of the iteration, the optimal visual networking method is obtained by decoding.

3.2.   Multivision measurement networking method

For the structural parts with easy centering, take the center of the target to be measured as the center of the sphere to build the visual sphere model, and combine the GEA and visual constraints. The theoretical global optimal camera networking method is obtained, and in this paper, it is called peering-type networking.

3.2.1.   Visual sphere model and parameters to be optimized

First, the target structure to be measured is centered (refer to the literature ^[27]); the visual sphere model is established based on this. In peering-type networking, the distance between the camera and the object to be measured is fixed in the measurement process, and the optical axis of the camera is toward the center of the sphere, so the camera networking is on the surface of the spherical model with a fixed radius, as shown in Figure 2. In the visual sphere model, each viewpoint represents a camera station point, and the angle on the camera projection plane is called the azimuth angle $\alpha$. The included angle on the vertical plane is called the elevation angle β, where $\alpha \in \left[\left.\mathrm{0, 2}\pi \right)\right.$, $\beta \in \left[-\pi /2, \pi /2\right]$. D is the equivalent focal length of the camera. In the visual sphere model, the camera is fixed on a spherical surface with a radius of D, and then the spatial position and orientation coordinates of camera i can be expressed as $\left({\alpha }_{i}, {\beta }_{i}, D\right)$. The external parameter vector of the camera can also be further estimated from the above information, where the rotation vector $R = {\left[\begin{array}{ccc}\alpha & \pi -\beta & \pi /2\end{array}\right]}^{T}$ and the translation vector $t = {\left[\begin{array}{ccc}0 & 0 & D\end{array}\right]}^{T}$. Assuming m cameras are given, the parameters to be optimized for the multiview measurement network are shown in Eq (9):

A reasonable camera spatial position is determined by $I = \left[x, y, z, \alpha, \beta, D\right]$, which has 6 parameters to be optimized. In addition, the constant parameters that affect the camera's position mainly include the focal length f and the baseline B. Theoretically, the spatial position coordinates (x, y) of the camera remain unchanged. Increasing z can increase the number of measurable points of the camera and improve the networking coverage. However, increasing z continuously will reduce the spatial resolution of the camera, resulting in the surface details of the target to be measured being ignored, and reducing the networking accuracy. Therefore, it is crucial to reasonably select the optimal distance of the camera. According to the conclusion of Literature ^[28] in Section 3 (this paper contains our team's the research work), the change in the baseline distance between the two cameras will directly lead to a change in the camera's spatial pose coordinates $\left(\alpha, \beta, D\right)$, thus affecting the measurement accuracy of the image depth. The depth information measurement error and baseline variation curve can be obtained from Eq (10) ^[28], as shown in Figure 3.

where f is the focal length and B is the baseline.

According to the above analysis, in terms of system accuracy, any two vision sensors in the measurement network can be combined into a binocular stereo vision system, and the baseline distance B can be calculated through the Cartesian coordinates of the viewpoint, so increasing the baseline distance B of the binocular vision in a certain range can effectively reduce the measurement error of the system. As shown in Figure 2, the changes in the azimuth angle and elevation angle are the key parameters to control the direction of the camera's optical axis, which directly determines the camera's spatial position and orientation coordinates. It is necessary to analyze the relationship between the two factors and the accuracy of the measurement network. In the actual binocular measurement process, to meet the requirements of the measurement task, the two azimuth angles, ${\mathtt{α}}_{1}$ and ${\mathtt{α}}_{2},$ may be different, so it is necessary to analyze the influence of the two azimuth angle changes on the measurement error. Based on the mechanism analysis results of the impact of the projection angle on the measurement accuracy in Literature ^[28], taking $\left[0, \pi /2\right]$ as an example, the change curve is shown in Figure 4, and the other three quadrants can be obtained based on the mutual complement. Δ represents the measurement error (unit: mm). Figure 4 shows that with an increasing ${\mathtt{α}}_{1}$ and ${\mathtt{α}}_{2}$, the measurement error $\Delta$ also increases.

In the same way, it can be deduced that the change curve of the relationship between the Elevation Angles ${\beta }_{1}$ and ${\beta }_{2}$ and the measurement error is shown in Figure 5. With the increase of the Elevation Angles ${\beta }_{1}$ and ${\beta }_{2}$, the measurement error $\varDelta$ first decreases, tends to be stable, and then increases. Figure 5 only shows the change curve between β ∈ [0, π/2]. The curve trend of β ∈ [-π/2, 0] and β ∈ [0, π/2] is the same, so the interval is $\left[-7{0}^{\circ }, -1{0}^{\circ }\right]$. In binocular stereo vision, the Azimuth Angle and Elevation Angle simultaneously restrict the accuracy of the measurement network.

To select a reasonable range of azimuth angles, based on the range of the interval, two complementary azimuth angles ${\mathtt{α}}_{1}$ and ${\mathtt{α}}_{2}$ are arranged, and a two-dimensional change curve is drawn, as shown in Figure 6. At this time, it is easy to observe the change in measurement error in the azimuth range. As shown in Figure 6, the measurement error $\Delta$ is evenly distributed within the interval $\left[2{5}^{\circ }, 4{5}^{\circ }\right]$ of α with a small error value. Therefore, the optimal interval of measurement error $\Delta$ within $\mathtt{α}\in \left[\left.\mathrm{0, 2}\pi \right)\right.$ is $\left[2{5}^{\circ }, \mathrm{ }\mathrm{ }\mathrm{ }\mathrm{ }\mathrm{ }\mathrm{ }4{5}^{\circ }\right]$, $\left[11{5}^{\circ }, 13{5}^{\circ }\right]$, $\left[20{5}^{\circ }, 22{5}^{\circ }\right]$ and $\left[29{5}^{\circ }, 31{5}^{\circ }\right]$.

3.2.2.   Target model to be measured and constraint conditions

The visual measurement process is affected by various kinds of constraints. The major influencing factors include the multiview measurement principle, the structural parameters of the camera itself, the measurement environment, etc. To establish a multiview measurement network, it is necessary to analyze and screen out the constraints of factors with significant influence weights from the complex constraints and establish a mathematical model. This process will directly affect the efficiency of the algorithm in the process of network design.

The main constraints are summarized as the following: the field angle constraint, visibility constraint, incidence angle constraint, depth of field constraint, common view constraint, curvature constraint and empty set constraint.

1) Field angle constraint. The field angle is defined as a measure of the imaging range in a camera system. It refers to the angle between two rays formed from the camera lens and along the boundary of the maximum object that can be contained by the lens. The larger the field of view angle, the larger the camera's field of view. If the angle between the boundary of the target to be measured and the lens exceeds the field of view angle, the part beyond the field of view angle will not be imaged in the camera, as shown in Figure 7(a). The visual area within the pyramid region represents the visual area, and the mathematical formula to constrain the pyramid region is the field of view angle constraint, as shown in Eq (11).

where $\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}} {v}$ is the unit direction vector of the camera's optical axis, ${\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}} {v}}_{a}$ is the optical axis of the camera and $\rho$ represents the field angle of the camera.

2) Visibility constraint. As shown in Figure 7(b), $\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}} {n}$ is the normal vector at measurement target point P. If the angle between the normal vector $\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}} {n}$ and the direction vector of the sensor viewpoint is equal to 90°, then the part on the right cannot be collected. The mathematical model of the visibility constraint is shown in Eq (12).

3) Constraints on incident angle constraint. As shown in Figure 7(c), in the actual visual measurement process, to reduce the pixel error value of the image points after two-dimensional imaging, it is not expected that the position of some viewpoints is coplanar with the surface of the measured object. ${\theta }_{max}$ is the maximum acceptable angle between the camera viewpoint direction vector ${\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}} {v}}_{a}$ and the target point normal vector $\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}} {n}$, which is defined as the incident angle. The constraint conditions of the incident angle are shown in Eq (13).

4) Depth of field constraint. A schematic diagram of the depth of field is shown in Figure 7 (d), where $\varDelta {L}_{1}$ represents the foreground depth, and $\varDelta {L}_{2}$ represents the back depth of the field. The object to be measured has a distance between the front and back depth of field, and the imaging is clear. Therefore, camera imaging should be constrained within this area, which is called a depth of field constraint, and the constraint formula is shown in Eq (14).

where F is the camera aperture, δ is the diameter of the allowable dispersion circle, L is the shooting distance and $f$ is the focal length of the camera.

5) Coview constraint. In the process of multiview visual measurement, any point P in space is observed by at least two cameras at the same time to attain high observation accuracy, as shown in Figure 7(e). Limited by the structure of the target under test, there is self-occlusion or mutual occlusion, and the P point can only be observed by one camera, so the spatial 3D coordinates of the P point cannot be obtained. Therefore, it is necessary to constrain each area to meet the multicamera coview condition, which is called a coview constraint, as shown in Eq (15):

6) Curvature constraint. According to the analysis in Section 3.2.1, the measurement accuracy of the visual measurement network is closely related to the camera spatial pose (α, β, D). The higher the surface complexity of the object structure to be measured, the greater the curvature is, and the more information it contains, which requires higher camera pose requirements. Therefore, the constraints need to be carried out based on the actual situation, and are called curvature constraints. The mapping relationship is shown in Eq (16).

where ${\rho }_{i}$ is the curvature of the object structure corresponding to the camera at the i-th station point.

7) Null set constraint. To ensure the independence of each view in the multiview measurement network and avoid the view overlap increasing the calculation amount, it is also necessary to limit the overlap part $\varLambda$ of each view, which is called a null set constraint. Equation (16) is as follows:

3.2.3.   Fitness function

In the GEA, the fitness function is used to represent the superiority and inferiority of each individual to determine the chance that it can be inherited. In the visual measurement networking design process, it represents the measurement networking accuracy. For the point cloud of the target to be measured, under the above constraints, it is reasonable to measure when it will be visible by two or more cameras. The parameters involved in visual network design are multidimensional, and the measurement accuracy of the point clouds are effectively estimated by using the covariance.

The line of sight intersection equation under reasonable measurement can be expressed in the nonhomogeneous form $Ax = b$, and the object point coordinates obtained from the intersection conditions are expressed in the normal equation ${A}^{T}AX = {A}^{T}b$, which can be converted into $X = {\left({A}^{T}A\right)}^{-1}{A}^{T}b$, and Eq (18) can be obtained:

where f represents the mapping relationship, and x represents the parameters related to A and b. Then, Eq (19) is obtained as follows.

The covariance matrix of the image point is shown in Eq (20).

According to the Monte Carlo simulation method ^[26], if the covariance of an image point in the point cloud cannot be obtained, that is, let ${\varSigma }_{xj}$ be the unit matrix, the Gaussian noise covariance of this point's coordinate is "1 pixel". The covariance matrix of object points obtained from the above analysis is shown in Eq (21).

where $J\text{'} = {\left[J{{\text{'}}_{1}}^{T}, J{{\text{'}}_{2}}^{T}\cdot \cdot \cdot, J{{\text{'}}_{m}}^{T}\right]}^{T}$, $J{\text{'}}_{j}$ is the first derivative matrix of the image point on the j-th camera to the measurement marker point, which is 2 rows and 3 columns, and the covariance matrix of the image point is shown in Eq (22).

If a mark point on the object structure to be measured is visible, its covariance matrix can be calculated by Eq (21), and the three diagonal elements of the covariance matrix are represented by ${\sigma }_{xi}, {\sigma }_{yi}, {\sigma }_{zi}$. Then, the measurement accuracy of the object point can be obtained as the maximum error of the three coordinates in the space, as shown in Eq (23).

If the object point is not visible, the measurement accuracy is defined as ${\sigma }_{i} = 10\kappa$, and $\kappa$ is the worst spatial resolution of all the measuring cameras. The measurement marker points on the target structure to be measured are classified into visible points and invisible points, which are numbered n₁ and n₂, respectively. Then, the improved individual fitness function table is shown in Eq (24):

According to genetic knowledge, individuals with poor fitness will be gradually eliminated. By analogy, in the measurement networking design, the larger the number of invisible measurement marks n₂, the worse the fitness will be, and this kind of networking design method will eventually be eliminated.

3.2.4.   Implementation of GEA in intelligent networking

An input operation is required before the GEA is implemented that includes the structure model of the target to be tested, the selection of constraints, the number and type of cameras, the value range of each parameter in the visual sphere model of the camera, the fitness function threshold and the measurement accuracy threshold.

The target structure to be measured is discretized into a uniform point cloud, the coordinates of the center point ${\left[{x}_{0}, {y}_{0}, {z}_{0}\right]}^{T}$ and the unit normal vector ${\left[{n}_{1}, {n}_{2}, {n}_{3}\right]}^{T}$ of the triangular grid area are calculated, and the radius r of the visual sphere is determined. The parameters to be optimized are coded and decoded, and the fitness function is calculated.

The algorithm's operation flow chart is shown in Figure 8. First, the structure of the target to be measured is input, as well as the number and type of cameras. The parameters ${x}_{j} = {\left[{\alpha }_{j}, {\beta }_{j}\right]}^{T}\mathrm{o}\mathrm{f}\;\mathrm{t}\mathrm{h}\mathrm{e}\;\mathrm{c}\mathrm{a}\mathrm{m}\mathrm{e}\mathrm{r}\mathrm{a}\;\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{e}$ that are the azimuth angle and the elevation angle, the rotation vector $R = {\left[{\alpha }_{j}, \pi -{\beta }_{j}, \pi /2\right]}^{T}$ and the translation vector $t = {\left[\mathrm{0, 0}, D\right]}^{T}$ of the camera external parameter matrix are encoded by binary code to generate the new individuals. To perform subsequent genetic operations, the individuals are divided into subpopulation A and subpopulation B. According to the constraint conditions, the fitness functions $f\left(A\right)$ and $f\left(B\right)$ corresponding to subpopulations A and B are calculated, respectively, to obtain the optimal individual ${x}_{op}$ of the two subpopulations initially generated. Except for the optimal individual, all the other individuals are replaced by the next generation. The fifth step in Section 3.1 considers whether large mutations, adaptive crossovers and mutations are needed. The above steps are repeated, update the iteration optimal individual ${x}_{opt}$ and record the iteration subalgebra k. When ${f}_{ave}/f\left({x}_{opt}\right)\ge Fit$, the fitness function exceeds the threshold, the algorithm converges, the global optimal solution is obtained and the iteration ends. The optimal individual is decoded, and the networking method of the visual measurement network is obtained.

4.   3D measurement experiment based on intelligent networking

4.1.   Simulation experiment analysis

To verify the correctness of the centering networking method, a simulation environment is set up, and the conditions required by the GEA are given. A rectangular plane with one side length 298 × 148 mm is set on the XOY plane, and then the normal vector of the plane is ${\left[0, 0, 1\right]}^{T}$. Taking the center of the plane and four corners within the plane, that is, there are a total of five marked circles, the diameter of the large center circle is ${\phi }_{1} = 59\;mm$ and the diameter of the four small circles is $\varphi = 29$ mm. For the convenience of calculation, the resolution of the camera is defined as 2000 × 2000 pixels, the equivalent focal length is 3000 mm and the radius of the visual sphere is 1500 mm. Then, the spatial resolution of the camera is 0.5 mm/pixel, the range of the azimuth angle is $\alpha \in \left[\left.0, 2\pi \right)\right.$ and the range of the elevation angle is $\beta \in \left[-\pi /2, \pi /2\right]$. Different numbers of cameras are used for the networking method, and 100 individuals are selected for iteration until the algorithm converges. The camera pose information of each networking camera is shown in Table 1, and the camera pose information is shown in Figure 8. Figure 9(a) is the result of two-camera networking, Figure 9(b) is the result of three-camera networking, Figure 9(c) is the result of four-camera networking, Figure 9(d) is the result of five-camera networking and Figure 9(e) is the result of six-camera networking.

According to the analysis in Section 3.2.1, the optimal range of the azimuth angle of the visual sphere network is $\left[1{0}^{\circ }, 7{0}^{\circ }\right]$. In the case of binocular vision, the azimuth angle optimal interval $\left[2{5}^{\circ }, 4{5}^{\circ }\right]$ should be considered. All the parameters of the visual sphere obtained by the above simulation experiments fall into the optimal interval, and the fitness is small when $-0.063$; that is, the maximum error of the 3D coordinates in space is 0.063 mm, and the absolute value decreases with increasing camera numbers, which proves the rationality of the centering networking method in this paper. Notably, the acceleration of the fitness reduction decreases gradually, indicating that the fitness will become stable when the number of cameras continues to increase, and the benefit of measurement networking will begin to decrease because of the measurement costs.

4.2.   3D measurement experiment of multiview networking for a real workpiece

The experimental platform is composed of an upper computer, mechanical arm system and camera system. The upper computer adopts a Windows 10 x 64-bit control system with 8.00 GB memory for the model ""Inter" (R)"Core" ('TM')"i5-4200H CPU@2.80GHz " ". The upper computer is used to control the motion of the manipulator, take pictures of the target to be measured and extracts its three-dimensional position information. A six-axis manipulator is used to accurately and quickly move the camera system to each viewpoint position of the network. The layout of the experimental platform is shown in Figure 10.

The workpiece to be tested is shown in Figure 11(a), and the 3D CAD model structure diagram of the model to be tested is obtained through the motion recovery structure. Then, the structure diagram is gridded through MATLAB, as shown in Figure 11(b).

In the experiment, the measurement distance is set to 300 mm, that is, the radius of the visual sphere is 300 mm. The range of the known camera azimuth angle is $\alpha \in \left[\left.\mathrm{0, 2}\pi \right)\right.$, and the range of the elevation angle is $\beta \in \left[-\pi /2, \pi /2\right]$. To ensure the best imaging, based on the optimal analysis of the azimuth angle and elevation angle in Section 3.2.1, the settings are as shown in Eqs (25) and (26).

Different numbers of cameras were used for the networking method. After running the GEA many times until convergence, the performance of the comprehensive fitness function under different camera numbers draws the fitness of 10, 15, 20, 22, 24, 26, 28 and 30 cameras, as shown in Table 2. The centering networking method with 24 cameras as an example is shown in Figure 12(a), and the dense 3D reconstruction results obtained under this networking method are shown in Figure 12(b).

4.3.   Precision analysis of measurement

The reconstruction measurement results are analyzed based on the photographic geometry idea, and the feasibility and accuracy of the core networking method are analyzed quantitatively. To obtain the mapping scale relationship between the actual size and the reconstruction size, the calibration plate is reconstructed and measured. Three squares, A, B and C, with rich reconstruction information are selected, and three intersections are recorded as ${A}_{1}{A}_{2}{A}_{3}$, ${B}_{1}{B}_{2}{B}_{3}$ and ${C}_{1}{C}_{2}{C}_{3}$, respectively. The results are shown in Figure 13.

The mapping scale Factor s represents the ratio between reconstruction size L_{reconstruction} and the actual size L_true, as shown in Eq (27). The scale factor remains unchanged under the same camera. The error between the reconstruction size and actual size can be compared and analyzed by using the scale factor calculated by the calibration plate, and the error calculation is shown in Eq (28). Thus, a reconstructed coordinate system with the pixel size in units is constructed. The size information is shown in Table 3.

In the formula, ${L}_{\text{reconstruction}}$ is the reconstruction size, ${L}_{true}$ is the actual size, e is the size error, $dr$ is the actual size of the target to be measured and $dt$ is the reconstruction size after photographic geometric mapping.

The shape of the object measured by the centering network is regular, and the size can be used as the evaluation target. The average of each mapping ratio in Table 3 is calculated to obtain s = 0.009573. Then, the reconstructed dimensions and errors of the target under the centering networking method are shown in Table 4.

Table 4 shows that the error between the reconstructed size and the actual size under the centering networking method is controlled within half a millimeter. Compared with the expensive noncontact measurement system, the experimental layout is simple, which saves significant costs and meet the needs of industrial topography measurements. The 3D reconstruction image is clear and has a fine texture, which meets the needs of human-computer interaction in industrial measurements. The correctness and feasibility of the centering networking method designed in this paper are verified.

5.   Conclusions

The intelligence of multivision networking enables 3D measurement technology based on vision to implement low cost and real-time measurements. In this paper, a multivision intelligence networking method based on a genetic evolution algorithm is proposed. By using the proposed method, centering multivision intelligence networking is established. GEA is a heuristic algorithm, and each operation may have different results, so it is necessary to deploy it as many times as possible and select the best individual according to the fitness function. In the centering networking method, the camera position coordinates and pose coordinates are designated as the parameters to be optimized in combination with the visual sphere model. Considering the constraints encountered in the camera measurements, the fitness function is improved to reduce the computation amount, and ultimately, an optimal centering networking method is obtained.

The following work further studies the intelligent visual networking measurement of irregular shapes based on the method proposed in this paper, and is called, scattered networking. It is expected that scattered networking will implement real-time 3D measurements of complex shapes and large structures with multivision flexible networking.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

This research was funded by Yangzhou City, 2021, leading talents of the "Green Yang Jinfeng project" (Innovation in Colleges and Universities) (Grant no.YZLYJFJH2021CX044) and the Science and Technology Talent support project of Jiangsu Province, China (Grant no. FZ20211137).

Conflict of interest

The authors declare no conflict of interest.