Multi-view registration of unordered range scans by fast correspondence propagation of multi-scale descriptors

This paper proposes a global approach for the multi-view registration of unordered range scans. Our method starts with the pair-wise registration, where multi-scale descriptor is selected for feature point and the propagation of feature correspondence is accordingly accelerated. Subsequently, we design an effective rule to judge the reliability of these pair-wise registration results. According to the judgment of reliability, we propose a model fusion method, which can utilize reliable results of pair-wise registration to augment the model shape. Finally, multi-view registration can be achieved by operating the pair-wise registration, reliability judgment, and model fusion alternately. The proposed approach can be applied to scene reconstruction and robot mapping. Experimental results conducted on public datasets show that the proposed approach can automatically achieve multi-view registration of unordered range scans. Compared with other related approaches, the proposed approach has superior performances in accuracy and effectiveness.


Introduction
Due to the wide applications, range scan registration has been attracted more and more attentions in many domains, such as robot mapping [1][2][3][4], computer vision [5,6] and computer graphics [7,8].Given the global reference frame, the goal of registration is to find optimal transformation(s) for one or more scans so as to transfer all scans into the unified frame.According to number of scans to be registered, this problem can be divided into pair-wise registration and multi-view registration.
For pair-wise registration, Besl et al. [9] proposed the iterative closest point (ICP) algorithm, which is one of the most popular registration approaches.Although this approach can achieve the pair-wise registration with good efficiency, it is unable to get desired results for registration of partially overlapping range scans.Besides, it is a local convergent approach.To obtain the desired results, good initial parameters should be provided to the ICP algorithm.For registration of partially overlapping scans, Chetverikov et al. [10] proposed the trimmed ICP (TrICP) algorithm, which introduces the overlap percentage to determine the overlapping parts so as to estimate accurate rigid transformation.Further, Phillips et al. [11] improve the performance of original TrICP algorithm in efficiency.To address local convergence, Genetic algorithm (GA) [12][13][14] and particle filtering [15] have been applied for global registration of scan pairs.To guarantee global optimality, initializations still require being reasonably good as otherwise spaces of parameters are too large for heuristic search.What's more, these global optimization approaches are requires high computation complexity.Subsequently, Yang et al. [16] proposed the first globally optimal algorithm, named Go-ICP.This approach is based on a branch-and-bound (BnB) scheme that searches the entire 3D motion space and able to produce reliable registration results regardless of the initialization.Besides, many feature match based approaches have been proposed for pair-wise registration [17][18][19].Among these approaches, one of the most efficient and robust approaches is multi-scale descriptors with correspondence propagation [19], which can be further improved in efficiency.
Although these above-mentioned approaches may obtain good results for pair-wise registration, most of them cannot deal with multi-view registration.Given a set of range scans, the task of multi-view registration is to find optimal transformation for each scans to the reference scan.Compared with pair-wise registration, multi-view registration is somewhat difficult due to many transformations required to be estimated.To solve this problem, many good solutions have been proposed.Among these solutions, the sequential registration approach is the simplest one [20].For multi-view registration, it alternately aligns and integrates two scans until all scans are integrated into one model.Although this approach is simple, it suffers from the error accumulation problem.To address this issue, Zhu et al. [21] proposed the coarse-to-fine registration approach, which sequentially aligns each scan to a model integrated by all other align scans.By traversing all scans, their transformations are sequentially refined by the pair-wise registration.Meanwhile, Govindu et al. [22] proposed multi-view registration approach based on motion averaging algorithm, which takes a set of available relative motions (pair-wise registration results) as input to simultaneously estimate all transformations for multi-view registration.Based on this work, Guo et al. [23] proposed the weighted motion algorithm for multi-view registration.This approach can improve the robustness and accuracy of multi-view registration by paying more attention to reliable pair-wise registration results.Recently, Arrigoni et al. [24] proposed multi-view registration approach based on the low-rank and sparse (LRS) matrix decomposition.It stacks all available relative motions into a large matrix and replaces the unavailable ones with zeros, then utilizes the LRS decomposition to recover global motions for multi-view registration.Moreover, Georgios et al. [25] cast the multi-view registration problem into the framework of clustering.For clustering, it utilizes the Expectation-Maximization (EM) algorithm to simultaneously estimate Gaussian Mixture Model (GMM) and all transformations for multi-view registration.
Although these above approaches may obtain desired results for multi-view registration, they should be provided with good initial registration parameters.Otherwise, it is difficult to obtain desired registration results.For global multi-view registration, Daniel et al. [26] proposed the approach for fully automatic registration of multiple range scans.It applied the pair-wise registration on all scan pairs involved in multi-view registration and then tested these results for surface consistency.Accordingly, reliable results of pair-wise registration can be checked for the estimation of multi-view registration.Since there are many scan pairs involved in multi-view registration, the computation complexity of this approach is very high.Subsequently, Guo et al. [27] proposed a feature matched approach for global multi-view registration.This approach is also based on the pair-wise registration and it is more efficient than previous approaches.However, it sometimes cannot find the minimum number of good feature matches for pair-wise registration, which leads to the failure of multi-view registration.Recently, Zhu et al. [28] proposed the global multi-view registration approach based on the construction of spanning tree.It designed the dual criterion to judge the reliability of pair-wise registration results.By viewing the first scan as the root node, all other scans can be added by the breadth-first search with the reliable results of the pair-wise registration.As most scan pair contains low overlap percentages, it is also required to align and check many scan pairs to construct a full spanning tree.Therefore, its computation complexity is also high.
For multi-view registration, this paper proposes a global approach for registration of unordered range scans.The contribution of this paper can be delivered as follows: 1) Improve the global pair-wise registration approach based on fast correspondence propagation of multi-scale descriptors; 2) Design the effective rule to judge the reliability of pair-wise registration; 3) Propose good model augmentation, which can properly augment the model shape for further pair-wise registration.To demonstrate its effectiveness, the proposed approach will be tested on public available data sets and compared with two related registration approaches.
The reminder of this paper is organized as follows.We briefly review pair-wise registration approach in Section 2. In Section 3, we present details of the proposed multi-view registration approach.All experimental results are displayed in Section 4. Finally, we conclude this work in Section 5.

Pair-wise registration
This section briefly reviews the TrICP algorithm and correspondence propagation of multi-scale descriptors for pair-wise registration.

TrICP algorithm
Suppose there are two partially overlapping scans in 3   , a data shape and a model shape  , where x denotes the overlap percentage and P x indicates the overlapping part of P to Q .Therefore, the overlap percentage can be calculated as , where ' p N denotes the number of point included in the subset P x .Subsequently, it is convenient to define trimmed mean square error (TMSE) as: where R is the 3 3  ´ rotation matrix, t  is the 3 1 ´ translation vector and p q denotes a pair of point correspondence.According to [], the task of pair-wise registration is to find an optimal rigid transformation ( , ) t R  , which can be obtained by minimizing the following objective function: where ( 2) l l = is a preset parameter.Actually, Eq. (1) has been solved by the TrICP algorithm, which achieves pair-wise registration by iterations.Given initial parameters 0 0 ( , ) t R  , three steps are included in each iteration: (1) Establish point correspondences between two range scans: (2) Update the overlap percentage and its corresponding subset: (3) Calculate the new transformation: A rigid transformation ( , ) t R  will be obtained by repeating step (1)-( 3) until the iteration k reaches the maximum number K or , where e denotes a given small positive number.Similar to the original ICP algorithm, the TrICP algorithm is also local convergent.To obtain desired results, good initial parameters should be provided.

Correspondence propagation of multiscale descriptors
For the pair-wise registration, multi-scale descriptors have been proposed in [].Given a point, L different support radii are considered.Within each support radius, all included range points are used to compute one variance matrix l C .Based on these L circles and variance matrix , it is convenient to compute multi-scale descriptors ( , ) N D , where N denotes the normal vector and D indicates the eigenvalue based vector.Accordingly, the pair-wise registration is solved by the effective approach displayed in Fig. 1.

Flow chart of the pair-wise registration approach based on correspondence propagation of multi-scale descriptors
As shown in Fig. 1, after the calculation of multi-scale descriptors, the correspondence of each point from 1 y to n y is established by performing the nearest neighbor (NN) search on the eigenvalue based vector D .For correspondence propagation, each of these point correspondence can be viewed as a seed match.Then, the propagation session exploits on both D and N to propagate each seed match (e.g., 1 ( , ) s y x ) into a set of matches (e.g., For each match set, the RANSAC algorithm should be used to find the maximum consensus set so as to calculate one optimal rigid transformation.Based on down sampled scans d Q and d P , a quality function formulated from distances errors is defined to identify the best one from all obtained rigid transformations.Finally, the TrICP algorithm refines results of pair-wise registration by taking the best transformation ( , ) best R t as input.
According to [19], the time complexity of each propagation operation is about where n and m denotes the number of points included in the down sampled scans d Q and d P , respectively.As the required number of propagation is n , the total complexity of correspondence propagation is . Although this approach is robust for the global pair-wise registration, the correspondence propagation is too time-consuming.

Global registration of multi-view scans
For multi-view registration of unordered scans, a global approach is proposed as shown in Fig. 2. Given a set of unordered scans, the frame of reference, without loss of generality, can be attached, to the first scan.As shown in Fig. 2, it achieves multi-view registration by pair-wise registration, reliability judgment of pair-wise registration and model augmentation, which are main operations of the proposed approach.Therefore, we will present more details of these three operations and the implementation of this approach.

R t
Fig. 2 The flowchart of the proposed approach for multi-view registration, which operates pair-wise registration, the judgment of pair-wise registration result and model augmentation alternately until all range scans are integrated into one model.

Pair-wise registration by fast correspondence propagation
In global registration, these are no prior information about initial registration parameters.Therefore, Lei et al. [19] proposed the multi-scale descriptors with correspondence propagation for global pair-wise registration.As shown in Fig. 1, this approach should do several operations to achieve pair-wise registration.Among these operations, correspondence propagation is the most important and time-consuming one.According to [19], multi-scale descriptors consist of the normal vector N and the eigenvalue based vector D , where D is rotation invariant.Hence, seed matches is established by performing the NN search on D .But these seed matches may be unreliable due to the low dimension of D .To obtain good point matches, the correspondence propagation is necessary to be applied.However, we think it is not required to apply correspondence propagation on each seed match.Because once the propagation is starts from one reliable seed match, the exploitation of N and D will certainly propagate it into a set of good matches.Hence, some reliable seed matches are enough for the correspondence propagation of multi-scale descriptors.The problem is how to find reliable seed matches.Fig. 3 illustrates both reliable and unreliable seed matches.Actually, ( , ) p q denote three reliable matches.But 1 3 ( , ) p q and 3 1 ( , ) p q may be mismatched due to noises or other reasons.
Therefore, one seed match 1 3 ( , ) p q or 3 1 ( , ) p q with small distance may not be reliable.While, the reliable seed match End End Find the best alignment ( , ) best R t by the quality function; Utilize TrICP to refine ( , ) best R t and obtain ( , ) R t .
Output: Pair-wise registration results ( , ) R t In this algorithm, the required number of correspondence propagation is n d .As the propagation of each seed match is same to that of the method proposed in [19], the total complexity of correspondence propagation is reasonably reduced to Obviously, small value of d will lead to low time complexity.Since the seed match with small distance may not be reliable, the value of d should not be set to small.Otherwise, there may be no way to apply correspondence propagation to reliable seed match, which will lead to unexpected pair-wise registration.In practice, 0.3 d = can obtain satisfactory registration results.

Reliability judgment of pair-wise registration
For these scan pairs involved in multi-view registration, some of them may contain high overlap percentages, other may have lows or even no overlap percentage.Given a pair of scans, one rigid transformation will be estimated by the proposed pair-wise registration algorithm.For the scan pair with high overlap percentage, the estimated transformation is reliable.However, it is unreliable for these scan pairs without high overlap percentages.Usually, it is difficult to predict or estimate the overlap percentage of each scan pair, so the pair-wise registration may be applied to scan pair without high overlap percentage and obtain unreliable transformation.As shown in Fig. 1, only reliable results can be utilized for the model augmentation.Therefore, it is required to judge the reliability of pair-wise registration.
As the TrICP algorithm is finally utilized to refine the result of pair-wise registration, it also determines overlapping areas and provides TMSE for each scan pair.Fig. 2 displays pair-wise registration results of two scan pairs, where one scan pair gets reliable registration due to its high overlap percentage and the other one gets unreliable registration due to its low overlap percentage or other reasons.As shown in Fig. 2, the TrICP algorithm determines suitable overlapping areas with small TMSE for reliable registration and finds false overlapping areas with very large TMSE for the unreliable registration.Therefore, TMSE is related to the reliability of pair-wise registration.
For the same registration result, the TMSE is affected by the point resolution of model shape.Fig. 3 demonstrates the same pair-wise registrations under different point resolutions of model shape.As shown in Fig. 3, high resolution reduces the TMSE and low resolution increases the TMSE.Therefore, we can judge the pair-wise registration is reliable if the TMSE satisfies the following condition: where Q d denotes the point resolution of model shape.Otherwise, the pair-wise registration is judged to be unreliable.To achieve multi-view registration, the pair-wise registration should be implemented many times in the proposed approach.During multi-view registration, it is easy to collect TMSEs from reliable registrations, which can be further utilized to judge the reliability of posterior registration results.Therefore, it is better to revise Eq. ( 6) as follows: where

Model augamentation
By the proposed judgment, it is easy to judge whether the pair-wise registration is reliable or not.For reliable registration, data shape should be transformed and merged into the model shape.The simplest method is to directly add points of transformed data shape into the model shape.However, this rude augmentation will result in redundant range point with multi-scale descriptors, which can reduce the efficiency of subsequent registration.Hence, model augmentation should be carefully designed.Accordingly, we propose the model augmentation illustrated in Fig. 4.
Based on the estimated rigid transformation ( , ) t R  , the data shape is first transformed into the reference frame of model shape as follows: Then both model shape and transformed data shape are divided into two independent parts: where Q x and P x denotes overlapping parts of these two shapes, A and B indicates non-overlapping parts of them.Accordingly, the augmented model shape is composed of raw model shape and transformed data shape as follows: where Similar to these two raw shapes, the down sampled data shape d P should be correspondingly transformed and merged into the down sampled model shape d Q .And they can be merged in the same way as these raw shapes.Besides, multi-scale descriptors should also be merged.As eigenvalue based vectors D are rotation-invariant, they can be merged as follows: (12) where where indicate normal vectors of two matched points.In this way, the model shape is properly augmented with corresponding multi-scale descriptors, so there is no need to recalculated multi-scale descriptors for the augmented model shape.
Compared with rude augmentation, the proposed augmentation method can delete redundant range point with multi-scale descriptors.
and runtime required to achieve multi-view registration for each approaches.Table 1 illustrates registration errors of each approach tested on four datasets.As shown in Fig. 7, these three approaches require almost the same number of pair-wise registrations, which is a little more than the number of range scans to be aligned.As all these three approaches utilize the proposed reliability judgment, it is reasonable to conclude that this judgment is very effective for pair-wise registration.Sometimes, FCPMA may require a little more number of pair-wise registrations than other two approaches.In some scan pairs with low overlap percentages, there are a small number of reliable seed matches, which are all outside the top 30% .For these scan pair, the application of OCPRA and OCPMA may obtain good pair-wise registration results due to the application of correspondence propagation to all seed matches.Since FCPMA only applies the correspondence propagation to the top 30% sorted seed matches, it is unable to get good registration results.Besides, OCPRA also may require a little more number of pair-wise registrations than other two approaches.This is because the rude augmentation results in many redundant points with multi-scale descriptors, which may reduce the robustness of pair-wise registration.Although FCPMA may require a little more number of pair-wise registrations, it is more efficient than other two approaches.By selecting the top 30% sorted seed matches, it can accelerate the correspondence propagation, which will reduce the runtime of pair-wise registration.With the model augmentation, it deletes repeated range point with multi-scale descriptors so as to further improve the efficiency of subsequent registration.Therefore, both fast correspondence propagation and model augmentation are good for the efficiency of multi-view registration.1, all these three approaches can obtain accurate registration results and results of OCPRA are a little more accurate than that of other two approaches.This is because the rude augmentation keeps many redundant range points, which is good for accurate registration.However, it leads to high computation complexity.Since many exiting approaches can refine initial registration parameters, efficiency is more important than accuracy in global multi-view registration.Accordingly, OCPRA is not a good choice.For efficient registration, the proposed model augmentation is necessary to delete redundant range point with multi-scale descriptors.As most reliable seed matches contains small distances, it is easy to find good match set by the application of correspondence propagation to the top 30% sorted seed matches.Compared with raw correspondence propagation, fast correspondence propagation not only improves efficiency, but also obtains more accurate multi-view registration results.
In one word, the proposed approach is reasonable and effective for the global registration of unordered range scans.

Comparsion
To demonstrate its performance, the proposed approach was compared with two related approaches; there are automatic multi-view registration approach based on surface consistency (SurfC) [26] and automatic multi-view registration approach based on spanning tree (SpanT) [28].Multi-view registration results are also reported in runtime and registration errors.

2.1 Accuacry and efficiency
For comparison of efficiency and accuracy, experiments were carried on four data sets.Fig. 8 illustrates the number of pair-wise registrations and runtime required to achieve multi-view registration for all competed approaches.What's more, Table 2 displays registration errors of all competed approaches tested on four datasets.To demonstrate comparison in a more intuitive manner, Fig. 9 displays the registration results of four data sets for all competed approaches in the form of cross-sections.As shown in Fig. 8, SurfC requires more number of pair-wise registrations than other two approaches and the proposed approach requires the least number of pair-wise registrations among these competed approaches.Actually, SurfC should apply pair-wise registration to all scan pairs involved in multi-view registration.Given N range scans, SurfC requires (  1) 2 N Npair-wise registrations.Therefore, this approach is inefficient.For multi-view registration, SpanT should find at least ( 1) Nreliable pair-wise registration of raw scan pairs to construct completed spanning tree.As most of raw scan pairs contain low overlap percentages, it is also requires a large number of pair-wise registrations.While, the proposed approach only requires a small number of pair-wise registrations to achieve multi-view registration.During multi-view registration, the proposed approach utilizes the reliable pair-wise registration to augment the model shape, which can increase the overlap percentage between the model shape and data shape.With the increased overlap percentages, it become easy to obtain reliable pair-wise registration.Hence, the number of required pair-wise registrations is reasonably reduced.Since the proposed approach utilizes the pair-wise registration by fast correspondence propagation of multi-scale descriptors, it is more efficient than other two approaches.As shown in Table 2 and Fig. 9, the proposed approach can obtain the most accurate results for multi-view registration among all competed approaches.Since these competed approaches utilize the pair-wise registration to achieve multi-view registration, they all suffer from the problem of error accumulation.In SurfC and SpanT, the pair-wise registration algorithm is directly applied to raw scan pairs.Since few raw scan pairs have high overlap percentages, few reliable results of pair-wise registration are very accurate, which will lead to large accumulated errors in multi-view registration.While, the proposed approach use reliable pair-wise registration results to augment model shape, which can increase overlap percentages of scan pairs to be aligned.With increased overlap percentages, results of pair-wise registration become more reliable and accurate, which will reduce accumulated errors in multi-view registration.Therefore, the proposed approach can always obtain the most accurate results for multi-view registration.

2.1 Robustness
For the comparison of robustness, all competed approaches were tested on Stanford Bunny with five groups of different order.Multi-view registration results are also reported in runtime and registration errors Table 3 illustrated registration results of different approaches.
As shown in Table 3, all competed approaches are robust to the order of multiple range scans and the proposed approach can obtain the most accurate and efficient registration results for Stanford Bunny under different orders.Although the order of input range scans may be changed, all these three approaches utilize reliable pair-wise registration results to estimate rigid transformations for multi-view registration.Therefore, they can always achieve good results for multi-view registration.As SpanT and the proposed approach only apply the pair-wise registration on some scan pairs, the change of scan order will cause their required number of pair-wise registration to be varied.Accordingly, the run time of these two approaches is varied under different scan orders.However, SurfC applies pair-wise registration on all scans pairs involved in multi-view registration; its run time is relatively stable under different scan orders.By only considering the accuracy, the proposed approach is very robust.

Conclusions
This paper proposes the global approach for multi-view registration of unordered range scans.This approach is based on the pair-wise registration, which is achieved by the fast correspondence propagation of multi-scale descriptors.For multi-view registration, the reliability judgment is designed to choose reliable pair-wise registrations, which are utilized to augment the model shape.By model augmentation, the overlap percentage of scan pair will be reasonable increased.With the increased overlap percentage, the pair-wise registration is able to obtain reliable accurate results, which will reduce the required number of pair-wise registration and accumulated errors for multi-view registration.Experimental results carried on public available data sets illustrated that the proposed approach can robustly achieve multi-view registration of unordered range scans with good accuracy and efficiency.
Although the proposed approach has good performance, it does not mean that this approach can solve all multi-view registration problems.Given a set of unordered scans, there may be no other scan that contains high overlap percentage to one scan.If this special scan is viewed as the reference scan, it is difficult to obtain reliable pair-wise registration for this scan, which will lead to the failure of multi-view registration.However, it should be noted that all existing approaches for multi-view registration proposed so far share this limitation as well.In the future, we will extend this approach to robot mapping, especially for large-scale environment mapping.

3 qFig. 3
Fig.3Demonstration of different seed matches, one solid line with bi-directional arrow denotes a reliable seed match, each dotted line with bi-directional arrow denotes an unreliable seed match, the direction and length of red line with arrow denote the normal vector and the eigenvalue based vector, respectively.

Fig. 4
Fig. 4 The illustration of TMSE for pair-wise registration of two scan pairs.In the 2nd row, each black line denotes the distance of one point pair located in the overlapping area.(a) Reliable registration.(b) Unreliable registration.

Fig. 5
Fig. 5 TMSEs under different point resolutions of model shape.(a) Low resolution results in large trimmed MSE.(b) High resolution reduces the trimmed MSE.

Fig. 6
Fig. 6 Illustration of model augmentation.(a) The point pairs located in the overlapping area, where each pair of points is connected by one line with arrow.(2) Model and data shape is merged into one shape, where each point pair is replaced by its intermediate point in the overlapping area.

D
denote eigenvalue based vectors of two matched points.While, normal vectors N are rotation-invariant, so they can be merged as follows:

Fig. 7
Fig. 7 Comparison of efficiency for different approaches based on multi-scale descriptors.(a) Required number of pair-wise registrations.(b) Run time.

Fig. 8
Fig. 8 Efficiency comparison of different approaches tested on four data sets.(a) Required number of pair-wise registrations.(b) Run time.

Fig. 9
Fig. 9 Registration results of different approaches in the form of cross-section.(a) Unordered scans (b) Reconstructed 3D models.(c) Results of SurfC.(d) Results of SpanT.(e) Ours results.
In this case, the total time complexity of correspondence propagation can be seriously reduced so as to result in efficient pair-wise registration.Based on what discussed above, an efficient pair-wise registration algorithm is summarized in Algorithm 1.Propagate the i th seed match into a set of matches ( , ) p q must contain small distance.According to this fact, it is better to sort all established seed matches in ascending order by their distances, then the correspondence propagation is only applied to top 100% d ´ sorted seed matches.i M by RANSAC;

Table 1
Comparison of accuracy for different approaches based on multi-scale descriptors, where small value indicates good performance and bold number denotes the best result..

Table 2
Comparison of accuracy for different approaches, where small value indicates good performance and bold number denotes the best result.

Table 3
Comparison of robustness for different approaches, where small value indicates good performance and bold number denotes the best result.