A Flexible Calibration Method Using the Planar Target with a Square Pattern for Line Structured Light Vision System

A flexible calibration approach for line structured light vision system is proposed in this paper. Firstly a camera model is established by transforming the points from the 2D image plane to the world coordinate frame, and the intrinsic parameters of camera can be obtained accurately. Then a novel calibration method for structured light projector is presented by moving a planar target with a square pattern randomly, and the method mainly involves three steps: first, a simple linear model is proposed, by which the plane equation of the target at any orientations can be determined based on the square’s geometry information; second, the pixel coordinates of the light stripe center on the target images are extracted as the control points; finally, the points are projected into the camera coordinate frame with the help of the intrinsic parameters and the plane equations of the target, and the structured light plane can be determined by fitting these three-dimensional points. The experimental data show that the method has good repeatability and accuracy.


Introduction
With the development of photo-electronics, image processing and electronics, computer vision technique is becoming increasingly relevant in industry for on-line inspection, component quality control, solid modeling and dimensional analysis [1][2][3]. Owing to the advantages of non-contact measurement, time efficiency, high flexibility and accuracy, line structured light techniques have found numerous applications [4][5][6][7]. A basic structured light vision system consists of a camera and a laser projector rigidly fixed with respect to the camera, and the working principle is laser triangulation. During the measurement, the projector projects light stripes on a measured object and the camera obtains images of the light stripes modulated by the depth of the object, then the 3D characteristic information of the measured surface can be acquired from the 2D deformed light stripe image.
The structured light vision system must firstly be fully calibrated before performing any 3D measurement. The goal of calibration is to establish the mapping relationship between the structured light plane and the computer image plane. The traditional approach for calibrating a structured light vision system incorporates two separate stages: camera calibration and projector calibration.
In the camera calibration stage, the intrinsic parameters and extrinsic parameters of camera model are usually calibrated by observing a calibration object whose geometry in 3D space is known with very good precision [8,9]. Camera calibration can be done very efficiently by using Tsai's method [10], in which an expensive calibration apparatus is usually required, e.g. a 3-D target. Zhang [11] proposed a flexible new technique for camera calibration by viewing a planar target from different unknown orientations. Accurate calibration points can be easily obtained using this method. Now, it is widely used in the camera calibration.
In the projector calibration stage, the coefficients of the structured light plane equation relative to the camera coordinate frame should be determined. Currently, different approaches for calibrating projector have been proposed in many literatures. In Robert Dewar's method [12], several thin non-coplanar threads are strained in the space illuminated by a light stripe, and then several bright light dots are obtained as the control points whose 3D coordinates can be measured by means of a theodolite. However, this method requires complicated and expensive equipment, and its accuracy is affected by width restrictions of the light stripe. Huynh [13] has proposed a method, in which the world points on the light stripe plane are generated based on the invariance of the cross-ratio. In the method, a 3D calibration target usually consisting of two or three planes orthogonal to each other is needed for getting the control points, and it is difficult to be manufactured accurately. Recently, a planar target method has been proposed by Zhou [14], in which the intersection points of the light stripe and grids on the target are obtained as the control points. Since the quantity of the grids is limited, the number of control points is still not enough for calibrating projector accurately. Sometimes, to generate the intersection points of the grids and the light stripes, the orientation of the target is not flexible and random in the space. To make the system convenient to use on-site and under limited space, Wei [15] has proposed a novel 1D target-based calibration method for structured light vision sensors by randomly viewing a 1D target from different unknown orientations positioned within the field of view. Similar to Zhou's method, the control points obtained are few in the method. Besides the above methods, there are other methods of calibrating a structured light vision system being presented in the literature [16][17][18]. But all these methods mainly have the following drawbacks: 1. it is difficult to generate large number of control points; 2. some methods require an elaborate and expensive calibration apparatus; 3. some methods are not suitable for field calibration.
In this paper, a novel approach for line structured light vision system calibration is proposed, in which a planar target with a square pattern is used. The proposed method only requires the camera to observe the planar target shown at a few (at least two) different orientations, and simultaneously the light stripe should be projected onto the target. Subpixel center localizations of the light stripe on the target are detected as the control points, which are sufficient to improve calibration accuracy. The paper is organized as follows. In the Method Section, a camera model is given first inspired by the work of Zhang, and then the detailed procedure of projector calibration is described, which includes working out the target plane, extracting center of the light stripe and fitting the structured light plane. Experimental studies are given in the next Section. The paper ends with conclusions.

Calibration of camera model
The pinhole projective model was used to project 3D scenes onto the 2D camera image plane in Zhang's method. The mapping between the world and image points proceeded through the four transformations, from world coordinates(X w ,Y w ,Z w ), via camera coordinates(X c ,Y c ,Z c ), undistorted image coordinates(x u ,y u ), and distorted image coordinates(x d ,y d ), to pixel coordinates(x p ,y p ), as shown in Table 1. Then, a nonlinear optimization function was established when the radial distortions were taken into consideration. The calibration was finished using the Levenberg-Marquardt (L-M) algorithm.
In the model, the optimization function aims at minimizing the distance error between the detected image point m and the backprojected image point m _ in 2D image pixels coordinate frame. However, the error criterion with respect to optimization and measurement is different while the measurement is implemented in 3D space. In other words, the equal pixel distance error in 2D image plane leads to diverse 3D metric distance error in different positions. The reasons mentioned above will cause accuracy decrease for 3D vision measurement [19]. Thus, the camera model is changed by minimizing the metric distance error between the projected points M _ and the real points M in 3D coordinate frame in this paper, and tangential distortion is also taken into consideration for improving accuracy, as shown in Table 2.
By means of the L-M algorithm, the model of this paper can be solved accurately. Thus, the intrinsic parameters of camera, the intrinsic matrix A~a A and the distortion coefficient K~(k 1 ,k 2 ,p 1 ,p 2 ), can be determined firstly.

Calibration of projector model
It can be observed from Fig. 1 that the 3D point can't be uniquely solved only by its known image point and the intrinsic parameters of camera model. Thus we need to determine the structured light plane, which is in the camera coordinate frame, to obtain the world coordinate from a known image point. It should be clarified that we assume that the camera coordinate frame is also the world coordinate frame. In this paper, a flexible method is given to determine the structured light plane, and the method Optimization function:  x d y d 1 Optimization function: mainly involves three steps: first, a linear algorithm is proposed to solve the plane equations of the target in the camera coordinate frame; second, the center points of the light stripe on the target are detected as control points, and these points are projected to the 3D space by means of the intrinsic parameters and the plane equations; finally, the structured light plane is determined by fitting the points in 3D.
2.1 Determination of the target plane. Currently, the targets with checkerboard pattern are often used to calibrate the structured light plane. However, the imaging quality of the light stripe in the white areas of the targets is usually very poor, as shown in Fig. 2, which will lead to the loss of the detection precision of light strip center. Different from the previous targets, a planar target with a larger black square pattern, on which imaging quality of the light stripe can be guaranteed, is used in this paper, as shown in Fig. 3. When the camera and a light stripe projector are fixed together, some calibration patterns are acquired by means of the planar target shown at a few (at least two) different orientations, and the light stripe needs to be projected onto the target simultaneously, as shown in Fig. 4.
To obtain the plane equation of the target, the image coordinates of four corner points in the pattern are firstly extracted using Bouguet's method [20], as shown in Fig. 5. By using the image coordinates and the intrinsic parameters, the undistorted image coordinates (x u ,y u ) T of the corner points can be solved, and the mapping from the undistorted image coordinates to the world coordinates (X w ,Y w ,0) T can be expressed as following: where s is an arbitrary scale factor, r i (i~1,2) is the ith column vector of the rotation matrix, t is the translation vector. Let m m~½x u ,y u ,1 T ,M M~(X w ,Y w ,1) T , H~r 1 r 2 t ½ , the equation (1) can be simplified as dividing out s from Eq. (2), where h      Lx~0, where L is a 8|9 matrix. As x is defined up to a scale factor, the solution is well known to be the right singular vector of L associated with the smallest singular value. Using the rotation matrix and translation vector, the equation of the target plane can be solved easily in the camera coordinate system, as shown in Fig. 6.

Subpixel center localization of the light stripe.
Both in calibration and in measurement the center of light stripe is required. In previous works, there are many methods being proposed for extracting the light stripe center, and Steger's method [21] is a representative one. The method extracts light stripe center based on the Hessian matrix of image intensity function at a pixel. It has high location precision, and has been widely used in vision measurement applications. However, due to multiple convolutions on the whole image, the method is slow and can not meet the requirements of real-time processing.
In this paper, we adopted a method presented in Reference [22] which combined Sobel and spatial-moment operator. Next we will show how this method works. 1). The edge points of light stripe are detected in pixel-level by Sobel operator [23], as shown in Fig. 7; 2). The gray gradient direction of the edge points are calculated as the normal direction of light stripe curve, and the cross section of light stripe can be obtained along the normal direction, as shown in Fig. 8; 3). A closed solution for extracting light stripe center is derived based on spatial moment theory, and the sub-pixel coordinates of light stripe center can be obtained fast in all cross sections, as shown in Fig. 9.
Using the intrinsic parameters and the plane equation of the target, all sub-pixel coordinates of the light stripe centre can be projected into the camera coordinate frame. In this way, the sufficient control points can be got for projector calibration.
2.3 Fitting the structured light plane. Given that the equation of the structured light plane is described by.

axzbyzczz1~0 ð7Þ
where a, b and c are the unknown coefficients. To obtain the accurate equation of the plane, the following work is carried out: move the target in the space repeatedly, and then the multi-group non-collinear projection points of the light strip centers in the camera coordinate frame can be obtained as control points. As shown in Fig. 10, all the control points should locate in the structured light plane theoretically. Thus, the structured light plane can be determined by fitting the control points (X i ,Y i ,Z i ) i~1,:::,n, and the computation formula for solving equations is as follows: Based on the intrinsic parameters and the light plane coefficients, the world coordinates of intersection points of the light plane with the world object surface can be solved.

Calibration experiment
The structured-light sensor designed in this work consists of a CCD camera with a 25 mm lens and a laser plane projector (wave length 650 nm, line width ,0.8 mm). In the experiments, the intrinsic parameters A and Kof the camera should be calibrated using the patterns from a checker board in advance. The precision of the grid on the checker board is 1mm. Nine patterns of the checker board are acquired by the camera at a spatial image resolution of 1280|940 pixels, as shown in Fig. 11. Using these patterns, the camera model given in section 2 is calibrated, and the intrinsic parameters A and K are obtained and listed in Table 3.
When the camera calibration is completed, nine patterns of a target with a square pattern (S~6 cm) are acquired by the camera, as shown in Fig. 12. Using the proposed method, the equations of the structured light plane can be solved:

Accuracy tests
To test the accuracy of line structured light vision system, the rectangular parallelopiped gauge blocks with known widths are measured using the system, and the measurement steps are as follows: 1). The light stripe is projected onto a gauge block, and the image of the light stripe is acquired by the camera, as shown in Fig. 13; 2). Two parallel lines l 1 and L 2 can be determined in the camera coordinate frame, as shown in Fig. 14(a); 3). Based on the spatial relationship in Fig. 14(b), the angle between the structured light plane p 1 and the background plane p 2 can be solved as  where s 1 is the distance between the lines l 1 and L 2 , and d 1 (the distance between the line l 1 and the plane p 2 ) is the real width of gauge block. Usually, to improve the accuracy of angle h, more than one gauge blocks are used to get the mean value in the test. With the help of the angle h, the system can be used to measure other gauge block's width based on Eq. (10). Because every block's width is known, the average absolute errors of measurement can be obtained by comparing with the real width, and they can show the error of the system.
Using the above measurement method, the methods in this paper and Zhou's [14] are compared: first, two group parameters of the structured light plane are obtained by using the two methods; then the same gauge block is measured respectively using the two group parameters with the help of the structured light system. In this test, four gauge blocks are measured using the system, and every block is measured six times, as shown in    Table 4 and 5, the measurement accuracy of the proposed method is higher than Zhou's, which has proved that the parameters of structured light plane obtained by the proposed method is more accurate. In the Zhou's method, since the control points are generated by intersecting of light stripe and limited grids in the target, a few control points can be obtained, and it is difficult to constrain the structured light plane better. In the proposed method, the planar target can be randomly moved, and all light stripe centers on the target image are extracted as the control points. More control points can achieve over-constraint of the structured light plane, which ensures the accuracy of the light plane equation.

Discussion and Conclusion
A novel calibration method for a structured light vision system by viewing a planar target from unknown orientations is proposed in this paper. In the method, the planar target with a square pattern is used to generate sufficient non-collinear control points for structured light stripe vision. This method provides a high accuracy, and is low-cost and easy to use on-site. The experiments conducted on a real structured light vision system that consists of one camera and one single light stripe plane laser projector reveal that the proposed approach is of high accuracy and is practical in the vision measurement applications. The proposed approach greatly reduces the cost of the calibration equipment and simplifies the calibrating procedure.