Vehicle autonomous localization in local area of coal mine tunnel based on vision sensors and ultrasonic sensors

Zirui Xu; Wei Yang; Kaiming You; Wei Li; Young-il Kim

doi:10.1371/journal.pone.0171012

Abstract

This paper presents a vehicle autonomous localization method in local area of coal mine tunnel based on vision sensors and ultrasonic sensors. Barcode tags are deployed in pairs on both sides of the tunnel walls at certain intervals as artificial landmarks. The barcode coding is designed based on UPC-A code. The global coordinates of the upper left inner corner point of the feature frame of each barcode tag deployed in the tunnel are uniquely represented by the barcode. Two on-board vision sensors are used to recognize each pair of barcode tags on both sides of the tunnel walls. The distance between the upper left inner corner point of the feature frame of each barcode tag and the vehicle center point can be determined by using a visual distance projection model. The on-board ultrasonic sensors are used to measure the distance from the vehicle center point to the left side of the tunnel walls. Once the spatial geometric relationship between the barcode tags and the vehicle center point is established, the 3D coordinates of the vehicle center point in the tunnel’s global coordinate system can be calculated. Experiments on a straight corridor and an underground tunnel have shown that the proposed vehicle autonomous localization method is not only able to quickly recognize the barcode tags affixed to the tunnel walls, but also has relatively small average localization errors in the vehicle center point’s plane and vertical coordinates to meet autonomous unmanned vehicle positioning requirements in local area of coal mine tunnel.

Citation: Xu Z, Yang W, You K, Li W, Kim Y-i (2017) Vehicle autonomous localization in local area of coal mine tunnel based on vision sensors and ultrasonic sensors. PLoS ONE 12(1): e0171012. https://doi.org/10.1371/journal.pone.0171012

Editor: Pietro Cipresso, IRCCS Istituto Auxologico Italiano, ITALY

Received: July 8, 2016; Accepted: January 13, 2017; Published: January 31, 2017

Copyright: © 2017 Xu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper.

Funding: This work was supported by the Natural Science Foundation of China under Grant (51474015, 51474015) and the Key projects of national key Research and Development program (2016YFC0801806).

Competing interests: The authors have declared that no competing interests exist.

Introduction

Safe mining production is moving towards having few or no people working within the coal mines [1]. Autonomous driving of various vehicles in local areaof coal mine tunnels are important to achieve this goal [2–4]. To achieve autonomous driving, vehicles in local areaof coal mine tunnels require independent localization capability. Large-scale wireless sensor networks can be deployed in underground tunnels, which enable a vehicle to measure the distances to surrounding anchor nodes by using an on-board wireless sensor node, and then locates itself by using a triangulation algorithm [5,6]. The accuracy of this method is greatly affected by the geometry of the triangle. Due to a limited strip-shaped space of the underground tunnel, most locating triangles are flat, leading to relatively high localization errors. In addition, the energy consumption required by wireless sensor nodes prevents them from being used continuously for a long time [7–10]. Therefore, it is difficult to adopt this method in practice.

Vision sensors and relative image process technology are widely used in mobile object positioning due to their advantages such as accurate, low cost, easy usage and a relatively small environmental influence [11–16]. Currently, researches on autonomous positioning in a limited space based on vision sensors, mainly install sensors on the objects which require locating, e.g. the wheel robots. Some researchers concentrated on SLAM (simultaneous localization and mapping) algorithm and its improved algorithm. These algorithms realized the localization by collecting information of surrounding environment and building maps based on vision sensors [17–20]. The MCL(Monte Carlo localization) method, which locates robot position by using recursive estimation, is the most common localization algorithm used in SLAM. Thrun et al [21] proposed the Mixture-MCL, a robust algorithm based on the MCL, which integrates two complimentary ways of generating samples in the estimation. Some other researchers applied vision sensors to recognize landmarks real-timely and matched these landmarks with an on-board database to locate the object [22–24]. E Olson [25] proposed AprilTags Visual Fiducial System and it locates camera’s position by recognizing the AprilTags which were deployed in advance. However, the environments of local area for underground coal mine such as fully mechanized coal mining face may change frequently [26–28]. So, problems such as high complexity and low update rate of map exist for SLAM method. And the method of database landmark is limited by the need to build a huge database in advance.

We propose an autonomous localization method for coal mine vehicle that uses double vision sensors to recognize barcode tags on both sides of the tunnel walls with the help of ultrasonic sensors. Two cameras are fixed to the top center of the vehicle as the vision sensors, and the ultrasonic sensors are mounted below the cameras. Barcode tags containing their own position information in coal mine tunnel are affixed to the walls at both sides of the underground tunnel in pairs. When a vehicle travels in the tunnel, the on-board cameras collect the images of barcode tags on the walls at both sides of the tunnel, and get the barcode tags’ local area position information by recognizing the barcode on images. The distance from the upper left inner corner point of the feature frame of barcode tags on the walls at both sides of the tunnel to the center of cameras, i.e., the center of the vehicle, can be calculated from the ratio between the actual barcode tag width and its width in the CMOS (Complementary Metal Oxide Semiconductor) imaging plane [29]. Since ultrasonic sensors have some advantages such as small size, simple hardware, high accuracy of distance estimation and good performance in complex electromagnetic environment [30], the distance from the vehicle center point to the left tunnel wall is measured by the ultrasonic sensors which are mounted below the cameras. Finally, the 3D coordinates of the vehicle center point in the tunnel’s global coordinate system can be obtained based on the positional relationship between the projection of the upper left inner corner point on the camera CMOS imaging plane and the image center point. The localization method of vehicle autonomous localization in local area of coal mine tunnel based on vision sensors and ultrasonic sensors is a kind of geometric localization. Its calculation complex is much lower than the method of SLAM and it needs not to establish a landmark database in advance, which leads to a good positioning performance in real time.

The rest of this paper is organized as follows: a barcode tag that is suitable for autonomous localization in local area of a mine tunnel vehicle is designed, and a barcode locating and recognition algorithm is proposed in Section 2; a local area mine tunnel vehicle autonomous localization model and a localization algorithm are proposed based on a dual camera distance measurement and ultrasonic sensor distance measurement in Section 3; the proposed vehicle autonomous localization method in local area of coal mine tunnel is tested and evaluated in Section 4; and the relevant conclusion is presented in Section 5.

Barcode tag design and recognition

Fig 1 shows an underground tunnel barcode tag deployment. It should be assumed that the tunnel walls are vertical on both sides, and the width of the tunnel which is represented by W_T is unchanged along underground tunnels in the local area of coal mine. The plane barcode tags are placed in pairs on the walls at both sides of the tunnel at interval distances of W_B. θ is the angle between the lines from any barcode vertex to the two vertexes of the barcode tag of the corresponding pair on the wall on the opposite side of the tunnel, where the line between vertexes on the same side is perpendicular to the tunnel wall. A global coordinate system o_wx_wy_wz_w is established in the tunnel as shown in Fig 1, where the transverse, longitudinal and vertical directions of the straight tunnel segment 1 are respectively the x_w, y_w and z_w axes of the global coordinate system o_wx_wy_wz_w. Due to weak light in the underground tunnel, images collected by ordinary cameras are usually blurred. To effectively recognize the barcode tags affixed to the walls, infrared cameras are installed to ensure effective recognition of barcodes even when there is very weak visible light.

Download:

Fig 1. Illustration of the deployment of barcode tags in a coal mine tunnel.

https://doi.org/10.1371/journal.pone.0171012.g001

Barcode tag design

UPC-A code (Universal Product Code-A) is a continuous barcode used to represent digital information and it can express a large amount of information. UPC-A code is mature, simple and easy to recognize. The barcode tags that are affixed to the tunnel walls use UPC-A code to represent their position coordinates in the global coordinate system. The barcode tag position coordinates are taken as the coordinates of upper left inner corner point of the feature frame of barcode tag.

Fig 2 shows a barcode tag designed based on UPC-A code. The background of the barcode tag is a white rectangle of length L (m) and width H (m). The pattern on the white rectangle is divided into two parts. The first part is the feature frame C, a black frame with lengths of outer sides of L_C and H_C respectively and lengths of inner sides of L_CI and H_CI respectively. As shown in Fig 2, the feature frame C contrasts strongly with the white background, which makes it easy to be extracted by cameras from the white background. In addition, the white color both inside and outside of the frame C increases the accuracy of extraction under usually gloomy light conditions in the coal mine tunnels. In Fig 2, the red arrows mark the upper left inner corner point P, the lower left inner corner point Q, and the upper right inner corner F of the feature frame C.

Download:

Fig 2. Design of barcode tag.

https://doi.org/10.1371/journal.pone.0171012.g002

The second part, G, is the main part of the barcode tag, i.e., the part from the leftmost black bar to the rightmost black bar surrounded by C, which consists of 59 black and white bars. The leftmost three bars (two black bars with a single white bar in between) are the start symbol, while the rightmost three bars are the end symbol, as shown in Fig 2. The five bars in the middle of G form the separation symbol, which is composed of three white bars with two black bars in between. The 24 bars between the start symbol and the separation symbol and the 24 bars between the separation symbol and the end symbol represent a 12-digit number, i.e., each digit is represented by four bars. In Fig 2, each digit represented by four bars has a width of 7b (b is module width), since each bar in the start symbol, the end symbol and the separation symbol has a width of b. Thus the width of the G is 95b. The first four digits, the middle four digits and the last three digits indicate the coordinates of the upper left inner corner point P of the feature frame C along the x_w, y_w and z_w axes respectively in the global coordinate system o_wx_wy_wz_w. The last digit is a check digit which is the remainder of the sum of the first 11 digits divided by 10. This digit is used to check whether the coordinates of P recognized by the camera are correct. The global coordinates of P are encoded as a barcode using the coding rules of UPC-A barcode listed in Table 1. In the coding rules of UPC-A barcode, each digit of the 4-digit number in the encoding column indicates a single bar whose value is the multiple of the module width b. The different combination of 4 bars indicates different number, for example, “3211” indicates number “0” according to the coding rules of UPC-A barcode. During barcode recognition of an image captured by a camera, the same rules are used for decoding.

Download:

Table 1. UPC-A barcode encoding rules.

https://doi.org/10.1371/journal.pone.0171012.t001

Locating and recognition of barcode tags

When a vehicle is traveling in the tunnel as shown in Fig 1, the on-board cameras capture the barcode tags affixed to the walls on both sides of the tunnel and images of some of the surrounding walls. If it is assumed that the resolution of the captured image on the CMOS imaging plane is m × n, i.e., the image pixels are arranged in n rows and m columns, then the pixel at row i(1 ≤ i ≤ n) and column j(1 ≤ j ≤ m) in the image can be represented by p(i, j). If it is assumed that the actual physical size of the image on the CMOS imaging plane is M × N, then each pixel has an actual side length M / N. Fig 3 shows the original image I₀ including a barcode tag that is captured by a camera with a resolution of 1280×960. In this figure, a pixel coordinate system with the upper left corner point as the coordinate origin is established. In this pixel coordinate system, the positive direction of the X-axis is from the image’s left side to its right side, and the positive direction of the Y-axis is from the image’s upper side to its lower side, thus the pixel coordinates of p(i, j) will be X = j, Y = i.

Download:

Fig 3. The original image I₀ captured by a camera.

https://doi.org/10.1371/journal.pone.0171012.g003

After the original image I₀ is obtained, the inner outline of the barcode tag feature frame C in the CMOS imaging plane can be extracted to locate the pattern of C within I₀. Then the barcode tag recognition can be realized by scanning the barcode within the feature frame C.

A. Locating of the barcode feature frame C pattern.

Fig 3 shows the original color image I₀. Only black and white colors are useful to recognize C, so the color image I₀ should be transformed to a gray scale image I₁firstly, as shown in Fig 4. Each pixel in I₁ can only have a pixel value corresponding to a brightness level from 0 to 255. A value of zero is used to indicate the lowest brightness (black color), while 255 represents the highest brightness (white color).

Download:

Fig 4. Gray scale image I₁.

https://doi.org/10.1371/journal.pone.0171012.g004

After obtaining the gray scale image I₁, a binary image I₂ containing only black and white pixels is generated using a binarization process. In image I₂, the pixel values 0 and 1 represent the black pixels and the white pixels respectively. For the binarization process from a gray scale image I₁ to a binary image I₂, a threshold t is used to determine the pixel value in I₂. A value 0 (black color) is given to pixel values less than t, and a value 1 (white color) is given to pixel values greater than or equal to t. To select the threshold value, OTSU algorithm [31–35] is used to maximize the variance of pixel values between different classes. Let n_w(t) and n_b(t) be the number of pixels that have values greater than or equal to the threshold t, or less than the threshold t, respectively, proportional to the total amount of pixels in I₁. Let μ_w(t) and μ_b(t) be the average pixel values of the group of pixel values that are greater than or equal to the threshold t and the group of pixel values less than the threshold t, respectively. Let μ(t) be the overall average pixel value in I₁, and g(t) be the variance between these two classes of pixel values. Then (1) (2)

The threshold t is given a value from 0 to 255 in turn, and the value of t that gives the max (g(t)) is selected as the threshold t. The binary image I₂ after the binarization process of I₁ is shown in Fig 5, where threshold t is given a value of 153 based on the OTSU algorithm.

Download:

Fig 5. The binary image I2.

https://doi.org/10.1371/journal.pone.0171012.g005

As can be seen from Fig 5, after binarization, noise patterns such as the wall texture in the gray scale image I₁ appear as a vast amount of tiny white porphyritic noise in the binary image I₂. These noise patterns may misdirect the camera when locating the pattern of the barcode tag feature frame C and increase the required computation to extract inner outline of C. For this reason, an opening operation is used to eliminate small noise patterns in image I₂ based on erosion followed by dilation. The erosion operation shrinks the boundaries of all white patterns in the image I₂, thus eliminating small white noise patterns such as the wall texture. The dilation operation can then recover the remaining eroded boundaries of the white patterns in I₂. When erosion and dilation are performed in the experiment, a 3 × 3 structure A is first created which can be considered as a binary image with 9 pixels of value 1. A is moved across image I₂ and compared to each pixel p(i,j) and its 8 neighboring pixels N₈(p), i.e. the 8 pixels around p(i,j) which are shown by the yellow area in Fig 6. During the erosion process, if p(i,j) and N₈(p) all have the same value of 1 as A, then pixel p(i,j) is set to value 1; otherwise it is set to 0. During the dilation process, if any element of p(i,j) and N₈(p) has a value of 1, then p(i,j) is set to value 1; otherwise it is set to 0. Fig 7 shows the image I₃ after the opening operation of erosion and dilation. As can be seen from this figure, the opening operation eliminates most of the small white noise patterns such as the wall texture in image I₂.

Download:

Fig 6. 8 neighbor N₈(p) of pixel p(i, j).

https://doi.org/10.1371/journal.pone.0171012.g006

Download:

Fig 7. Resulting image I₃ after opening operation.

https://doi.org/10.1371/journal.pone.0171012.g007

After obtaining image I₃ with most of the white noise patterns eliminated, the outlines of all the remaining white patterns can be extracted. During this extraction, for each white pixel p_w(i, j), a check is performed on whether its 8 neighboring pixels N₈(p) are all white. If any of 8 neighboring pixels is not white, then p_w(i,j) is a boundary pixel. If all 8 neighboring pixels are white, then p_w(i,j) is not a boundary pixel. The extraction is completed once the pixel coordinates of the boundary pixels have been determined. After all of the white pattern outlines in image I₃ have been extracted, their areas are calculated. The outline of white pattern with the largest area is the barcode tag’s white background outline, and the outline of second largest white area is the inner outline of the feature frame C pattern which can be used to locate the black pattern of the feature frame C, as shown in Fig 8. The red line marks the inner outline of the feature frame C.

Download:

Fig 8. Located pattern of the barcode feature frame C.

https://doi.org/10.1371/journal.pone.0171012.g008

Within the CMOS imaging plane, the projection points of the upper left inner corner point P and the lower left inner corner point Q of C are denoted by P′ and Q′, respectively. The length of P′Q′ is used to calculate the distance from P to the camera center point, i.e., the center point of the vehicle in the camera projection model. After the pattern of the barcode feature frame C has been located, the pixel coordinates of P′ and Q′ need to be determined to calculate the length of P′Q′ in the pixel coordinate system. P′ is determined to be the point that has the minimum sum of X and Y pixel coordinates out of all the points on the inner outline of the C pattern, while Q′ is the point with the largest ratio of the difference between its Y pixel coordinate to Y pixel coordinate of P′ and its X pixel coordinate. The pixel coordinates of P′ and Q′ can be calculated using the following formulas (3) and (4) (3) (4) where, x_i and y_i are the X and Y pixel coordinates of a point on the inner outline of the C pattern, respectively. In Fig 8, a green circle and a green cross are used to indicate projection points P′ and Q′ in the CMOS imaging plane, respectively. Fig 9 presents the locating process of the barcode tag feature frame C pattern.

Download:

Fig 9. Locating process of the pattern of the barcode tag feature frame C.

https://doi.org/10.1371/journal.pone.0171012.g009

B. Recognition of barcode information.

After the pattern of C is located in the images obtained by the cameras, progressive scanning is used to recognize the barcode within the pattern C in image I₂. In the binary image I₂, let the leftmost and the rightmost two pixels on the i^th row of the inner outline of the C pattern be p(i, s) and p(i, r). Then the coordinate values on the X axis of the pixels on the i^th row between p(i, s) and p(i, r) are within the range s < j < r. If any pixel p(i, j) and its neighboring pixel p(i, j + 1) have different values (0 or 1), these two pixels are the boundary points of the black and white bars. All boundary points on the i^th row can be determined in the same way. With these boundaries, the width B_u of each of the 59 black or white bars can be calculated. The total width of the 59 barcode bars is (5) where, u is the number of barcode bars. The module width is b = S_B / 95. Each bar has a width of c = B_u / b module width. With the exception of the start symbol on the left, the end symbol on the right and the separation symbol in the middle, each set of 4 bars from the left to the right of the remaining bars are grouped together, and then match with predefined encoding rules to obtain the corresponding digital information. Progressive scanning of the content surrounded by the C pattern gives the coordinates of P(0.3m, 3m, 0.6m). Fig 10 shows the recognition process of the encoded barcode information.

Download:

Fig 10. Recognition process of the encoded barcode information.

https://doi.org/10.1371/journal.pone.0171012.g010

Dual cameras localization model and localization algorithm

Fig 11 illustrates the vehicle autonomous localization in local area ofa coal mine tunnel using dual cameras. P₁ and P₂ are the upper left inner corner points of the feature frame C of the barcode tags affixed to both side walls respectively. In the same tunnel, the upper left inner corner points P of all barcode tag feature frames share the same value H_p on the z_w axis in the global coordinate system o_wx_wy_wz_w. As shown in Fig 11, two cameras are mounted side by side on the vehicle, and the angle between the optical axis and the cross section of the tunnel is α. Since the camera is quite small compared to the tunnel space, the two camera lenses can be considered as a single point O, which is also the center point of the vehicle. The coordinates of O can be denoted as (x_wo, y_wo, z_wo) in the global coordinate system o_wx_wy_wz_w and the height to the tunnel bottom can be denoted as H_o. In the following experiments, three ultrasonic sensors r₁, r₂ and r₃ are installed beneath the left camera to measure the distance from O to the tunnel wall on the left side. The effective measurement angle for each ultrasonic sensor is 40°. The measurement angle of the three ultrasonic sensors can reach almost 120°, which ensures accurate measurement of the distance even when vehicle is turning left or right. When the vehicle is traveling, the image capturing area of the camera is approximately a quadrangular pyramid. When the quadrangular pyramid completely covers a barcode tag on the wall, the camera can locate and recognize the barcode tag using the above locating and recognition algorithm. It can be assumed that when coal mine vehicleat a measurement point, the recognized upper left inner corner points P₁ and P₂ of feature frames of barcode tags affixed to the walls on both sides of the tunnel have global coordinates of P₁ (x_w1, y_w1, z_w1) and P₂ (x_w2, y_w2, z_w2), z_{w1 =} z_w2 = H_p, respectively.

Download:

Fig 11. Vehicle dual camera autonomous localization illustration.

https://doi.org/10.1371/journal.pone.0171012.g011

Distance measurement

For autonomous localization within the local area of tunnel, a traveling vehicle needs to measure the distances D₁ and D₂ from O to the upper left inner corner points P of C on the walls on both sides of the tunnel. Fig 12 shows the projection model of feature frame C of the barcode tag affixed to a tunnel wall in a camera CMOS imaging plane [36, 37]. In this figure, the blue rectangle is a CMOS imaging plane and PQ is the inner side of C with an actual length of H_CI. The actual physical length of line segment from P′ (the projection point of P) to Q′ (the projection point of Q) on the CMOS imaging plane is (6) where, d_pq is the distance between P′ and Q′ in the CMOS imaging plane pixel coordinate system.

Download:

Fig 12. Projection model of the barcode tag feature frame C on the CMOS imaging plane.

https://doi.org/10.1371/journal.pone.0171012.g012

Due to the camera internal structure, the projection point S of the camera center O on the CMOS imaging plane is also the center point of the CMOS imaging plane. OS is the focal length of the camera with a length f. SP′ is the line segment between the projection point P′ and the center point S. Its actual physical length is (7) where, d_sp is the distance between S and P′ in the CMOS imaging plane pixel coordinate system.

Since OS ⊥ SP′, the length of OP′ can be calculated as shown below (8)

Since the triangle OP′Q′ and triangle OPQ are similar, we obtain (9)

Then the distance D between P and O can be calculated using formulas (6), (8) and (9) (10)

Localization

After obtaining the distances D₁ and D₂, the vehicle center point O is within the intersection circle e (with center point E) of the two spheres that have P₁ and P₂ as their centers and D₁ and D₂ as their radii, as shown in Fig 13.

Download:

Fig 13. The circle e containing the vehicle center point O.

https://doi.org/10.1371/journal.pone.0171012.g013

To determine the position of the vehicle center point O within e, the relevant spatial geometric relationships within the tunnel are drawn as shown in Fig 14. In this figure, o_wx_wy_wz_w is the global coordinate system of the tunnel. Since the barcode tags on the left and the right sides are symmetric, and F is the right upper inner corner point of C, P₁F with length W_T, is therefore perpendicular to the tunnel wall. Choosing the minimum distance of all distances between the vehicle center point O and left wall, which were measured respectively by the three ultrasonic sensors at the same time, as d. Do plane V at distance d from the wall on the left side. The plane V intersects with the circle e at points T and T′, which are the two candidates for the vehicle center point O. If P′ is above the CMOS imaging plane center S, i.e., the pixel coordinate y_p>n / 2, then the vehicle center point O is T; otherwise, the center point is T′. R is the midpoint of TT′ in V.

Download:

Fig 14. Spatial geometric diagram for the localization of the vehicle center point O.

https://doi.org/10.1371/journal.pone.0171012.g014

In Fig 14, the coordinate system o_p1x_p1y_p1z_p1 is established with P₁ as its origin, where the x_p1 axis is perpendicular to the tunnel wall, the y_p1 axis is along the longitudinal direction of the tunnel, and the z_p1 axis is along the vertical direction of the tunnel. The coordinate system o_ex_ey_ez_e is established with the center point E of the circle e as its origin, where the x_e axis is along the horizontal direction of the tunnel ground, the y_e axis is perpendicular to the plane of e, and the z_e axis is along the vertical direction of the tunnel.

The coordinates of T and T′ are firstly calculated on the z_w axis in the global coordinate system o_wx_wy_wz_w. In Fig 14, the equation for the plane V in the coordinate system o_p1x_p1y_p1z_p1 is x_p1 = d. The two points K(x_kp, y_kp, z_kp) and J(x_jp, y_jp, z_jp) can be selected on V. They are the intersection points of P₁P₂ and P₁F with V, respectively. Therefore, their coordinates in the coordinate system o_p1x_p1y_p1z_p1 are (d,−dtanθ,0) and (d,0,0), respectively. The transformation relationship from the coordinate system o_p1x_p1y_p1z_p1 to the coordinate system o_ex_ey_ez_e is: (11) where, R_p1e is the rotation matrix from the coordinate system o_p1x_p1y_p1z_p1 to the coordinate system o_ex_ey_ez_e, and T_p1e is a translational matrix.

Therefore, the coordinates of K and J in the coordinate system o_ex_ey_ez_e can be calculated using formula (11) (12) (13)

After obtaining the coordinates of K and J in the coordinate system o_ex_ey_ez_e, the equation of V in o_ex_ey_ez_e can be calculated as follow (14) where, a = P₁E − (d cos θ + d sin θ tan θ).

In the coordinate system o_ex_ey_ez_e, the equation for circle e is (15) where, r is the radius of e, and .

Therefore, the coordinates of T and T′ in the coordinate system o_ex_ey_ez_e can be calculated using formulas (14) and (15) as shown below (16) (17)

Since the z_w axis of the global coordinate system o_wx_wy_wz_w is in the same direction as the z_e axis of the coordinate system o_ex_ey_ez_e, and the upper left inner corner point P of feature frame C in the global coordinate system o_wx_wy_wz_w has a coordinate value H_p on the z_w axis, the coordinate values on the z_w axis of T and T′ in the global coordinate system are (18) (19)

After obtaining these coordinate values, the coordinate value of z_wo on the z_w axis of the vehicle center point O in the global coordinate system o_wx_wy_wz_w can be calculated based on the relationship between pixel coordinate y_p of P′ and n / 2.

In Fig 14, R is the midpoint of TT′. The coordinate values of R on the x_w and y_w axis in the global coordinate system share the same coordinate values as T or T′ on the x_w axis and y_w axis, respectively. The x_e of R in the coordinate system o_ex_ey_ez_e is −a / tan θ, so the length of ER is −a / tan θ. Since ER ⊥ P₁P₂, the lengths of P₁R and P₂R respectively are (20) (21) where, P₂E = W_T/cos θ − P₁E. Since (22) (23) the two plane coordinate values (x_tw,y_tw) and (x_tw′,y_tw′), where y_tw < y_tw′, of T and T′ in the global coordinate system o_wx_wy_wz_w can be calculated based on formulas (22) and (23). As the camera is placed obliquely forward on the center of the vehicle, the coordinates (x_tw,y_tw) that have a smaller value on the y_w axis are the coordinates of T or T′ in the global coordinate system o_wx_wy_wz_w, according to the geometric relationship shown in Fig 14, i.e., the plane coordinates of the vehicle center point O in the global coordinate system is (x_wo,y_wo).

Tests and performance evaluation

Tests in a straight corridor

In order to evaluate the performance of the proposed autonomous localization method for coal mine vehiclein local area using double vision sensors to recognize barcode tags on both sides of tunnel walls with the help of ultrasonic sensors, this method was first tested in a straight corridor that simulates a straight tunnel. The corridor under test had a length of 10m, a width W_T of 1.9m, and a height of 3m, as shown in Fig 15. The global coordinate system o_wx_wy_wz_w was established with the x_w axis, y_w axis and z_w axis along the longitudinal, horizontal and vertical directions of the corridor respectively. Eight pairs of barcode tags were affixed to the walls on both sides of the corridor. The interval distance W_B between any two neighboring barcode tags on the same side was 0.4m. The length L and width H of the tested barcode tags were 0.4m and 0.3m respectively. The actual physical lengths of the inner sides L_CI and H_CI of the feature frame C were 0.343m and 0.168m respectively. The upper left inner corner point P of the frame C was 0.4m high from the ground. The angle θ between the line from the upper left inner corner point P to the corner point P of the adjacent barcode on the opposite side of the wall and the line which is perpendicular to the corridor wall is 10.2°.

Download:

Fig 15. The physical corridor environment.

https://doi.org/10.1371/journal.pone.0171012.g015

The vehicle used in the experiments was a lithium-battery powered crawler car that could be manipulated by mobile phones through WIFI. The car was capable of spinning on the spot, as shown in Fig 16. Two TY803-130 infrared cameras were installed on the test car. The angles α between the optical axes of the two cameras and the perpendicular line to the corridor wall were both 35°. The relevant parameters of the infrared camera are shown in Table 2. The height of the vehicle center point O from the ground was 0.24m. Three JSN-SR04T ultrasonic sensors r₁, r₂ and r₃ were mounted on the pillar beneath the left camera. Their data was read using an Arduino DUE development board. Ultrasonic sensors were used to transmit ultrasonic signals to measure the distance every 200ms. The relevant parameters are shown in Table 3.

Download:

Fig 16. Test car.

https://doi.org/10.1371/journal.pone.0171012.g016

Download:

Table 2. Camera parameters.

https://doi.org/10.1371/journal.pone.0171012.t002

Download:

Table 3. Ultrasonic sensor parameters.

https://doi.org/10.1371/journal.pone.0171012.t003

Fig 17 shows a topology diagram of the test in the corridor, with the black rectangles on both sides showing the barcode tags. In the experiment, the car started from coordinates (1, -0.5) and traveled along the green track until it reached coordinates (1,5). The on-board cameras took 20 pictures in total when the car passed the 10 measurement points marked by crosses. Fig 18 shows the images captured by dual cameras at three of the measurement points at (1, -0.5), (0.5, 4.5) and (1, 5).

Download:

Fig 17. Topology diagram under test in the corridor.

https://doi.org/10.1371/journal.pone.0171012.g017

Download:

Fig 18. The original images captured by the on-board cameras at three of the measurement points.

(a) The image captured at (1, -0.5) on the left side; (b) The image captured at (1, -0.5) on the right side; (c) The image captured at (0.5, 4.5) on the left side; (d) The image captured at (0.5, 4.5) on the right side; (e) The image captured at (1, 5) on the left side; (f) The image captured at (1, 5) on the right side;

https://doi.org/10.1371/journal.pone.0171012.g018

Fig 19 shows the locating results for the images in Fig 18 using the proposed barcode tag locating algorithm at the measurement points (1, -0.5), (0.5, 4.5) and (1, 5). From Fig 19, we can see that the proposed algorithm can effectively locate the pattern of the feature frame C of the barcode tags. A progressive scanning method was then used to recognize the barcode and obtain the coordinates of the upper left inner corner point P in the global coordinate system.

Download:

Fig 19. Locating results of the patterns of barcode tag feature frame C.

(a) Locating of the feature frame C patterns on left side at (1,-0.5); (b) Locating of the feature frame C patterns on right side at (1,-0.5); (c) Locating of the feature frame C patterns on left side at (0.5,4.5); (d) Locating of the feature frame C patterns on right side at (0.5,4.5); (e) Locating of the feature frame C patterns on left side at (1,5); (f) Locating of the feature frame C patterns on right side at (1,5);

https://doi.org/10.1371/journal.pone.0171012.g019

After the feature frame pattern was located and the barcode was recognized, the pixel coordinates of the projection points P′ and Q′ of the upper left inner corner point P and the lower left inner corner point Q on the CMOS imaging plane could be calculated for each pair of barcode tags. The distances D₁ and D₂ from the two upper left inner corner points P of each pair of barcode tags to the vehicle center point O was calculated using formulas (6), (7) and (10). The coordinates of O, (x_wo, y_wo, z_wo), in the global coordinate system were computed based on formulas (18), (19), (22) and (23), with the distance d, from O to the left wall, measured by the ultrasonic sensors.

To evaluate the accuracy of the localization of O using the proposed localization algorithm, the error of z_w axis localization in the global coordinate system was defined as (24) where, z_o(i) is the calculated coordinate of O on the z_w axis at the i^th measurement point and z_r(i) is the actual coordinate of O on the z_w axis. Therefore, the average error of z_w axis localization is (25)

Similarly, the error of plane coordinate localizationfor O in the global coordinate system can be defined as (26) where, x_o(i) and y_o(i) are the calculated coordinates of O on the x_w and y_w axis respectively at the i^th measurement point, while x_r(i) and y_r(i) are the actual coordinates respectively. For U measurement points, the average error of plane coordinate localization for O is (27)

Table 4 shows the coordinates of the upper left inner corner points P₁ and P₂ of C in pairsrecognized at each of the ten measurement points in the global coordinate system o_wx_wy_wz_w, and their localization results and errors using the proposed algorithm.

Download:

Table 4. Localization results and errors.

https://doi.org/10.1371/journal.pone.0171012.t004

The localization results of the vehicle center point O on the z_w axis in Table 4 are plotted in Fig 20. From Fig 20, it can be seen that the proposed localization algorithm can effectively compute the coordinates of O on the z_w axis. The average error on the z_w axis for the ten measurement points is 0.083m, calculated using formula (25). This shows that the proposed algorithm can quite accurately locate the z_w coordinate of the vehicle center O in a straight corridor.

Download:

Fig 20. Localization results of the z_w axis coordinate of the vehicle center O in a straight corridor.

https://doi.org/10.1371/journal.pone.0171012.g020

Fig 21 shows the localization results of the plane coordinates of the vehicle center point, based on the results in Table 4. As can be seen from Fig 21, the proposed algorithm can also effectively compute the plane coordinates at each measurement point. Fig 22 shows the error surface of the plane coordinate localization, where E_p is the plane coordinate localization error of O at each measurement point. As shown in Fig 22, the localization error varies from 0.021m to 0.270m. When the values on the axes x_w and y_w reach 0.5m and 1m respectively, the error E_p reaches the maximum value which is 0.2695m. When the values on the axes x_w and y_w reach 1m and 4m respectively, the error E_p reaches the minimum value which is 0.021m. This is because when the experimental vehicle is at the coordinates (0.5, 0.5), the distance between the camera and the barcode tag on the right wall is minimum. Consequently the length of P′Q′ gets its maximum value, leading to a maximum error of distance D. In a similar way, when the experimental vehicle is at the coordinates (1.1, 4.0), the distances between the cameras and the barcode tag on the both sides of walls are relatively long. Consequently, the length of P′Q′ gets its minimum value, leading to a minimum error of distance D.

Download:

Fig 21. Results of the plane coordinate localization of the vehicle center point O in a straight corridor.

https://doi.org/10.1371/journal.pone.0171012.g021

Download:

Fig 22. Error surface of plane coordinate localization of the vehicle center point O in a straight corridor.

https://doi.org/10.1371/journal.pone.0171012.g022

The average error is calculated as 0.142m using formula (27). This result shows the accuracy of the proposed algorithm for locating the plane coordinates of the vehicle center point O.

Tests in a curved tunnel

The curved tunnel test site was the underground tunnel at the Research Center of Tunnel and Underground Engineering of Beijing Jiaotong University. The actual environment of tunnel was as shown in Fig 23(a) and 23(b). The cross-sectional shape of the tunnel was horseshoe-shaped with a width of 4m. Fig 23(b) shows six pairs of barcode tags symmetrically deployed on both sides of the tunnel. Since there were pipelines on the right tunnel wall, the barcode tags were affixed to the pipelines 0.2m from the right wall, so the actual tunnel width W_T was 3.8m. The length L and width H of the barcode tags in this experiment were 0.8m and 0.43m respectively. The lengths of the inner sides L_CI and H_CI of the feature frame C were 0.665m and 0.325m respectively. The upper left inner corner point P of C was at a height of 0.6m from the ground, and the angle θ between the line between the two upper left inner corner points and the line perpendicular to the tunnel wall was 9.93°.

Download:

Fig 23. Test environment of the underground tunnel.

(a) Real figure of the curved tunnel; (b) Deployment graph of the barcode tags;

https://doi.org/10.1371/journal.pone.0171012.g023

In the curved tunnel test, the height H_o of the vehicle center point O from the ground was 0.35m; and the angle between the optical axis of the two cameras and the line perpendicular to the wall was 30°. The parameters of the cameras and the ultrasonic sensors are the same as the previous straight corridor experiment.

Fig 24 shows the experiment topology in the curved tunnel, with the black rectangles on both sides representing the barcode tags. A global coordinate system o_wx_wy_wz_w was established at the entrance of the tunnel with the x_w, y_w and z_w axis representing the longitudinal, horizontal and vertical directions of the tunnel entrance respectively.

Download:

Fig 24. Experiment topology in the curved tunnel.

https://doi.org/10.1371/journal.pone.0171012.g024

In this test, the vehicle started from the entrance coordinates (2,1), and traveled along the green track until coordinates (9,11). Twelve pictures were captured when passing the six measurement points, which were marked as crosses in Fig 24. Fig 25 shows the images that were collected by the cameras on both sides at three measurement points with coordinates (2, 1), (2.5, 3.5) and (3.5, 5.5).

Download:

Fig 25. The original images collected at the three measurement points by the on-board cameras.

(a) The images collected on left side at the measurement point (2,1); (b) The images collected on right side at the measurement point (2,1); (c) The images collected on left side at the measurement point (2.5,3.5); (d) The images collected on right side at the measurement point (2.5,3.5); (e) The images collected on left side at the measurement point (3.5,5.5); (f) The images collected on right side at the measurement point (3.5,5.5);

https://doi.org/10.1371/journal.pone.0171012.g025

Fig 26 shows the locating results using the proposed algorithm for the images collected in Fig 25 at the measurement points (2,1), (2.5,3.5) and (3.5,5.5). Similarly, from Fig 26, it can be seen that the barcode locating and recognition algorithm can effectively locate the patterns of C. Progressive scanning was then also used to obtain the global coordinates of the upper left inner corner point P of C.

Download:

Fig 26. Locating results of the patterns of barcode tag feature frame C.

(a) Locating of the feature frame C patterns on left side at (2,1); (b) Locating of the feature frame C patterns on right side at (2,1); (c) Locating of the feature frame C patterns on left side at (2.5,3.5); (d) Locating of the feature frame C patterns on right side at (2.5,3.5); (e) Locating of the feature frame C patterns on left side at (3.5,5.5); (f) Locating of the feature frame C patterns on right side at (3.5,5.5);

https://doi.org/10.1371/journal.pone.0171012.g026

Similar to the straight corridor test, the cameras calculated the distances from the vehicle center point O to the upper left inner corner points of feature frame C on both sides of tunnel walls for each measurement point. The distance d from O to the left wall was then acquired from the ultrasound sensors can help to calculate the coordinates (x_wo, y_wo, z_wo) of the vehicle center point O at each measurement point.

Fig 27 shows the localization results of the coordinates in the z_w axis of O in the curved tunnel. Using formula (25), the average error of z_w axis localization of vehicle center point O for the six measurement points was 0.151m. Fig 28 shows the localization results of the plane coordinates for the six measurement points using the proposed algorithm. Using formula (27), the average error of plane coordinate localization was 0.381m. Therefore, the proposed localization algorithm is quite accurate and feasible in curved tunnels for both locating the vehicle center point on the z_w axis and locating the plane coordinates.

Download:

Fig 27. Localization results of the vehicle center point O on the z_w axis in the curved tunnel.

https://doi.org/10.1371/journal.pone.0171012.g027

Download:

Fig 28. Results of the vehicle center point O plane coordinate localization in the curved tunnel.

https://doi.org/10.1371/journal.pone.0171012.g028

Conclusions

For autonomous vehicles to travel in local area of a coal mine tunnel, a self-localization capability is required. We propose an autonomous localization method in local area of coal mine tunnel that uses on-board vision sensors to recognize barcode tags affixed to the tunnel walls in pairs and ultrasonic sensors to measure the distance to the tunnel walls.

Using the proposed barcode tag locating and recognition algorithm, the vehicle vision sensors effectively located the feature frame from the collected images containing barcode tags affixed to the tunnel walls on both sides and were able to recognize the barcode tags.
The spatial geometric relationship between the vehicle center point and the barcode tags was established based on the distance between the vision sensors and the upper left inner corner points of the barcode tag feature frame in pairs and the distance from the vehicle center point to the left side tunnel wall. The 3D coordinates of the vehicle center point could then be calculated in the tunnel’s global coordinate system.
For evaluation of the proposed algorithm, experimental results in a straight corridor have shown that average error of z_w axis localization is 0.08m for locating the vehicle center point and average error of plane coordinate localization is 0.142m for locating the vehicle center point. Experimental results in the curved underground tunnel have shown that error of z_w axis localization is 0.151m for locating the vehicle center point and average error of plane coordinate localization is 0.381m for locating the vehicle center point.

Author Contributions

Conceptualization: ZX.
Data curation: ZX.
Formal analysis: ZX WL.
Funding acquisition: WY.
Investigation: ZX KY.
Methodology: ZX WY YK.
Resources: ZX WY.
Software: ZX WL.
Supervision: WY.
Validation: WY ZX.
Visualization: ZX WY.
Writing – original draft: ZX WY.
Writing – review & editing: ZX WY.

References

1. Wang J. Development and prospect on fully mechanized mining in Chinese coal mines. J CHINA COAL SOC. 2014; 1(3):253–60.
- View Article
- Google Scholar
2. Huh S, Lee U, Shim H, Park JB, Noh JH, editors. Development of an unmanned coal mining robot and a tele-operation system. Control, Automation and Systems (ICCAS), 2011 11th International Conference on; 2011.
3. Ahmad R, Singh A, Muzamil R, Beg MS, editors. Use of WSN for detection of metal landmines using unmanned vehicle. Indiacom; 2015.
4. Brown C. Autonomous vehicle technology in mining. ENG MIN J, 2012, 213(1): 30.
- View Article
- Google Scholar
5. Asmaa L, Hatim KA, Abdelaaziz M. Localization algorithms research in wireless sensor network based on Multilateration and Trilateration techniques. 2014 Third IEEE International Colloquium in Information Science and Technology (CIST); 2014: IEEE.
6. Cheng G. Accurate TOA-based UWB localization system in coal mine based on WSN. Physics Procedia. 2012; 24:534–40.
- View Article
- Google Scholar
7. Wang H, Agoulmine N, Ma M, Jin Y. Network lifetime optimization in wireless sensor networks. IEEE J SEL AREA COMM. 2010; 28(7):1127–37.
- View Article
- Google Scholar
8. Niu X, Huang X, Zhao Z, Zhang Y, Huang C, Cui L. The design and evaluation of a wireless sensor network for mine safety monitoring. IEEE GLOBECOM 2007-IEEE Global Telecommunications Conference; 2007: IEEE.
9. Usha D, Paramasivan B. Multi-Velocity Energy-Efficient Reliable Routing Protocol for Coal Mine Monitoring. IJCA. 2014; 87(3).
- View Article
- Google Scholar
10. Razzak MI, Hussain S, Minhas AA, Sher M. Collaborative image compression in wireless sensor networks. IJCC. 2010;8(1).
- View Article
- Google Scholar
11. You K, Yang W, Han R. The video collaborative localization of a miner’s lamp based on wireless multimedia sensor networks for underground coal mines. Sensors. 2015; 15(10):25103–22. pmid:26426023
- View Article
- PubMed/NCBI
- Google Scholar
12. Liu L, Zhang X, Ma H. Optimal node selection for target localization in wireless camera sensor networks. IEEE T VEH TECHNOL. 2010; 59(7):3562–76.
- View Article
- Google Scholar
13. Li W, Portilla J, Moreno F, Liang G, Riesgo T. Improving target localization accuracy of wireless visual sensor networks. IECON 2011-37th Annual Conference on IEEE Industrial Electronics Society; 2011: IEEE.
14. Karakaya M, Qi H. Collaborative localization in visual sensor networks. ACM T SENSOR NETWORK. 2014; 10(2):18.
- View Article
- Google Scholar
15. Straw AD, Branson K, Neumann TR, Dickinson MH. Multi-camera real-time three-dimensional tracking of multiple flying animals. J R SOC INTERFACE. 2010:rsif20100230.
- View Article
- Google Scholar
16. Mehta S, Burks T, Dixon W. Vision-based localization of a wheeled mobile robot for greenhouse applications: A daisy-chaining approach. COMPUT ELECTRON AGR. 2008; 63(1):28–37.
- View Article
- Google Scholar
17. Lee D, Kim D, Lee S, Myung H, Choi H-T. Experiments on localization of an AUV using graph-based SLAM. Ubiquitous Robots and Ambient Intelligence (URAI), 2013 10th International Conference on; 2013: IEEE.
18. Fu M, Zhu H, Yang Y, Wang M, Li Y. Multiple map representations for vehicle localization and scene reconstruction. 17th International IEEE Conference on Intelligent Transportation Systems (ITSC); 2014: IEEE.
19. Trivun D, Šalaka E, Osmanković D, Velagić J, Osmić N. Active SLAM-based algorithm for autonomous exploration with mobile robot. Industrial Technology (ICIT), 2015 IEEE International Conference on; 2015: IEEE.
20. Zhang L, Shen P, Zhu G, Wei W, Song H. A Fast Robot Identification and Mapping Algorithm Based on Kinect Sensor. Sensors. 2015; 15(8):19937–67. pmid:26287198
- View Article
- PubMed/NCBI
- Google Scholar
21. Thrun S, Fox D, Burgard W, Dellaert F. Robust Monte Carlo localization for mobile robots. ARTIF INTELL. 2001, 128(1–2):99–141
- View Article
- Google Scholar
22. Chen K-C, Tsai W-H. Vision-based autonomous vehicle guidance for indoor security patrolling by a SIFT-based vehicle-localization technique. IEEE T VEH TECHNOL. 2010; 59(7):3261–71.
- View Article
- Google Scholar
23. Zhou Y, Liu W, Huang P. Laser-activated RFID-based indoor localization system for mobile robots. Proceedings 2007 IEEE International Conference on Robotics and Automation; 2007.
24. Chen X, Jia Y. Indoor localization for mobile robots using lampshade corners as landmarks: Visual system calibration, feature extraction and experiments. INT J CONTROL AUTOM. 2014; 12(6):1313–22.
- View Article
- Google Scholar
25. Olson E. AprilTag: A robust and flexible visual fiducial system. 2011 IEEE International Conference on Robotics and Automation. 2011:3400–3407.
26. Sasmito AP, Birgersson E, Ly HC, Mujumdar AS. Some approaches to improve ventilation system in underground coal mines environment–A computational fluid dynamic study. TUNN UNDERGR SP TECH. 2013; 34:82–95.
- View Article
- Google Scholar
27. Zhang Y, Yang W, Han D, Kim Y-I. An integrated environment monitoring system for underground coal mines—wireless Sensor Network subsystem with multi-parameter monitoring. Sensors. 2014; 14(7):13149–70. pmid:25051037
- View Article
- PubMed/NCBI
- Google Scholar
28. Forooshani AE, Bashir S, Michelson DG, Noghanian S. A survey of wireless communications and propagation modeling in underground mines. IEEE COMMUN SURV TUT. 2013;15(4):1524–45.
- View Article
- Google Scholar
29. Cossairt O, Gupta M, Nayar SK. When does computational imaging improve performance? IEEE T IMAGE PROCESS. 2013; 22(2):447–58.
- View Article
- Google Scholar
30. Hirata S, Kurosawa M K, Katagiri T. Real-time ultrasonic distance measurements for autonomous mobile robots using cross correlation by single-bit signal processing. 2009 IEEE International Conference on Robotics and Automation; 2009:3601–3606.
31. Chauhan AS, Silakari S, Dixit M, editors. Image Segmentation Methods: A Survey Approach. 2014 Fourth International Conference on Communication Systems and Network Technologies; 2014.
32. Fabijanska A, editor A survey of thresholding algorithms on yarn images. 2010 Proceedings of VIth International Conference on Perspective Technologies and Methods in MEMS Design; 2010.
33. Dirami A, Hammouche K, Diaf M, Siarry P. Fast multilevel thresholding for image segmentation through a multiphase level set method. Signal Process. 2013; 93(1):139–53.
- View Article
- Google Scholar
34. Bansal S, Maini R. A Comparative Analysis of Iterative and Ostu'sThresholding Techniques. IJCA. 2013; 66(12).
- View Article
- Google Scholar
35. Jun O. A threshold selection method from gray-scale histograms. IEEE T SYST MAN CY-S. 1988(1):62–6.
- View Article
- Google Scholar
36. Fiala M, editor Automatic projector calibration using self-identifying patterns. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05)-Workshops; 2005: IEEE.
37. Rigler R, Mets Ü, Widengren J, Kask P. Fluorescence correlation spectroscopy with high count rate and low background: analysis of translational diffusion. EUR BIOPHYS J BIOPHY. 1993; 22(3):169–75.
- View Article
- Google Scholar

[ref1] 1. Wang J. Development and prospect on fully mechanized mining in Chinese coal mines. J CHINA COAL SOC. 2014; 1(3):253–60.
View Article
Google Scholar

[2] View Article

[3] Google Scholar

[ref2] 2. Huh S, Lee U, Shim H, Park JB, Noh JH, editors. Development of an unmanned coal mining robot and a tele-operation system. Control, Automation and Systems (ICCAS), 2011 11th International Conference on; 2011.

[ref3] 3. Ahmad R, Singh A, Muzamil R, Beg MS, editors. Use of WSN for detection of metal landmines using unmanned vehicle. Indiacom; 2015.

[ref4] 4. Brown C. Autonomous vehicle technology in mining. ENG MIN J, 2012, 213(1): 30.
View Article
Google Scholar

[7] View Article

[8] Google Scholar

[ref5] 5. Asmaa L, Hatim KA, Abdelaaziz M. Localization algorithms research in wireless sensor network based on Multilateration and Trilateration techniques. 2014 Third IEEE International Colloquium in Information Science and Technology (CIST); 2014: IEEE.

[ref6] 6. Cheng G. Accurate TOA-based UWB localization system in coal mine based on WSN. Physics Procedia. 2012; 24:534–40.
View Article
Google Scholar

[11] View Article

[12] Google Scholar

[ref7] 7. Wang H, Agoulmine N, Ma M, Jin Y. Network lifetime optimization in wireless sensor networks. IEEE J SEL AREA COMM. 2010; 28(7):1127–37.
View Article
Google Scholar

[14] View Article

[15] Google Scholar

[ref8] 8. Niu X, Huang X, Zhao Z, Zhang Y, Huang C, Cui L. The design and evaluation of a wireless sensor network for mine safety monitoring. IEEE GLOBECOM 2007-IEEE Global Telecommunications Conference; 2007: IEEE.

[ref9] 9. Usha D, Paramasivan B. Multi-Velocity Energy-Efficient Reliable Routing Protocol for Coal Mine Monitoring. IJCA. 2014; 87(3).
View Article
Google Scholar

[18] View Article

[19] Google Scholar

[ref10] 10. Razzak MI, Hussain S, Minhas AA, Sher M. Collaborative image compression in wireless sensor networks. IJCC. 2010;8(1).
View Article
Google Scholar

[21] View Article

[22] Google Scholar

[ref11] 11. You K, Yang W, Han R. The video collaborative localization of a miner’s lamp based on wireless multimedia sensor networks for underground coal mines. Sensors. 2015; 15(10):25103–22. pmid:26426023
View Article
PubMed/NCBI
Google Scholar

[24] View Article

[25] PubMed/NCBI

[26] Google Scholar

[ref12] 12. Liu L, Zhang X, Ma H. Optimal node selection for target localization in wireless camera sensor networks. IEEE T VEH TECHNOL. 2010; 59(7):3562–76.
View Article
Google Scholar

[28] View Article

[29] Google Scholar

[ref13] 13. Li W, Portilla J, Moreno F, Liang G, Riesgo T. Improving target localization accuracy of wireless visual sensor networks. IECON 2011-37th Annual Conference on IEEE Industrial Electronics Society; 2011: IEEE.

[ref14] 14. Karakaya M, Qi H. Collaborative localization in visual sensor networks. ACM T SENSOR NETWORK. 2014; 10(2):18.
View Article
Google Scholar

[32] View Article

[33] Google Scholar

[ref15] 15. Straw AD, Branson K, Neumann TR, Dickinson MH. Multi-camera real-time three-dimensional tracking of multiple flying animals. J R SOC INTERFACE. 2010:rsif20100230.
View Article
Google Scholar

[35] View Article

[36] Google Scholar

[ref16] 16. Mehta S, Burks T, Dixon W. Vision-based localization of a wheeled mobile robot for greenhouse applications: A daisy-chaining approach. COMPUT ELECTRON AGR. 2008; 63(1):28–37.
View Article
Google Scholar

[38] View Article

[39] Google Scholar

[ref17] 17. Lee D, Kim D, Lee S, Myung H, Choi H-T. Experiments on localization of an AUV using graph-based SLAM. Ubiquitous Robots and Ambient Intelligence (URAI), 2013 10th International Conference on; 2013: IEEE.

[ref18] 18. Fu M, Zhu H, Yang Y, Wang M, Li Y. Multiple map representations for vehicle localization and scene reconstruction. 17th International IEEE Conference on Intelligent Transportation Systems (ITSC); 2014: IEEE.

[ref19] 19. Trivun D, Šalaka E, Osmanković D, Velagić J, Osmić N. Active SLAM-based algorithm for autonomous exploration with mobile robot. Industrial Technology (ICIT), 2015 IEEE International Conference on; 2015: IEEE.

[ref20] 20. Zhang L, Shen P, Zhu G, Wei W, Song H. A Fast Robot Identification and Mapping Algorithm Based on Kinect Sensor. Sensors. 2015; 15(8):19937–67. pmid:26287198
View Article
PubMed/NCBI
Google Scholar

[44] View Article

[45] PubMed/NCBI

[46] Google Scholar

[ref21] 21. Thrun S, Fox D, Burgard W, Dellaert F. Robust Monte Carlo localization for mobile robots. ARTIF INTELL. 2001, 128(1–2):99–141
View Article
Google Scholar

[48] View Article

[49] Google Scholar

[ref22] 22. Chen K-C, Tsai W-H. Vision-based autonomous vehicle guidance for indoor security patrolling by a SIFT-based vehicle-localization technique. IEEE T VEH TECHNOL. 2010; 59(7):3261–71.
View Article
Google Scholar

[51] View Article

[52] Google Scholar

[ref23] 23. Zhou Y, Liu W, Huang P. Laser-activated RFID-based indoor localization system for mobile robots. Proceedings 2007 IEEE International Conference on Robotics and Automation; 2007.

[ref24] 24. Chen X, Jia Y. Indoor localization for mobile robots using lampshade corners as landmarks: Visual system calibration, feature extraction and experiments. INT J CONTROL AUTOM. 2014; 12(6):1313–22.
View Article
Google Scholar

[55] View Article

[56] Google Scholar

[ref25] 25. Olson E. AprilTag: A robust and flexible visual fiducial system. 2011 IEEE International Conference on Robotics and Automation. 2011:3400–3407.

[ref26] 26. Sasmito AP, Birgersson E, Ly HC, Mujumdar AS. Some approaches to improve ventilation system in underground coal mines environment–A computational fluid dynamic study. TUNN UNDERGR SP TECH. 2013; 34:82–95.
View Article
Google Scholar

[59] View Article

[60] Google Scholar

[ref27] 27. Zhang Y, Yang W, Han D, Kim Y-I. An integrated environment monitoring system for underground coal mines—wireless Sensor Network subsystem with multi-parameter monitoring. Sensors. 2014; 14(7):13149–70. pmid:25051037
View Article
PubMed/NCBI
Google Scholar

[62] View Article

[63] PubMed/NCBI

[64] Google Scholar

[ref28] 28. Forooshani AE, Bashir S, Michelson DG, Noghanian S. A survey of wireless communications and propagation modeling in underground mines. IEEE COMMUN SURV TUT. 2013;15(4):1524–45.
View Article
Google Scholar

[66] View Article

[67] Google Scholar

[ref29] 29. Cossairt O, Gupta M, Nayar SK. When does computational imaging improve performance? IEEE T IMAGE PROCESS. 2013; 22(2):447–58.
View Article
Google Scholar

[69] View Article

[70] Google Scholar

[ref30] 30. Hirata S, Kurosawa M K, Katagiri T. Real-time ultrasonic distance measurements for autonomous mobile robots using cross correlation by single-bit signal processing. 2009 IEEE International Conference on Robotics and Automation; 2009:3601–3606.

[ref31] 31. Chauhan AS, Silakari S, Dixit M, editors. Image Segmentation Methods: A Survey Approach. 2014 Fourth International Conference on Communication Systems and Network Technologies; 2014.

[ref32] 32. Fabijanska A, editor A survey of thresholding algorithms on yarn images. 2010 Proceedings of VIth International Conference on Perspective Technologies and Methods in MEMS Design; 2010.

[ref33] 33. Dirami A, Hammouche K, Diaf M, Siarry P. Fast multilevel thresholding for image segmentation through a multiphase level set method. Signal Process. 2013; 93(1):139–53.
View Article
Google Scholar

[75] View Article

[76] Google Scholar

[ref34] 34. Bansal S, Maini R. A Comparative Analysis of Iterative and Ostu'sThresholding Techniques. IJCA. 2013; 66(12).
View Article
Google Scholar

[78] View Article

[79] Google Scholar

[ref35] 35. Jun O. A threshold selection method from gray-scale histograms. IEEE T SYST MAN CY-S. 1988(1):62–6.
View Article
Google Scholar

[81] View Article

[82] Google Scholar

[ref36] 36. Fiala M, editor Automatic projector calibration using self-identifying patterns. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05)-Workshops; 2005: IEEE.

[ref37] 37. Rigler R, Mets Ü, Widengren J, Kask P. Fluorescence correlation spectroscopy with high count rate and low background: analysis of translational diffusion. EUR BIOPHYS J BIOPHY. 1993; 22(3):169–75.
View Article
Google Scholar

[85] View Article

[86] Google Scholar

Figures

Abstract

Introduction

Barcode tag design and recognition

Barcode tag design

Locating and recognition of barcode tags

A. Locating of the barcode feature frame C pattern.

B. Recognition of barcode information.

Dual cameras localization model and localization algorithm

Distance measurement

Localization

Tests and performance evaluation

Tests in a straight corridor

Tests in a curved tunnel

Conclusions

Author Contributions

References