Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

RRT-CS: A free-collision planner for capsule-like SCORBOT by iterated learning

  • Hung Nguyen,

    Roles Methodology, Supervision

    Affiliation HUTECH Institute of Engineering, HUTECH University, Ho Chi Minh, Vietnam

  • Thanh Phuong Nguyen,

    Roles Methodology, Project administration

    Affiliation HUTECH Institute of Engineering, HUTECH University, Ho Chi Minh, Vietnam

  • Song Hung Nguyen,

    Roles Software, Validation

    Affiliations Faculty of Mechanical Engineering, Ho Chi Minh City University of Technology (HCMUT), Ho Chi Minh, Vietnam, Vietnam National University-Ho Chi Minh City (VNU-HCM), Ho Chi Minh, Vietnam

  • Ha Quang Thinh Ngo

    Roles Conceptualization, Writing – original draft, Writing – review & editing

    nhqthinh@hcmut.edu.vn

    Affiliations Faculty of Mechanical Engineering, Ho Chi Minh City University of Technology (HCMUT), Ho Chi Minh, Vietnam, Vietnam National University-Ho Chi Minh City (VNU-HCM), Ho Chi Minh, Vietnam

Abstract

In this study, we present an enhanced Rapidly-exploring Random Trees (RRT) algorithm integrated with a visual servoing technique for recognizing unknown environments. The robotic platform utilized is the SCORBOT-ER-VII, which consists of five links, servo motors, gearboxes, and an end-effector. Several target objects are used to define the initial position, obstacles, and destination. To evaluate the effectiveness and robustness of our approach, we conducted both numerical simulations and hardware experiments across three test scenarios, ranging from obstacle-free environments to complex obstacle configurations. The results indicate that planning time increases proportionally with scenario complexity. The trajectory smoothing process accounts for less than 10% of the total processing time, while path shortening constitutes one-third, and RRT-based profile generation comprises the remaining two-thirds. These findings clearly demonstrate the efficiency of our approach in terms of computational time, making it well-suited for real-world applications.

1 Introduction

In the field of robotic control, autonomy is a key characteristic for exploring various environments [1,2], including unknown or foggy maps [3], dynamic obstacles, and emergency rescue scenarios [4]. As a fundamental research topic in autonomous robotics, motion planning has attracted significant attention. Most profile generation procedures require searching for multiple collision-free trajectories from the starting point to the destination [57]. In some cases, planning optimization also aims to minimize travel distance while satisfying additional constraints.

Probabilistic Roadmaps (PRM) and Rapidly-exploring Random Trees (RRT) are classified as sampling-based path planning algorithms, where a point is sampled within the working map and its nearest neighbor is explored from the existing tree. In the early stages of development, these methods prioritized rapid planning solutions over optimality. In [8], researchers highlighted that RRT is not asymptotically optimal and proposed Neural RRT*—a novel approach incorporating learning-based capabilities for potentially optimal path planning. This strategy is particularly suitable for practical applications such as autonomous driving and warehouse robotics. In [9], investigators leveraged convolutional neural networks to detect potential collisions while considering an optimal distance metric to estimate sample costs based on path curvature.

In this paper, our contributions are threefold: (i) introducing the robotic platform and related techniques, (ii) developing a motion planning scheme that integrates a robot control strategy with an advanced training model for a computer vision-based approach, and (iii) validating the proposed scheme on a real-world robotic system. The structure of this paper is as follows: Previous Works section reviews related works, followed by a description of the robotic platform, its parameters, the training model, and the camera setup in Preliminaries section. The Proposed Approach section presents the proposed approach, including the conceptual design, motion planner, and post-processing steps. Simulations and Experiments section discusses the experimental results, and Conclusions section concludes the paper with insights and directions for future research.

2 Previous works

Generally, RRT planners could be decomposed into various primitives while alterations or likenesses among them are visible [1013]. Many scholars focused on the parameters and heuristics in sampling-based planners and their implementations in optimizing motion of hardware platform. The existing challenges in the sampling-based path planning are to cost large memory of computer, low speed of convergence, depend on nearest neighbor search and require post processing. On account of these troubles, several efforts for enhancing this motion planner have been made in recent years.

2.1 Sampling algorithm

Sampling process guides robot to extend the configuration space. Because it could spend a lot of time and cost in wide area and not all configurations are uniform, some algorithms have been investigated to overcome such as Voronoi graph [14,15], probabilistic roadmap method [16] or goal biasing [17].

2.2 Autonomous exploration

The ability of exploration becomes one of the key characteristics of robot in the complicated environment. In both visibility region [18] and multi-level region [19], adaptive sampling scheme is useful for autonomous robot even motion planner fails to discover.

2.3 Parameter metric

Most of the sampling-based strategies rely on metrics for their expandable search. Picking a proper metric is always hard for an operator so that it requires a good estimation and better understanding of this system [20,21]. Also, it can be used multiple times during the trajectory planning procedure.

2.4 Collision checking

One of the critical issues to evaluate the motion planner is to generate free-collision path. In some cases, it is classified into which provides free-collision path and which contains the paths colliding with any obstacle. Collision checking would be called several times while robot is moving [2224].

2.5 Robotic singularity

Conventionally, an industrial manipulator could be controlled in both joint space and Cartesian space. For joint-space driving commands, a reference set of joint positions should be provided. Later, it is translated or rotated each joint to the desired joint positions. For Cartesian-space motion commands, both the orientational and positioning pose of robotic gripper must be inserted. Then, motion planner would compute the inverse position and velocity kinematics for all robotic links. In this stage, a robot singularity in which its end-effector becomes blocked in certain direction, might arise. Consequently, there are several techniques to avoid such as vector field [25], geometric method [26] or genetic algorithm [27].

For further analysis, Table 1 summarizes the key limitations and its impact of the state-of-the-art researches in related topics. They are categorized in three groups which represent the separated applications. Then, in the same domain, they are classified owing to the specific techniques or definite design. Each of those researches is investigated to specify the key challenges and its effects in the theme of robotic control.

thumbnail
Table 1. List of the systematic analysis for the state-of-the-art researches.

https://doi.org/10.1371/journal.pone.0323045.t001

3 Preliminaries

To clarify our works, several components including hardware platform, model training and interactive objects, system configuration for both 5-DOF (Degree-Of-Freedom) robot and digital camera.

3.1 Robotic platform

In this system, five DOFs robotic arm as Fig 1 is deployed to manipulate in front of objects. Its mechanical structure is vertical articulated hardware. There are five servo motors and harmonic drives, belt and pulley, and optical encoders to feedback signals. To pick an object, it is necessary to attach one robotic gripper at the end-effector. Optionally, two-finger gripper or three-finger gripper should be considered. However, to demonstrate the obstacle avoidance algorithm, robot arm without gripper is sometimes utilized.

thumbnail
Fig 1. Workspace of the proposed robotic platform, (1) base platform, (2) joint 1, (3) socket head screw M5, (4) joint 3, (5) socket head screw M3 and (6) end-effector.

https://doi.org/10.1371/journal.pone.0323045.g001

3.2 Model training and target objects

Computer vision has attracted a lot of researches in various domains such as robotics, machining design, or information technology. It assists to capture, grasp, and analyze the highly understanding level of visual contents. To match with our purpose, object recognition, localization and segmentation are available to study. Currently, You Only Look Once (YOLO) [39] is very popular to detect an object with significantly high accuracy.

To customize our detector, there are four steps such as data collection, data preparation, model training and model inference. In Fig 2, three kinds of objects are start, obstacle and goal. Dataset is collected from different views and shapes to provide precise detection. Robot must begin its task from start, avoid obstacle and toward to goal. Since these obstacles have rectangular shapes, the point-cloud coordinates of top-view corners of bounding box are A, B, C and D respectively. It is projected four corners down to table surface with depth h = depth of ABCDdepth of table as Fig 3.

To obtain the 3D coordinate, the camera-inspired relative calibration is depicted as Fig 4. For more details, mathematical relations of these parameters are

(1)

where

: intrinsic matrix of digital camera, : focal length, : principal points

: 2D image coordinate

: 3D world coordinate

: extrinsic matrix of digital camera, : camera rotation, : camera translation

Using the Software Development Kit (SDK) of our camera, then intrinsic matrix of those parameters is measured as

(2)

Later, relative transformations should be identified to convert the real-world coordinate to image coordinate on a Charge Coupled Device (CCD) sensor of camera. We assume that X, Y, Z are the actual coordinates of the object (mm) relative to the camera in space and x, y are the coordinates of the pixel (mm) on the CCD optical sensor. In Fig 3, we apply Thales theorem

(3)

We have the conversion matrix of image coordinates on CCD and real coordinates as follows

(4)

In which matrix:

Regularly, the image coordinates would be moved to the left corner of the image and considered as coordinates (0,0). The mm coordinates are converted to pixel coordinates.

(5)(6)

where,

are the pinhole center coordinate

are the length of each pixel in mm in the x and y direction

The formula to convert mm coordinates on the frame to pixel coordinates is as follows

(7)(8)

Hence,

(9)(10)

K: Internal parameter matrix of the camera

We suppose that this camera has coordinates relative to a different origin system. As a result, we have a general formula that transforms from the system

(11)

where,

M: external parameter matrix used to convert object coordinates from global coordinates to camera coordinates. In order to convert a pixel coordinate to real coordinates, we proceed as following

(12)

: pixel coordinate and : global coordinates

Camera distance to the workspace (mm). We have the internal matrix of the camera . In order to convert a point from one coordinate system to another (in this case, change from camera coordinate system to robot coordinate system), we need to find the transfer matrix M

(13)

where,

PR: The known coordinates in advance of the robot coordinate system

PC: The known coordinates in advance of the camera coordinate system

Mtrans: transformation matrix

To determine the M matrix according to above equations, it is necessary to find any four points in local coordinates by moving robotic arm to four different points in the workspace as Fig 5. After that, its values in pixels are recorded by using a camera as Fig 6.

thumbnail
Fig 5. Demonstration of the first step for points taking process.

https://doi.org/10.1371/journal.pone.0323045.g005

thumbnail
Fig 6. Demonstration of the second step for points taking process.

https://doi.org/10.1371/journal.pone.0323045.g006

For avoiding the appearance of outliers, four selected points as Table 2 must have at least one point which is not on the same plane as the remaining points (specifically z much be different).

At that point, applying these parameters to the formular, we get:

(13)

Therefore, we obtain relative parameters during our calibration as Table 3.

thumbnail
Table 3. Results of relative parameters in the process of camera calibration.

https://doi.org/10.1371/journal.pone.0323045.t003

3.3 Forward kinematic

Systematically, forward kinematic challenge could be solved by defining Denavit-Hartenberg (D-H) parameters as Table 4. It is a method to find homogenous transformation matrices among serial links of robot arm as [40]. is the distance from the center of 4th joint to the center of our gripper as Fig 7. Owing to this design, the homogenous transformation matrix : ca is generally well-defined as below

(14)(15)(16)(17)(18)(19)

Thus, the tool coordinate is computed as

(20)

where

3.4 Inverse kinematic

Target solution of inverse kinematic is to find the potential configurations of the set of joint angles θ of this robotic platform when the end-effector towards to destination

(21)

In this stage, all values of homogenous transformation matrix in equation (8) are given. By solving trigonometric equation, we have

(22)(23)(24)(25)

Hence,

(26)

Let

(27)(28)

Then, equation (14) becomes

(29)(30)

We get

(31)(32)(33)(34)

In case that there are multiple inverse kinematic results, unique solution must be voted according to these constraints on the joint limit angle.

3.5 Camera installation

The set-up configuration of vision system is eye2hand method. It consists of two steel bars, metal load and base. Since it looks like console beam, there is a counter-load in the other end of bar while digital camera is hung in one end. This configuration ensures global coverage, stable image.

To identify the dimension of camera standee, some geometric constraints should be considered. With vertical FOV (Field of View), it must cover the working space of robot such . The potential dimension could be estimated as

(35)(36)

In addition, force/moment is analyzed to balance without tipping over. The mass of counter-load must satisfy below condition as

(37)

where

: force by weight of counter-load

: force by weight of steel bar. It locates at the center of bar

: force by weight of digital camera

4 The proposed approach

In this investigation, the proposed design as Fig 8 comprises four components such workspace data capture, motion planner, profile generation and robotic platform. Objects in the working space are captured by a powerful camera and sent to host computer. Later, data is extracted to achieve necessary spatial information of start, goal, and obstacles. Path planner sketches out the candidate traveling routes and elect the shortening collision-free one. Then, motion data is exchanged and interpolated using inverse kinematic procedure. Additionally, for each link, various profiles are generated to ensure the smooth motion in time-space domain. Our design is firstly validated to obtain reference path in Matlab software, consequently command data is input to drive mechanical robot. Whole system could be supervised by an operator via digital camera.

4.1 General concept

Main program which is embedded into microprocessor, involves three fundamental steps. Initially, the surrounding context that could be captured by digital camera, plays as input of sub-program. Secondly, motion planner is triggered to elect the shortening collision-free route in RRT path planning sub-program. Thirdly, driving command is performed in the level of motion controller.

From the visual data, it is fed to YOLO algorithm for object detection involved in path planning. It returns the type of objects, 2D bounding boxes and its centroid pixel coordinate of each box. Moreover, software development kit (SDK) [41] of camera is utilized to get 3D coordinates of multiple point-clouds. Subsequently, it is multiplied with extrinsic matrix to transform these 3D coordinates to the robot basement. According to the type of detected objects as above, the occupied volume in workspace could be determined via its shape and centroid coordinate.

To lessen the heavy computations, robotic platform requires to simplify its links into a simpler geometric model such as ellipsoids [42], cylinders [43], and spheres [44]. In this work, our concept is to deploy a model-based capsule for stimulating the robotic links. This approach is not only well fitted in the shape of robotic arm but also deliver the efficient computations. Due to this design, each solid segment represents the skeleton of robotic manipulator. Similarly, an object or obstacle is also modeled as the shaded capsules and the distance among capsules of the closest pair is very sensitive to avoid collision. We assume that robotic platform at the state occupies its workspace in the Cartesian space . Then, an object with its state also occupies an area in the environment . It is supposed that the Euclidean distance vector from position of robot to position of object : , consequently the norm of relative distance vector between robot and object is identified as

(38)

In Fig 9, it is denoted that robotic links are represented as geometric model with different dimensions, for instance long or short capsule, indicating hardware consumption in workplace. Thus, our concept of capsule-like SCORBOT is proposed as Fig 10. This capsule shape comprises a cylinder with two semi-spherical ends, has the same width as its physical size of link. Henceforth, value becomes distance between two axes. Our robotic platform is modeled as three bounding volumes since they have the highest probability of collision when robot is moving.

thumbnail
Fig 10. Illustration of capsule-like robot in our research.

https://doi.org/10.1371/journal.pone.0323045.g010

4.2 Path planner

The core idea of our path planning is to randomly generate a tree in the searching space, and maintain the Euclidian distance among nodes less than or equal to step size. If a new node might be in collision, it should be ignored and get another new node. This scheme is terminated if the goal is reached, and then the path planning is obtained. Due to our design, the proposed path planner is illustrated as Fig 11. From the start and goal, the step of inverse kinematic returns a random configuration . Goal bias factor is defined as upper limit to improve the planning convergence in faster and higher success rate. If the random probability is equal or less than its value, would be the goal configuration. Otherwise, it would be random configuration in range of and . In fact, this factor should be experimentally tuned based on a trade-off between exploration and convergence. A higher speeds up convergence but increases the risk of local minima, while a lower ensures better exploration but slows convergence. In our best knowledge, the proposed algorithm achieves a 30% faster convergence rate in simple scenarios when equals to . In more complicated environments, our algorithm ensures higher success rates if the value of this factor is .

thumbnail
Fig 11. The proposed path planning scheme for SCORBOT-ER-VII.

https://doi.org/10.1371/journal.pone.0323045.g011

After obtaining , the nearest configuration would be found and the Euclidian distance between and nearest one should be computed. If is greater than step size, becomes the nearest one plus configuration with step size length. Or else, new configuration would be immediately. After that, this configuration would be checked for joint limit and collision conditions.

4.3 Path post processing

In fact, path planning by traditional RRT is not optimal since it has many redundant nodes and sharp turning point as Fig 12. This path is not proper for driving manipulator in term of traveling distance and smoothness. The modified RRT path planning scheme must satisfy two requirements such that reducing path nodes are free collision and sudden change should be avoided for reliably mechanical system and safely human interaction.

thumbnail
Fig 12. Description of traditional RRT path (red color) and post processed RRT path (green color).

https://doi.org/10.1371/journal.pone.0323045.g012

Our motivation to shorten traveling path is to assign a temporary configuration as at start, and then connect with checked for potential collision sequentially. If there is no collision, the validation for next configuration should be kept without changing . Else, changing to previous configuration , add to the reduced tree, and do the same collision-checking procedure for the next configuration . It would be ended when the checking process reaches to the last configuration as Fig 13.

thumbnail
Fig 13. Description of traditional RRT path (green color) and path shortening technique (red color).

https://doi.org/10.1371/journal.pone.0323045.g013

To ensure the smoothness of traveling path, cubic spline interpolation should be deployed. There are two reasons such that the spline curve produces the continuation of the acceleration vector and preserves the sufficient smoothness even in the presence of small curvatures. In Fig 14, this technique is implemented into RRT-CS scheme so that (i) the number of nodes is two, it would process linear interpolation for simplicity, and (ii) node number is larger than two, it fits several cubic spline interpolation curves through those nodes with the same pre-defined traveling time.

thumbnail
Fig 14. Description of traditional RRT path (red color) and post processed RRT path (green color) (a) and path shortening technique (b).

https://doi.org/10.1371/journal.pone.0323045.g014

5 Simulations and experiments

To prove the effectiveness and feasibility of our approach, some test scenarios have been launched as Fig 15. In front of robotic manipulator, there is a bar console to hang one digital camera. The advantages of eye-to-hand camera are to deliver global coverage, stable image and not hinder the motion of robot. Both simulation and experiment as Fig 16 are entirely executed on computer with 4.5GHz CPU Intel i7-6700HQ, 16GB RAM, card GPU NVIDIA GTX960M and Windows OS 11.

Our test scene is inspired by the real-world circumstances, i.e., unknown environment which robot was not taught before. Additionally, the experimental validation of this robotic platform is proceeded in both daylight and night. In these tests, three different scenarios from free-obstacle to hard obstacle are illustrated. The first case as Fig 17 is the simplest one when manipulator only generates motion trajectory from red cube to red sphere, and there is no obstacle. In the second scenario as Fig 18, two yellow rectangular boxes are placed between cube and sphere. Our robotic manipulator must compute more flexible traveling path to avoid yellow boxes. In last case as Fig 19, the number of obstacles is more and its height is greater. To overcome these yellow boxes, robot must try its best to travel smoothly and stably.

In our research, most of validations should be evaluated by three criteria such as completeness, path quality and timing characteristics. In the view of completeness, the success rate is to identify the ability of searching the superior result if possible. Otherwise, it returns failure in finite time. Consequently, the second criterion measures path length and smoothness of trajectory. In addition, traveling time is also important to estimate how good planner is. This criterion is to assess whether motion planner is proper in the real-time applications. An offline planner would capture the environmental situation once, then it generates a path in advance. During the motion execution, it does not need to re-generate traveling trajectory until it ends. Reversely, a real-time planner must update the system state continuously and produce an up-to-date trajectory with very short time. It does not require the intervention of an operator. In the dynamic environment, this criterion becomes a very critical factor for the autonomous system.

In the sampling-based planner, the completeness of path planning means the probability of searching convergence to 1 when the number of iterations tends to infinity. For our works, each test case is validated during 50 times and comparative results are shown as Table 5. Test case 3 has the lowest number of successes runs because more obstacles are added. The first test case is perfect to plan traveling route without any obstacle. In test case 2, not all runs found a good solution and robot still makes an effort to drive.

In Fig 20, path planning result by traditional RRT (red color) is still not optimal because of the random characteristics of motion planner. After using our method, traveling trajectory (green color) is re-constructed and satisfies our requirements. Fig 21 demonstrates the experimental result when our robotic hardware is used. For more details, traveling distance as Fig 22 by the proposed approach is shorter than traditional RRT, approximately 36%. Furthermore, it can be seen visually that in the test case 1, our algorithm releases the smoother curve for each joint as Fig 23 and Fig 24. These results might help to reduce any vibration in the robotic mechanism when it moves rapidly.

thumbnail
Fig 20. Simulation result of the proposed system using our motion planning scheme in case 1.

https://doi.org/10.1371/journal.pone.0323045.g020

thumbnail
Fig 21. Experimental result of the proposed system using our motion planning scheme in case 1.

https://doi.org/10.1371/journal.pone.0323045.g021

thumbnail
Fig 22. Comparative result of path length between our approach (green) and traditional RRT scheme (red) in case 1.

https://doi.org/10.1371/journal.pone.0323045.g022

thumbnail
Fig 23. Experimental result of joint angle using traditional RRT scheme in case 1.

https://doi.org/10.1371/journal.pone.0323045.g023

thumbnail
Fig 24. Experimental result of joint angle using our approach in case 1.

https://doi.org/10.1371/journal.pone.0323045.g024

Similarly, test case 2 as Fig 25 seems to be more difficult because more obstacles are added. Theoretically speaking as Fig 26 and Fig 27, our method generates better traveling route and ensure collision-free motion. Comparing to the other method, the proposed approach achieves shorter path, roughly 21% as Fig 28 and Fig 29. It also means that robot can reach to destination in the earlier time. Besides, path planning by our scheme as Fig 30 and Fig 31 certifies the smoother curves although there exist more fluctuations owing to complex obstacles. Test case 3 as Fig 32 indicates the most complicated challenge for the reason that there are three obstacles between start and target point. Consequently, robot must try its best to overcome these obstacles. In Fig 33, host computer handles larger computations to generate more flexible paths. In our comparison, the proposed approach attains 19% shorter traveling route than traditional method. Likewise, the smoothness of motion profile as Fig 34 is reserved while robot warrants free-collision trajectory.

thumbnail
Fig 25. Experimental result of the proposed system using our motion planning scheme in case 2.

https://doi.org/10.1371/journal.pone.0323045.g025

thumbnail
Fig 26. Simulation result of the proposed system using our motion planning scheme in case 2.

https://doi.org/10.1371/journal.pone.0323045.g026

thumbnail
Fig 27. Simulation result of the proposed system using our motion planning scheme in case 3.

https://doi.org/10.1371/journal.pone.0323045.g027

thumbnail
Fig 28. Comparative result of path length between our approach (green) and traditional RRT scheme (red) in case 2.

https://doi.org/10.1371/journal.pone.0323045.g028

thumbnail
Fig 29. Comparative result of path length between our approach (green) and traditional RRT scheme (red) in case 3.

https://doi.org/10.1371/journal.pone.0323045.g029

thumbnail
Fig 30. Experimental result of joint angle using traditional RRT scheme in case 2.

https://doi.org/10.1371/journal.pone.0323045.g030

thumbnail
Fig 31. Experimental result of joint angle using our approach in case 2.

https://doi.org/10.1371/journal.pone.0323045.g031

thumbnail
Fig 32. Experimental result of the proposed system using our motion planning scheme in case 3.

https://doi.org/10.1371/journal.pone.0323045.g032

thumbnail
Fig 33. Experimental result of joint angle using traditional RRT scheme in case 3.

https://doi.org/10.1371/journal.pone.0323045.g033

thumbnail
Fig 34. Experimental result of joint angle using our approach in case 3.

https://doi.org/10.1371/journal.pone.0323045.g034

Table 6 describes the comparative time between different schemes. From test case 1 to test case 3, total planning time is increased proportionally in respect to the hardness of trial scenarios. In three schemes, smoothness takes the least time, i.e., less than 10%, and can be ignored if necessary. Shortening procedure spends one third portion of total processing time while RRT profile generation costs two third.

Besides, it is essential to compute the quantitative smoothness metrics to provide a rigorous assessment as Table 7. The first indicator is path curvature which measures the deviation of the trajectory from a straight-line path. If a path is perfectly smooth, the change in angle (θ) between consecutive waypoints should be minimal.

(39)

where

: Total number of waypoints in the trajectory

: Heading angle of the robot at waypoint

From equation (39), it is well-recognized that higher values of would indicate more abrupt turns and less smooth motion. If , the path is a perfect straight line. If is large, the trajectory has many sharp turns, signifying jerky motion. Secondly, jerk (J) indicator is defined as the rate of change of acceleration. It represents how quickly acceleration varies.

(40)

where

: Total time duration of motion

: Position of the robot along the trajectory

Jerk indicator illustrates motion stability. If its value is high, the robot accelerates or decelerates abruptly. At that moment, vibrations could occur in this system. Third indicator is total variation of joint angles .

(41)

where

: Number of robot joints

: Joint angle of at waypoint

In equation (41), it measures total changes in joint angles over the trajectory. In the case that high means joints move erratically, leading to unstable motion. Otherwise, low value of indicator specifies joint movements are gradual and smooth. In this study, we have incorporated a comparative quantitative analysis of the conventional RRT [45] and our approach. It could be observed evidently that the proposed strategy gains the improvements in both indicators, and our smoothness evaluation is quantitative, statistically valid, and technically robust.

From those results, it can be seen clearly that our technique enables rapid obstacle detection, adaptive path planning, and trajectory re-generation in dynamic environments. This integration involving RRT scheme and visual servoing technology enhances the robustness and adaptability of robotic motion planning. With the visual approach, it plays a role as the senses of the system, continuously capturing and interpreting data from surrounding. In Table 8, several competitions between our approach and conventional method are carried out.

thumbnail
Table 8. Competitive performance between traditional method and our approach.

https://doi.org/10.1371/journal.pone.0323045.t008

.

Threats to validity.

Although the capsule-like model is effective for collision avoidance by reason of its smooth shape and simplified mathematical representation. Nevertheless, several barriers in handling irregular obstacles and highly dynamic environments could affect on the accurate manipulation and computational efficiency. To summarize the pros and cons of capsule-like model, Table 9 demonstrates the evaluation of possible solutions if our method is utilized. In fact, in the working environment of extremely irregular obstacles, the proposed approach could be implemented the switching mechanism to deploy capsule-like model for simplified objects and complex convex hulls or voxel-based prototype for irregular objects. Secondly, in the narrow space, capsule-based model produces a uniform safety margin which is not proper to varying constraints. An adaptive capsule of sizing design might be suitable for dense obstacles. In the highly dynamic environments, frequent re-computations of capsule-based distance could be computationally expensive. It requires to implement predictive motion model, i.e., Kalman filter or deep learning-based technique to anticipate dynamic object movements and reduce unnecessary re-computation.

thumbnail
Table 9. List of the performance validation for our model and possible solutions in the complicated environments.

https://doi.org/10.1371/journal.pone.0323045.t009

In point of fact, the path shortening and smoothing technique introduce additional computational overhead, but the impact depends on the implementation strategy. Path shortening removes unnecessary nodes from the RRT-generated trajectory by connecting non-consecutive nodes directly while ensuring collision-free movement. Henceforth, path shortening slightly rises computation time but remains feasible with optimizations. In term of path smoothing, it could be performed by using the flexible interpolators such as cubic spline, to generate continuous and differentiable motion trajectories. Therefore, path smoothing is computationally lightweight in most cases and feasible with local fitting methods.

6 Conclusions

In this study, a novel motion planner by using vision-based approach for capsule-like SCORBOT in different scenarios was developed. Primarily, robotic theory including forward kinematic and inverse kinematic, camera installation and system setup were mentioned. Several image processing techniques have been integrated in order to obtain the object coordinate, object recognition and classification. Our concept is to propose the motion planning for capsule-like model, combine the shorten and smoothen profile, as well as vision-based approach. To verify the effectiveness of the proposed method, both simulations and experiments have been accomplished. From these results, it could be seen obviously that our approach is robust, available, and feasible for the real-world applications.

Future work is a must. Our method is available for various industrial manipulators with the intricate grippers. However, with the untrained objects for instance irregular shapes or unknown profiles, it is essential to integrate the advanced machine learning techniques such as deep reinforcement learning or Q-learning for enhanced object shape recognition in cluttered environment. Besides, the adaptive control algorithms, i.e., model predictive control or adaptive neural controller, could be implemented to dynamically tune planning parameters to deal with varying constraints. In addition, collaborative multi-robot system may be explored to extend the applicability of our approach. Also, extensive simulation results with supplementary runs per scenario as well as wider experiments are needed to statistically validate robustness.

Supporting information

Acknowledgments

We acknowledge Ho Chi Minh City University of Technology (HCMUT), VNU-HCM for supporting this study.

References

  1. 1. Reda M, Onsy A, Haikal AY, Ghanbari A. Path planning algorithms in the autonomous driving system: A comprehensive review. Robotics and Autonomous Systems. 2024;174:104630.
  2. 2. Lau BPL, Ong BJY, Loh LKY, Liu R, Yuen C, Soh GS, et al. Multi-AGV’s Temporal Memory-Based RRT Exploration in Unknown Environment. IEEE Robot Autom Lett. 2022;7(4):9256–63.
  3. 3. Lindqvist B, Agha-Mohammadi AA, Nikolakopoulos G. Exploration-RRT: A multi-objective path planning and exploration framework for unknown and unstructured environments. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE; 2021, 3429–35.
  4. 4. Yuan C, Liu G, Zhang W, Pan X. An efficient RRT cache method in dynamic environments for path planning. Robotics and Autonomous Systems. 2020;131:103595.
  5. 5. Wahab MNA, Nefti-Meziani S, Atyabi A. A comparative review on mobile robot path planning: Classical or meta-heuristic methods?. Annual Reviews in Control. 2020;50:233–52.
  6. 6. Salzman O, Halperin D. Asymptotically Near-Optimal RRT for Fast, High-Quality Motion Planning. IEEE Trans Robot. 2016;32(3):473–83.
  7. 7. Zohaib M, Pasha SM, Javaid N, Iqbal J. IBA: Intelligent Bug Algorithm–A novel strategy to navigate mobile robots autonomously. In Communication Technologies, Information Security and Sustainable Development: Third International Multi-topic Conference, IMTIC 2013, Jamshoro, Pakistan, December 18--20, 2013, Revised Selected Papers 3. Springer International Publishing; 2014, 291–9.
  8. 8. Wang J, Chi W, Li C, Wang C, Meng MQ-H. Neural RRT*: Learning-Based Optimal Path Planning. IEEE Trans Automat Sci Eng. 2020;17(4):1748–58.
  9. 9. Yu J, Chen C, Arab A, Yi J, Pei X, Guo X. RDT-RRT: Real-time double-tree rapidly-exploring random tree path planning for autonomous vehicles. Expert Systems with Applications. 2024;240:122510.
  10. 10. Noreen I, Khan A, Habib Z. Optimal Path Planning using RRT* based Approaches: A Survey and Future Directions. ijacsa. 2016;7(11).
  11. 11. Zhang H, Wang Y, Zheng J, Yu J. Path Planning of Industrial Robot Based on Improved RRT Algorithm in Complex Environments. IEEE Access. 2018;6:53296–306.
  12. 12. Zhou C, Huang B, Fränti P. A review of motion planning algorithms for intelligent robots. J Intell Manuf. 2022;33(2):387–424.
  13. 13. Wahab MNA, Nefti-Meziani S, Atyabi A. A comparative review on mobile robot path planning: Classical or meta-heuristic methods?. Annual Reviews in Control. 2020;50:233–52.
  14. 14. Denny J, Qin D, Zhou H. A Fast and Approximate Medial Axis Sampling Technique. 2021 IEEE International Conference on Robotics and Automation (ICRA). 2021:10213–9.
  15. 15. Lathrop P. Motion Planning Algorithms for Safety and Quantum Computing Efficiency (Doctoral dissertation, UC San Diego). 2023.
  16. 16. Marcucci T, Petersen M, von Wrangel D, Tedrake R. Motion planning around obstacles with convex optimization. Sci Robot. 2023;8(84):eadf7843. pmid:37967206
  17. 17. Li Y, Wei W, Gao Y, Wang D, Fan Z. PQ-RRT*: An improved path planning algorithm for mobile robots. Expert Systems with Applications. 2020;152:113425.
  18. 18. Li B, Chen B. An Adaptive Rapidly-Exploring Random Tree. IEEE/CAA J Autom Sinica. 2021;9(2):283–94.
  19. 19. Orthey A, Akbar S, Toussaint M. Multilevel motion planning: A fiber bundle formulation. The International Journal of Robotics Research. 2024;43(1):3–33.
  20. 20. Wong C, Mineo C, Yang E, Yan X-T, Gu D. A Novel Clustering-Based Algorithm for Solving Spatially Constrained Robotic Task Sequencing Problems. IEEE/ASME Trans Mechatron. 2020;26(5):2294–305.
  21. 21. Zhang X, Belfer R, Kry PG, Vouga E. C-Space tunnel discovery for puzzle path planning. ACM Trans Graph. 2020;39(4).
  22. 22. Tu H, Deng Y, Li Q, Song M, Zheng X. Improved RRT global path planning algorithm based on Bridge Test. Robotics and Autonomous Systems. 2024;171:104570.
  23. 23. Li Y, Wei W, Gao Y, Wang D, Fan Z. PQ-RRT*: An improved path planning algorithm for mobile robots. Expert Systems with Applications. 2020;152:113425.
  24. 24. Zohaib M, Pasha SM, Javaid N, Salaam A, Iqbal J. An Improved Algorithm for Collision Avoidance in Environments Having U and H Shaped Obstacles. SIC. 2014;23(1).
  25. 25. Yao W, de Marina HG, Lin B, Cao M. Singularity-Free Guiding Vector Field for Robot Navigation. IEEE Trans Robot. 2021;37(4):1206–21.
  26. 26. Baron N, Philippides A, Rojas N. A robust geometric method of singularity avoidance for kinematically redundant planar parallel robot manipulators. Mechanism and Machine Theory. 2020;151:103863.
  27. 27. Rebouças Filho PP, da Silva SPP, Praxedes VN, Hemanth J, de Albuquerque VHC. Control of singularity trajectory tracking for robotic manipulator by genetic algorithms. Journal of Computational Science. 2019;30:55–64.
  28. 28. Karaman S, Walter MR, Perez A, Frazzoli E, Teller S. Anytime Motion Planning using the RRT*. 2011 IEEE International Conference on Robotics and Automation. 2011:1478–83.
  29. 29. Cheng X, Zhou J, Zhou Z, Zhao X, Gao J, Qiao T. An improved RRT-Connect path planning algorithm of robotic arm for automatic sampling of exhaust emission detection in Industry 4.0. Journal of Industrial Information Integration. 2023;33:100436.
  30. 30. Wang J, Li B, Meng MQ-H. Kinematic Constrained Bi-directional RRT with Efficient Branch Pruning for robot path planning. Expert Systems with Applications. 2021;170:114541.
  31. 31. Wang J, Chi W, Li C, Wang C, Meng MQ-H. Neural RRT*: Learning-Based Optimal Path Planning. IEEE Trans Automat Sci Eng. 2020;17(4):1748–58.
  32. 32. Elfizar E. Implementation of Enhanced Axis Aligned Bounding Box for Object Collision Detection in Distributed Virtual Environment. J Appl Data Sci. 2024;5(3):1286–98.
  33. 33. Wei X, Liu M, Ling Z, Su H. Approximate convex decomposition for 3D meshes with collision-aware concavity and tree search. ACM Trans Graph. 2022;41(4):1–18.
  34. 34. Shah H, Ghadai S, Gamdha D, Schuster A, Thomas I, Greiner N, et al. GPU-Accelerated Collision Analysis of Vehicles in a Point Cloud Environment. IEEE Comput Graph Appl. 2022;42(5):37–50. pmid:35613062
  35. 35. Li C, Zhang X, Gao H, Wang R, Fang Y. Bridging the Gap Between Visual Servoing and Visual SLAM: A Novel Integrated Interactive Framework. IEEE Trans Automat Sci Eng. 2021;19(3):2245–55.
  36. 36. Hu K, Chen Z, Kang H, Tang Y. 3D vision technologies for a self-developed structural external crack damage recognition robot. Automation in Construction. 2024;159:105262.
  37. 37. Tang Y, Qi S, Zhu L, Zhuo X, Zhang Y, Meng F. Obstacle avoidance motion in mobile robotics. Journal of System Simulation. 2024;36(1):1–26.
  38. 38. Ye L, Wu F, Zou X, Li J. Path planning for mobile robots in unstructured orchard environments: An improved kinematically constrained bi-directional RRT approach. Computers and Electronics in Agriculture. 2023;215:108453.
  39. 39. Ngo HQT. Design of automated system for online inspection using the convolutional neural network (CNN) technique in the image processing approach. Results in Engineering. 2023;19:101346.
  40. 40. Ullah MI, Ajwad SA, Irfan M, Iqbal J. Non-linear Control Law for Articulated Serial Manipulators: Simulation Augmented with Hardware Implementation. ElAEE. 2016;22(1).
  41. 41. Nguyen TP, Nguyen H, Ngo HQT. Visual application of navigation framework in cyber-physical system for mobile robot to prevent disease. International Journal of Advanced Robotic Systems. 2023;20(2):172988062311622.
  42. 42. Ruan S, Poblete KL, Wu H, Ma Q, Chirikjian GS. Efficient Path Planning in Narrow Passages for Robots With Ellipsoidal Components. IEEE Trans Robot. 2022;39(1):110–27.
  43. 43. Baxter J, Yousefi MR, Sugaya S, Morales M, Tapia L. Deep prediction of swept volume geometries: Robots and resolutions. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE; 2020, 6665–72.
  44. 44. XUE Z, LIU J, WU C, TONG Y. Review of in-space assembly technologies. Chinese Journal of Aeronautics. 2021;34(11):21–47.
  45. 45. Karaman S, Walter MR, Perez A, Frazzoli E, Teller S. Anytime Motion Planning using the RRT*. 2011 IEEE International Conference on Robotics and Automation. 2011:1478–83.