Fig 1.
(A) The hexapod walking robot HECTOR is inspired by the stick insect Carausius morosus. For its design, the relative positions of the legs as well as the orientation of the legs’ joint axes have been adopted. The size of the robot has been scaled up by a factor of 20 as compared to the biological example which results in an overall length of roughly 0.9 m. All 18 drives for the joints of the six legs are serial elastic actuators. The mechanical compliance of the drives is achieved by an integrated, sensorized elastomer coupling. The bio-inspired control of walking is achieved via a conversion of the WALKNET approach. Bottom view (B) and rendered side view (C) of the front segment of HECTOR. The upper compartment of HECTOR’s front segment has been equipped with a panoramic camera system and an embedded hardware module for processing of visual information. This allows the robot to perform visually-guided tasks such as collision avoidance or navigation.
Fig 2.
(A) The dedicated hardware module for bio-inspired vision processing consists of a carrier board—providing physical interfaces and power-management functionality—as well as an Apalis Zynq processing module. (B) The Apalis Zynq processing module is based on a Zynq-7000 SoC and provides a dual-core ARM Cortex-A9 processor, an Artix-7 based programmable FPGA fabric, as well as an Epiphany multicore coprocessor. The processing module allows the highly efficient implementation of bio-inspired algorithms for the processing of visual information provided by the camera system.
Fig 3.
The controller framework used for implementing the visual collision avoidance model in simulation and on the embedded hardware module [16].
The dashed box (Vision-Based Direction Controller) indicates the algorithm used for controlling HECTOR’s behavior based on nearness estimation from optic flow.
Fig 4.
The bio-inspired vision processing framework implemented in HECTOR is realized by the novel Apalis Zynq CoM (green box).
The processing of the image data is implemented within the Xilinx Zynq SoC (blue box) consisting of a programmable logic (FPGA) and a dual-core CPU (ARM Cortex-A9). The computationally expensive steps of the vision-based direction controller are implemented via FPGA-based IP-Cores while the remaining processing steps are implemented in software on the CPU. (Explanation of abbreviations: CSI: Camera Serial Interface; SPI: Serial Periphal Interface; ReMap: Remapping and downscaling IP-Core; SA: Sensivity Adaption IP-Core; μr: Contrast-weighted relative nearness Map IP-Core; HPF: High-Pass Filter IP-Core; LPF: Low-Pass Filter IP-Core; EMD: Elementary Motion Detector IP-Core; ME: Motion Energy IP-Core; ANV: Average Nearness Vector IP-Core; VDMA: Video Direct Memory Access IP-Core; AXI-Lite: Advanced eXtensible light-weight Interface Bus; AXI-HP: Advanced eXtensible high-performance Interface Bus).
Fig 5.
Body model for generation of stance trajectories in omnidirectional walking.
(A) Body model (top view) with pull points between front leg coxae and hind leg coxae which define the end points of the longitudinal body axis (blue). Control vectors with angle γ w.r.t. the longitudinal body axis are constructed at the pull points. These vectors define the displacement of the pull points (B) and lead to a rotation and shift of the longitudinal axis which is normalized to the robot length afterwards (C). (D) Rotation and shift of longitudinal body axis lead to a displacement of the foot points of all legs with ground contact (displacements calculated with body model). These displacements are used in the single leg controllers to move the robot on a desired trajectory. (E) Exemplary movement of the body axis with constant heading direction γ.
Fig 6.
(A) Software simulation of the experimental setup (top view). A 3D reconstruction of the Teleworkbench was used to optimize the threshold n0 and gain g parameters of the vision-based direction controller in simulation. In the experimental setup visual markers (goal markers) were placed to indicate four different goal positions. The robot was placed 4 m apart and opposite of the goal position facing an object (i.e. a bush) located in the center of the arena. An experimental trial was successful, when the robot reached the goal position without it’s front segment crossing a radius of r = 0.5 m around the object (dashed circle). (B) Experimental trial in the real world. After parameter optimization the visual collision avoidance task was performed on the physical robot. A visual marker was placed on top of the robot’s front segment to obtain the relative direction to the goal α [Eq (8)].
Fig 7.
360° panoramic images of (A) the real environment and (B) the simulated environment reconstructed using photogrammetry. Both images represent the orthographic reprojections using a cylindrical lens positioned at the center of the experimental arena (x, y = [0.0, 0.0]). (C) and (D) depict the color-coded local root-mean-square (RMS) contrast. To compare the local contrast distributions along the horizontal axis for the real and the simulated environments, the vertical mean RMS contrast was computed (E).
Fig 8.
(A) Length of simulated trajectories (color-coded) in the simulated environment (see Fig 6A) for different combinations of the weighting function parameters gain g = [0.0, 0.1, …, 2.0] and threshold n0 = [0.0, 2.0, …, 50.0] [see Eq (9)]. The size of the simulated environment is 7 m x 7 m (length x width). When the trajectory crossed a circle of a radius of 0.5m around the center of the object (dashed line in B-D) a collision was assumed (white areas). B-D) Simulated trajectories (n = 10) in the reconstructed environment. Starting positions are given as S and goal positions as G. Weighting function parameters were set to (B) g = 2.0 and n0 = 0.0, (C) g = 2.0 and n0 = 50.0 and (D) g = 1.0 and n0 = 24.0.
Fig 9.
(A) Trajectories obtained in the real-world scenario for different goal positions (color-coded; G). The dotted circle (r = 0.5 m) indicates the object (bush) located in the center of the arena. The distance between the starting position and the goal position was 4 m. (B) Trajectories obtained in simulation for different goal positions (color-coded; G). For each goal position n = 3 trials were performed. (C) Spatial probability density distribution for all trajectories obtained from simulation. Different combinations of the weighting function parameters [see Eq (9)] threshold n0 = [0.0, 2.0, …, 50.0] and gain g = [0.0, 0.1, …, 2.0] were used. For each parameter combination n = 3 trials were performed. (D) Vertical mean RMS contrast plotted in polar coordinates as also depicted in Fig 7E.
Table 1.
Total FPGA resources for bio-inspired processing.
Table 2.
FPGA IP-core resources for bio-inspired processing.
Table 3.
Comparison of the software and FPGA-based hardware implementation.
Fig 10.
Power dissipation for the implementation of the vision-based direction controller on the Zynq SoC.
Power consumption of the IP-cores implemented on FPGA (Explanation of abbreviations: ReMap: Remapping and downscaling; SA: Sensivity Adaption; HPF: High-Pass Filter; LPF: Low-Pass Filter; EMD: Elementary Motion Detector; ME: Motion Energy; ANV: Average Nearness Vector).