Figure 1.
Basic concept of robotics active sensing system with in situ adjustable sensor morphology.
In order to sense a possibly unknown target object in uncertain environment, a passive mechanical structure is used by a robotic system to probe the object via suitable motion. A camera will observe this physical interaction and transduce the deformation of the structure due to the arising physical stimuli into useful geometrical information as a sensing output. Based on the output, a fabrication/attachment controller and a motion controller can decide the necessity and the way to adjust the sensor morphology in situ, i.e. the shape, size and connection of the mechanical structures, and/or the suitable motion to initiate different physical interactions. The red lines correspond to the sensing output obtained from the camera, while the green lines correspond to the involved processes during the physical interactions between the robot and the target object.
Figure 2.
Hardware and software implementation of the proposed concept.
(a) Complete workspace of the experiment which includes a robot manipulator equipped with HMA handling units on its end effector (b) The robot’s end effector which is composed of a solid HMA block which is fed to HMA supplier. Fabricated HMA units can be connected to HMA connector. A camera is mounted to perform visual processing tasks during sensing. (c) Software implementation of the proposed approach which is composed of two main parts: the in-situ adjustment of the sensor morphology, and the active sensing via motion (d) Flowchart showing the visual processing algorithm used for softness and temperature case studies.
Figure 3.
Different physical interactions and sensing characteristics enabled by adjusting the sensor morphology, and purposive motion, in situ.
(a) model of the physical interaction for discriminating the softness of the target object (b) corresponding sensing characteristics, i.e. range and sensitivity (c) model of the physical interaction for discriminating the temperature of the object (d) the corresponding sensing characteristics (note: the standard deviation for temperature sensing range is divided by two for the sake of clarity).
Figure 4.
Implementation of autonomous in situ adjustment of sensor morphology.
The whole process includes the construction of the unit which is followed by the gripping of that unit by the HMA connector on the end effector of the robot. (a) Construction and attachment process of the mechanical structure used for discriminating the softness of objects. (b) Construction and attachment process of HMA mechanical structure for discriminating the temperature of objects. Please also see Video S1.
Figure 5.
Implementation of autonomous active sensing via suitable motion.
Each sequence starts with the approaching of the HMA mechanical structure onto the object, followed by the corresponding physical interaction monitored by a visual processing algorithm (a) Image sequences showing the softness discrimination experiment on a sponge block where the deflection angle of the HMA string is calculated. (b) Image sequence of the same experiment for an aluminum block (c) Temperature sensing of the same sponge block which is at a room temperature. (d) Temperature sensing on the aluminum block which is equipped with heaters to adjust the temperature of the object to a fixed level approximately at 120°C. The temperature sensor attached to the connection surface between the HMA connector and the aluminum block shows the gradual increase from room temperature to 59.3. Please also see Video S1.