Fig 1.
Overview of elastic semi-circular biological structures.
Examples of different species with elastic endogenous and exogenous biological structures that are trackable as semi circles with the approaches described in this report. From top-left to top right, guineafowl puffer (Arothron meleagris), greater sage-grouse (Centrocercus urophasianus), green tree frog (Hyla cinerea), prairie chicken (Tympanuchus cupido), magnificent frigatebird (Fregata magnificens), siamang (Symphalangus syndactylys), elephant seal (Mirounga angustirostris). All photos are public domain.
Fig 2.
A) Audio and video data were collected in the Jaderpark Tier- und Freizeitpark an der Nordsee, Germany. B) Air sacs were automatically tracked with two approaches: the lesser performing Hough Transformation (see sample here) versus the very good performing DeepLabCut tracking (see here sample) with Landau Circle Estimation (DLC+: see here for a sample). C) For a subset of the data, air sacs were tracked manually and compared to the automatically tracked radii. DLC estimated radii had a high correlation of r > 0.8 with the manually tracked radii. D) Acoustic parameters of two different kinds of calls were analyzed and related with air sac inflation as a proof of concept. E) All data and code are shared open access. The figure was created with BioRender. All images are photos by Wim Pouw and Mounia Kehy.
Table 1.
Information on individuals: Sex, age class, and age for all siamang at Jaderpark are given.
Data were only used from five of the six individuals; Tristan, the newborn, was not considered.
Fig 3.
Image processing to increase Hough Transform circle detection success.
Four pre-processing steps in four different frames with various backgrounds are shown to illustrate the necessity of the preprocessing steps for the Hough Transform approach to work. Only after the last pre-processing step are air sacs correctly found with the Hough Transform. All images are photos by Wim Pouw and Mounia Kehy.
Fig 4.
Example of our Hough Transform approach.
Examples: Hough Transform tool applied to a video of a siamang (see here) and a green tree frog (Hyla cinerea; see sample here), with satisfactory performance for some purposes such as position tracking or large-scale changes in inflation (but after investigation of tracking performances we favor DLC+). The left image contains a photo by Wim Pouw and Mounia Kehy. The picture on the right is from the youtube video and analyzed by us for testing.
Table 2.
Parameters iterated for parameter optimization in Hough Transform.
After initial testing, six parameters were chosen to iterate over different setting combinations to find the best performance. The best performance was found by correlating the found radius results to the manually tracked radii and choosing the parameter combination, which showed the highest correlation over all videos.
Fig 5.
DLC labeling approach for DLC+.
The five points to be tracked on the air sac outline are depicted. Note that two outline ends, which we call fixed points, are clearly defined. The other three points are defined relative to these fixed points. Firstly, it is the middle point between the start and end point (vertically defined by being on the edge of the air sac). The two other points are again the middle of the start or end and the adjacent middle point. In this way, we ensure that DLC can always determine the points, even when some are not determinable, without relating them to the two fixed start and end points. As such, some points are only relationally but nevertheless systematically definable. Photos taken by Wim Pouw and Mounia Kehy.
Fig 6.
Automatically tracked radii in comparison to manually labeled radii for both approaches: DLC+ and Hough Transform.
A) Comparison of DLC+ trackings and manual tracking of air sac radii, both in pixel units, for a set of > 1000 frames from nine different video scenes. Perfect fit would mean r = 1.0 and direct match in pixels. We find that the DLC+ radii match very well with the manual trackings, r = 0.86, as calculated on all frames over nine videos. B) Comparison of Hough Transform tracking and manual tracking for the same set of > 1000 frames. The correlation coefficient (r) for the nine test videos was 0.23. Parameters can be optimized for each video specifically, and when set adequately for individual videos, we see correlations close to the one for DLC+ trackings. As a trendline, in turquoise, we see the second-best correlation for one video with r = 0.53; in red, the best correlation with r = 0.8.
Table 3.
Reliability comparison with manually tracked ground-truth data between tracking algorithms.
For comparison, only automatically tracked radii below 270 px were used, as this is the maximum radius tracked with the Hough Transform and approximately the maximum radius found manually (max: 266 px). This also applies to smoothed radii. While smoothing increases tracking success for Hough Transform, tracking works equally well, if not better, without smoothing using DLC+ tracking. Notice that these correlations feature the whole subset of data. The Hough Transform works very well for particular examples; DLC+ works better on average and, across the board, has a high performance.
Fig 7.
Significant correlations between air sac inflation (as air sac radius in pixel) and acoustic parameters.
The figure is divided into results of adult siamangs (left; panel a and b) and immature siamangs (right; panel c and d). The top panel shows the pooled data. Air sac radius in pixel on the x-axis is shown against four different acoustic parameters (from left to right): sound amplitude, f0 or fundamental frequency in Hertz, Wiener entropy and spectral Centroid in Hertz. For adults, we find clear correlations: The more the air sac is inflated, the higher the sound amplitude produced. This is in line with model predictions (see [34]). The fundamental frequency (f0) is also positively correlated with the air sac inflation, and seems to be the only stable relationship reflected in both the male and female individual. Mean Entropy is negatively correlated with air sac inflation, meaning, the more inflated the air sac is, the more tonal the produced sound. The Spectral Centroid is negatively correlated as well, indicating the higher the inflation, the more energy in the lower frequencies. We do not see those relationships in immature not fully grown individuals. In the middle panel significant correlation coefficients for all tested acoustic parameters are shown. Notice, that none of the acoustic parameters showed a significant relation in immatures (indicated by white circles). The bottom panel divides the pooled data. For adults we divide by sex into male and female, for the immatures we divide by ageclass into subadult and juvenile.
Fig 8.
Relation between air sac inflation for a boom call, with acoustics of bark immediately following the boom.
The top panel shows the relationship between the last observed radius of the air sac during a boom call, plotted against the subsequent bark’s acoustic parameters. A) shows the results for the acoustic parameter mean sound amplitude, which does not relate to the inflation of the air sac in this case. B) shows the relation of the mean spectral centroid (given in Hertz) of the bark relative to the inflation of the preceding boom. In C), correlation estimates are shown for the suite of acoustic parameters included in these exploratory analyses, but only statistically significant correlation coefficients are shown. Mean values and minimum and maximum values of the acoustic parameters in the bark were tested against the radius of the last boom.