Recording hardware setup

Previous work has shown that the use of multiple microphones, assembled to form an array, can provide significant advantages for the task of acoustic insect detection in noisy environments [11]. Therefore, this study investigates microphone array assemblies rather than individual microphones. The two arrays chosen for use in this study are described below, compared in Table 1 and shown in Fig 1.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Table comparing the properties of the two microphone arrays and their microphones as given in their data sheets [13,14].

https://doi.org/10.1371/journal.pone.0350946.t001

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Recording hardware used.

The low-noise measurement microphone array assembly (right) and the ReSpeaker Core v2.0 MEMS microphone array in its custom-made housing (left), pictured inside the anechoic box.

https://doi.org/10.1371/journal.pone.0350946.g001

2.1.1. Measurement Microphone Array (MM).

The goal of the MM device in this study was to highlight the cutting edge of technical feasibility in order to obtain good data on the limits of acoustic insect detection. To exclude the possibility of a microphone of higher quality improving on the results, the aim was to select the best microphones available. When comparing microphones, a high sensitivity was prioritised over a low self-noise level [12]. Fig 2 shows a comparison of the frequency responses of the two microphone types used in the two array systems.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Microphone frequency response comparison.

Plot comparing typical free-field frequency responses of a Brüel & Kjaer type 4955 low-noise measurement microphone and a Knowles SPU0414HR5H MEMS microphone. The Brüel & Kjaer type 4955 is used in the measurement microphone array and the Knowles SPU0414HR5H is used in the ReSpeaker Core v2.0. As is relevant to this study, the frequency response up to a frequency of 8kHz is plotted. The data for this plot was generated by digitalising frequency response plots given in the respective microphone data sheets [13,14].

https://doi.org/10.1371/journal.pone.0350946.g002

The MM is an array consisting of four low-noise, Brüel & Kjaer type 4955 measurement microphones. They were assembled to form an array using a self-designed 3D-printed fixture made from PA-12 (CAD file supplied as supplementary material S1 CAD File). The fixture places the microphones in a star-like shape, seen in Fig 1. The detailed properties of this array assembly are listed in Table 1 in the MM column. The reader interested in further details regarding the Brüel & Kjaer type 4955 measurement microphones shall be referred to the data sheet [13].

The MM was powered by a Brüel & Kjaer Nexus 2690 Conditioning Amplifier. Using the Nexus, the four signals were high-pass filtered at 20 Hz, low-pass filtered at 10 kHz, and amplified by 20 dB. The analogue signal was then fed to an A/D converter made by Roga Instruments called DAQ4. The gain function on the DAQ4 was disabled, AC-coupled mode was enabled and a digital high-pass filter with 0.3 Hz was activated. All recordings were digitalised using a sample rate of 48 kHz using the software DasyLab 2022 and saved as single-precision (32-bit, floating-point) TDMS-files, each 14:13 min long.

Acoustic lab environment: Anechoic box

An anechoic box was built to shield the recording environment from environmental noises. The design consisted of two wooden boxes lined with different-density acoustic foams. By suspending the smaller box from springs inside the bigger box, a double wall structure was obtained that provided good noise shielding for frequencies above 100 Hz. For more details on the development, design and testing of this anechoic box, please refer to [12].

A insect-rearing cage made from all-mesh material was placed inside the box. The cage was set up to contain two small tomato plants. Two types of LED stripes placed on top of the cage provided light. The light setup was slightly changed for recordings of Trialeurodes vaporariorum. Because the LED lights strongly attracted these insects, the lights needed to be positioned on the side wall of the rearing cage at the height of the microphones to guide these insects closer to the microphones. Due to their very low-level flight sounds, the original recording setup with the lights placed on top of the cage was unsuccessful, as insects flying around the lights placed on top of the cage were too far away from the microphones to be recorded. The recordings were conducted overnight to minimise the influence of noise from people working in the building. The setup of the cage and lights is described in further detail in [11] and [12].

The MM was mounted to a fixture and placed outside the cage, directly facing the mesh, very close to, but not touching the mesh. The RS was placed inside the cage by attaching the housing to a wooden stick placed inside the potting soil. Data and power supply cables attached to the RS were routed out of the cage through a mesh sleeve attached to one of the cage walls. Both arrays were placed approximately 30 cm above the box and insect cage floor.

Insect recordings

The purpose of the original project was to build a system applicable to the context of protected horticulture. Therefore, all insects recorded are likely to be found in a European greenhouse. Further, the goal was to portray a range of different sound levels. Table 2 gives an overview of the insects selected for this study, their body length and average sound pressure level (SPL) from the recordings. The range of insects selected extends from small flies to bigger bugs and covers some beneficial as well as pest insects. Expanding on previous work [11], this study includes insects that were difficult to record and yielded a very low sound sample number, regardless of extensive recording efforts. Examples of this are small pest insects such as Myzus persicae, Trialeurodes vaporariorum and Tuta absoluta. The MM data used in this study is a subset of the InsectSound1000 dataset that was published in [12]. The RS data used in this study is unique to this study and a subset of the InsectSound1000-MEMS dataset available at [16]. Fig S1 Fig allows for a visual comparison between the different insect sounds and the effect the different recording hardware has on the actual data recorded. The Fig contrasts selected spectrograms of the same insect species recorded with the two different microphone arrays. As this study did only involve non-vertebrate insects, no permits were required under German law for the recordings made in this study.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Overview of insect species investigated.

https://doi.org/10.1371/journal.pone.0350946.t002

As derived in [12], the experimental setup, which physically ensures all sounds recorded in one session must come from one species, represents the only way to generate labelled insect sound recordings. By placing only insects of one species inside the anechoic box at a time, all sounds recorded in one session should be from this one insect species.

For every recording, only one cage containing three to 50 insects of one species was placed inside the anechoic box. Bigger, louder insects were recorded using only a few individuals, while smaller insects were recorded in higher numbers to ensure that three to four nights of recording time would yield roughly the same amount of insect sound events in the recordings.

Unfortunately, not all recordings used in this study were recorded simultaneously using both microphone arrays. Varying technical problems with both microphone arrays meant that on some nights valid data from only the RS could be saved, while on other nights, only the MM recordings produced valid data. Out of a total of 48 recording nights processed in this study, three produced valid data for only one of microphone arrays. In an effort to maximise the available data to train each of the two models, it was decided against discarding the data recorded on these semi-successful recording nights. In total, 742 h of MM recordings and 709 h of RS recordings were collected and used for further data processing.

Environmental sound recordings

Background sounds were recorded in a greenhouse and used to simulate noisy conditions by mixing the insect sound recordings from the acoustic lab with these background sounds from the greenhouse. The greenhouse was located at the facilities of the JKI in Braunschweig and used for research purposes, therefore no permits were required for the recordings made in this study. Six different locations inside the greenhouse were simultaneously recorded using both microphone arrays, in the same way as for the laboratory recordings. At every recording location, two different orientations of the microphone arrays were recorded for 5 min. The process for generating this environmental sound dataset is described in detail in [11]. The dataset containing the simultaneous background noise recordings of both microphone arrays is available at [17].

Model architecture

This study processed multichannel data from both the MM and the RS. It is fundamental to this comparison that the deep learning models utilised here can exploit the benefits of this multichannel data. Deep learning classification models directly processing raw audio data have been proposed for such applications [18]. Previous work has shown the benefits of such multichannel raw audio data processing models for insect sound recognition, especially when dealing with strong background noises [11]. This study uses the same model as that proposed in [11] to compare the potential of the two microphone array setups. To process the RS data, only the input size was modified from four to six channels. This raw audio data processing acoustic detection model will be referred to as the NBF WaveNet.

As the name suggests, this model is made up of two components. The NBF layer, for neural beamforming is designed to learn spatial filtering by mimicking the calculations of a filter and sum beamforming algorithm. In classic signal processing, such beamforming algorithms are used to extract directional information from multichannel sound signals [19] as a part of signal preprocessing. In this study, the data was not explicitly preprocessed using such spatial filtering. Instead, the NBF layer is part of the deep learning model and trained to perform similar calculations implicitly.

This second part of the model is a WaveNet classifier used to process and classify the raw, now single-channel, prefiltered signals generated by the NBF layer. Originally introduced as a generative model for music and speech synthesis [20], the WaveNet architecture has since been shown to be a powerful sound classification model [21–23]. A more detailed description of the implementation, based on [24] and [23] is given in [11]. The Jupyter Notebook containing the full code used in this study is available at [25].

SPL-Weighted loss

Previous work has shown that the NBF WaveNet model used in this study is a significant improvement upon a naive single-channel spectrogram-based model as it effectively utilises spatial information encoded in the multichannel recordings. However, the model also turned out to be difficult and unstable to train, as only two out of five training attempts converged in all of the training steps in the environmental sound simulation in a previous study [11].

It is well understood that the first layers of a deep learning model are the most difficult to train. This is because rounding errors will degrade the training signal (gradient) when backpropagating through the model from the last to the first layer during training, famously called the vanishing gradient problem [26]. In the case of the NBF WaveNet model, this very first layer, however, is the NBF layer, which is of great importance to the overall model performance, as it is the only part of the network that can learn spatial filtering capabilities. Combining the difficulty of training this first layer with its importance makes for difficult and unstable model training.

In addition, the dataset of insect sounds in the environmental sound simulation must be ranked as very challenging, or even partially unsolvable. When simulating full-scale environmental sounds, the SNR is much smaller than in related tasks [27] and can sometimes drop below −30 dB. Previous work concluded that this dataset of insect sounds contained so many very quiet, and therefore very difficult to classify, samples, it seemed to hinder model convergence during training. This led to a sub-human performance even on louder, easy-to-classify samples in some models trained and tested in simulated greenhouse conditions [11].

In an attempt to stabilise training in the environmental sound simulation, this study proposes the use of a difficulty-weighted loss function for training in simulated noisy conditions. Its idea is to mimic the approach of a human learner tasked with solving a partially unsolvable problem. By simply multiplying the commonly used categorical cross-entropy loss value with a measure of classification difficulty, getting the easy samples right is emphasised during training.

Quantifying classification difficulty for individual samples is a very much non-trivial task in most fields (e.g., image recognition). For the task of audio classification in the presence of noise, however, the SNR of a given sample provides a simple and reasonable estimate of classification difficulty. This is as long as the frequency content of both the noise and the signal dataset do not vary too much between samples. In theory, if noise and signal sounds are of purely different frequencies, they can be distinguished no matter the SNR. In practice, the sound samples used to represent the environmental sounds in this study contain signals mostly described by a distribution (pink noise). The insect sound samples, while specific to each of the classes in detail, can collectively be assumed to have a relatively similar frequency distribution.

Furthermore, given that the environmental noise simulated is roughly the same level for every sample, calculating just the insect sample SPL alone is sufficient to provide a good approximation of classification difficulty in this study. By writing the SPL of every sample to its filename and simply reading these values while building the dataset, the proposed loss function provides close to no computational overhead during training. The SPL was calculated in the logarithmic unit dB to provide values in a reasonable value range. This leads to the following formulation of the proposed loss function in equation 1:

(1)

where CCE denotes the commonly used categorical cross-entropy loss function. Preliminary experiments showed that while final results at the end of all training steps in the environmental sound simulation did not vary significantly, training in the early steps was accelerated, and convergence was stabilised when using the SPL-weighted loss function as soon as the simulated noise amplitude is greater than zero. Contrary to a priori excluding the quieter, more difficult samples from training, training on the full dataset with the proposed loss function allows the training process to determine what is possible or impossible to hear.

Model training

Training a deep learning model involves making several choices which determine the success of the undertaking. The parameters of model training that lead to the results presented in this study, as well as their choice, are described briefly below.

Model evaluation

Model testing was also done in the environmental sound simulation in the same way as for training and validation. Test data was generated by mixing sounds from separated insect sound test sets and environmental sound test sets. Because the environmental sound samples are chosen at random from the environmental sound test set, the evaluation metrics fluctuate slightly. Depending on whether a quiet insect sample is mixed up with a loud or a quiet environmental sound sample, it will be either easier or more difficult to classify. Therefore, all evaluations involving data containing simulated background sounds were repeated 10 times to account for these fluctuations. The results presented here are the mean values of these ten repetitions.

Because the models are initialised with random values at the beginning of training, the results vary slightly for each training repetition. Each model was trained five times on clean data. The best three models, or however many converged during this first step of training, were then subsequently fine-tuned and refine-tuned at increasing noise amplitudes following the process described in Model Training.

Data preparation

The data pipeline feeding the data to the deep learning models involved many steps, from recording the raw audio data to model training and testing. Most of these steps, however, are identical to a previous study based on a subset of the MM recordings [11]. Therefore, only a summary of the data pipeline is provided below.

Comparison of the two datasets

In total, 742 h of recordings were evaluated for the MM and 709 h for the RS. In general, the microphones of the MM generated recordings that allowed the extraction of more sound samples per hour. In total, 98 648 insect sound samples could be extracted from the MM recordings, whereas the RS recordings only yielded 9221 samples. On average, over all the insect classes, almost 11 times more sound samples could be extracted from the MM recordings than from the RS recordings. This must be explained, first, by the much greater microphone sensitivity of the microphones in the MM (see 1). Greater microphone sensitivity means that sounds either quieter or further away from the microphones are recorded and subsequently can be extracted. Second, the MM has an exceptionally low self-noise level. Lower self-noise in the measurement chain means that quieter or farther away sounds, if recorded, can be distinguished from the background noise of the recordings. As a big portion of the insect sounds is of very low intensity, this second threshold leads to a much worse performance of the relatively noisy RS compared to the MM.

Fig 3 shows three scatter plots detailing the numbers of the two datasets that form the basis of the insect detection system comparison. The first plot compares the number of recording hours evaluated for both microphone arrays. The second plot shows the number of sound samples that could be extracted from these recording hours for every insect class in the two datasets. The third plot compares the average number of samples that could be extracted per recording hour for the two microphone array setups to illustrate the varying difficulties in generating the datasets.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Scatter plots comparing the properties of the RS and MM dataset.

The plots compare the recording time, the number of samples and the average number of samples extracted per recording hour.

https://doi.org/10.1371/journal.pone.0350946.g003

Unfortunately, not all recordings were done fully simultaneously due to various technical problems, as described in section Insect Recordings. This is why the number of recording hours used differs slightly for the two microphone array setups. This can be observed in the recording time plot of Fig 3. The data points of almost all insect species lie on the diagonal between the x- and y-axis, but the dataset of the RS relies on a few more recording hours for the Rhaphigaster nebulosa and on a few less for the M. persicae and T. absoluta compared to the MM.

By dividing the data from the second plot of Fig 3 (number of samples in the dataset) by the data from the first plot of Fig 3 (recording time), one can calculate the average number of samples extracted per recording hour for each insect species. This is depicted in the third plot of Fig 3. This value represents a sort of insect recording yield rate and gives an intuitive understanding of how difficult it is to generate a reasonable size dataset for each of the different insect species with either one of the two different microphone arrays used.

When evaluating this third plot of Fig 3, Bombus terrestris clearly appears to be the easiest to record with both microphone setups. Interestingly, the order of many other insects differs in this plot between the two recording setups compared. For example, Episyrphus balteatus has the second highest yield rate for RS recordings but is outperformed by many other insect species, including Bradysia difformis and Coccinella septempunctata, in terms of yield rate in the MM recordings. This must be explained by a different composition of the two datasets. While the MM dataset contains a mix of quiet and loud sounds, the inferior properties of the RS lead to a dataset containing only relatively loud sounds. As the ratio of low-level to high-level sounds differs for every insect species, so does the order of recording yield rates for the two datasets.

Besides the obvious influence of the different insects’ sound levels, a difference in audible insect activity has a big influence on the recording yield rate that is depicted in the third plot of Fig 3. Extreme examples of this are T. absoluta, which has a relatively loud flight sound but hardly ever moves when kept inside a small cage with a healthy tomato plant. On the contrary, the flight sound of Aphidoletes aphidimyza is so quiet, it is inaudible to human ears. These small insects, however, were very active during the recordings made in this study.

The composition of the different datasets regarding the different sample SPL values is further detailed in Fig 4 and 5. These compare the SPL distribution of the sound samples of every insect species in the two datasets. At first glance, it becomes obvious that both datasets contain much more low SPL samples than high SPL samples. This is especially clear when observing the log-scale y-axes of these plots.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Histograms comparing the number of sound samples of a given sound pressure level (SPL) in the two datasets for the different insect species.

SPL values for the RS dataset are estimated based on an approximation calculation, likely overestimating real values.

https://doi.org/10.1371/journal.pone.0350946.g004

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Histograms comparing the number of sound samples of a given sound pressure level (SPL) in the two datasets for the different insect species and the environmental sounds.

Continuation of Fig 4. SPL values for the RS dataset are estimated based on an approximation calculation, likely overestimating real values.

https://doi.org/10.1371/journal.pone.0350946.g005

An analysis of the spectrograms representing samples of different SPL in the dataset showed that the higher-level sounds seem to clearly show flight sounds of the different insects, whereas lower-level sounds instead appear to be non-flight sounds originating from other types of insect motion (e.g., walking or folding and unfolding wings).

The comparison between the two microphone setups in Fig 4 and 5 clearly shows the ability of the MM setup to record and extract sounds of much lower SPL compared to the RS. While the quietest sound samples to be found for each of the displayed insects in Fig 4 and 5 in their MM datasets reach sound levels as low as 5 dB, none of the sounds in any of the RS datasets is below a SPL of 38 dB. This must be explained by the lower sensitivity and the greater self-noise of the RS system (see Table 1). These two properties combined lead to RS recordings that do not allow sounds below 38 dB to be distinguished from the background sounds and self-noise of the recordings.

However, while the general principle holds, it must be mentioned that the RS SPL values displayed in these plots in Fig 4 and 5 are calculated based on a rough estimate of a correlation factor (see section SPL Calculation) and must be interpreted with caution. Looking at the plot for B. terrestris and E. balteatus, it appears that the correlation factor used to estimate the gain in the RS system is likely leading to an overestimation of the real SPL values in the RS dataset. It appears unlikely that the RS recordings yielded more loud sound samples than the MM recordings, especially because the number of recording hours is identical (compare Fig 3), meaning these two insects were recorded fully parallel with both recording devices. From this reasoning, it seems the SPL values of the RS dataset are overestimated by up to 10 dB. Therefore, the real lower bound for insect sounds that can be recorded in such a way that allows for their extraction from the RS recordings could lie as low as 28 dB instead of the 38 dB as suggested by the plots in Fig 4 and 5.

Finally, the last plot in Fig 5 shows a comparison of the SPL distribution of the environmental sound samples recorded in the greenhouse with both microphone arrays. As was the objective of estimating the correlation factor (see section SPL Calculation), the two SPL distributions appear to be well aligned. Comparing their value ranges to the insect sound sample SPL distributions allows for an estimation of the SNR that the mixed recordings will yield during model training and testing in the environmental sound simulation.

Fig 6 shows histograms comparing the SNR distribution in test datasets generated from mixing insect and environmental noise sounds scaled to the three different noise levels used during training and testing of the Deep Learning models presented in the following section. The figure also shows typical SNR ranges found in two related research areas, bird sound recognition [33] and human far-field speech enhancement [34], for comparison. The comparison clearly shows the extreme signal conditions that insect sound classification tasks are facing.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Histograms comparing the SNR distribution in tests generated from mixing insect and environmental noise sounds scaled to different noise levels for both microphone arrays.

As described in section Model Evaluation, the test datasets are generated by mixing the insect sounds from the test set with randomly selected environmental noise sounds from the noise test set. The histogram was generated by running this randomised mixing process 10 times, as is done during model evaluation. The SNR was calculated by subtracting the SPL of the insect sound sample from the SPL of the environmental noise sound sample used for mixing. The SPL of the environmental noise sound samples was calculated in the same way as described in section SPL Calculation for the insect sounds. An operating range for bird song classification, that is considered challenging [33] and a SNR range for current speech enhancement research [34] are marked for comparison.

https://doi.org/10.1371/journal.pone.0350946.g006

Results of training two deep learning models on the different datasets

Fig 7 and Fig 8 show confusion matrices comparing the best results of the training attempts made on models using either the MM or RS data. Further, precision, recall and F1 scores of the models are given in S2 Fig. All models were trained to distinguish all nine of the species contained in this dataset. The results presented are the best runs out of five training attempts.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. Confusion matrices comparing the best attempts of training an MM and RS model in the environmental sound simulation, showing relative values.

Classes denoted as: a Bombus terrestris, b Episyrphus balteatus, c Rhaphigaster nebulosa, d Coccinella septempunctata, e Aphidoletes aphidimyza, f Bradysia difformis, g Tuta absoluta, h Myzus persicae, i Trialeurodes vaporariorum, in order of subjective classification difficulty.

https://doi.org/10.1371/journal.pone.0350946.g007

Download:

• PNG
  larger image
• TIFF
  original image

Fig 8. Confusion matrices comparing the best attempts of training an MM and RS model in the environmental sound simulation, showing absolute values.

Classes denoted as: a Bombus terrestris, b Episyrphus balteatus, c Rhaphigaster nebulosa, d Coccinella septempunctata, e Aphidoletes aphidimyza, f Bradysia difformis, g Tuta absoluta, h Myzus persicae, i Trialeurodes vaporariorum, in order of subjective classification difficulty.

https://doi.org/10.1371/journal.pone.0350946.g008

As to be expected, the results of training on the pure laboratory data, without the addition of any noise, show a much better result for the model trained on the MM data. The first confusion matrix displayed on the right (orange) in Fig 7 shows a clear diagonal, indicating that the model is correctly recognizing the insect sounds in the majority of the cases tested. The model trained on the RS data (Fig 7, left, blue,) on the other hand, only learned to recognise three out of the eight classes securely, albeit with higher confidence than the MM model.

This can be explained by the different volumes and composition of the two datasets, already discussed in section Comparison of the Two Datasets. The sparse data on some of the quieter insects and the greater imbalance between the classes in the RS dataset mean RS trained models struggle to pick up on the patterns of these underrepresented classes. Similarly, the different composition of the datasets means that the majority of the sounds contained in the RS dataset are louder and therefore easier to recognize when present in an adequate quantity to be learned by the model. The large proportion of very quiet sounds in the MM dataset means models trained on this data face a more difficult problem.

This becomes apparent when looking at the absolute numbers of test samples correctly classified displayed in Fig 8 instead of the relative values displayed in Fig 7. For example, the MM model was only able to correctly recognise 53% of all B. terrestris sounds without the presence of background noise, while the RS model was able to recognise 92% of them in the same conditions. However, in absolute numbers the MM model still correctly classified almost 500 sounds more than the RS model.

Fig 9 compares the SPL of samples that were classified correctly or incorrectly by the MM model for the class of B. terrestris in conditions without background noise. Combing these results with the comparison of the SPL distribution in the two datasets introduced in Fig 4 suggests that the different composition and volume of the two datasets mean that the quiet B. terrestris sounds the MM model is likely to misclassify are simply not present in the RS data.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 9. Histogram displaying the sound pressure level (SPL) of Bombus terrestris test samples that were correctly or incorrectly classified.

Classification was done by the MM model trained and tested on pure laboratory recordings.

https://doi.org/10.1371/journal.pone.0350946.g009

When comparing the results of both models on the pure laboratory sounds with the results after retraining these models on data mixed with background sound recordings scaled down to 1% of their original amplitude, in Fig 8 one can observe a very different effect of the background sounds on the two models. While the RS model is barely affected by the presence of the low amplitude background sounds in the data, the MM model loses 26% of its average accuracy. Furthermore, the MM model starts losing the ability to recognise the sound of T. absoluta and M. persicae (“g” and “h” in Fig 8).

This different reaction of the two models to the introduction of even only very mild background noise must also be explained by the different composition of the two datasets. As the MM dataset contains mainly low-level sounds, these are easily masked when simulating the presence of background sounds, causing the model to struggle with their classification. The loud sounds contained in the RS dataset, on the other hand, do not get masked by the downscaled background sounds.

When continuing the training in the environmental noise simulation by training both models on sounds mixed with background sounds downscaled to 10% of their original amplitude, one even sees a slight improvement in the average accuracy of the RS model, as displayed in Fig 8. However, this is most likely just a numerical effect of the repeated training restarts, and not related to the changes in the data occurring in this step of training. The MM, on the other hand, loses the ability to recognise the sounds of T. vaporariorum (“i” in Fig 8).

Mixing the insect sounds with full-scale background sound recordings represents the best approximation of realistic sound conditions in the environmental sound simulation. When continuing the training of both models on data as such, one sees that both models are clearly affected by the presence of noise, as displayed at the bottom of Fig 7 and 8. The MM model loses the ability to recognise all but one class and starts guessing, as indicated by the column pattern in the confusion matrix. The RS model is also affected, as the full-scale background sounds are also louder than the majority of the sounds in the RS dataset. However, because of the different structure of the dataset, the model is still able to recognise most of the sounds in the three classes it was performing best on during the previous steps of training. Because the loud samples of these three classes make up a very big portion of the total dataset, their recognition remains stable. Therefore, this comparison surprisingly showed a model trained and tested on the lower quality and, for most classes, smaller RS dataset outperforming a model trained and tested on the MM dataset under simulated greenhouse background noise conditions. Interestingly, looking at the raw number of correctly classified B. terrestris samples as displayed in Fig 8, the MM model still correctly classifies almost as many samples as the RS model because of the large difference in test dataset size.