The authors have declared that no competing interests exist.
Conceived and designed the experiments: IP. Performed the experiments: IP KF IR. Analyzed the data: IP KF IR. Contributed reagents/materials/analysis tools: IR. Wrote the paper: IP KF.
Monitoring traps are important components of integrated pest management applied against important fruit fly pests, including
A number of major pests of commercial olives and fruits worldwide are monitored by McPhail type traps as thoroughly described in [
For insect species that are of large economic importance such as
The so called ‘decision protocol’ is used as a rule of thumb. In practice, reports from monitoring traps are not accepted blindly but serve as supportive evidence. Certified state entomologists adapt the rule to the particularities of different geographical parts of the country and integrate diverge sources of information before granting permission for large-scale spraying. Reports coming out of manual monitoring of traps are accepted with a varying degree of trust. The main difficulty of this procedure is that a large number of glass McPhail traps must be strategically placed on olive orchards, sometimes on distant and remote locations and numerous people should place, maintain and inspect the traps on a 5-day basis from the end of spring till the end of fall. The pest-managers must discern
The classification of the insects is done automatically, as they fly into the trap, the moment they enter, and collective results are transmitted to the central monitoring agency on a scheduled basis without human intervention.
Our modified traps are equipped with a Geographical Positioning System (GPS) receiver. Geographical coordinates are transmitted in addition to insect count data. Therefore, one can be sure that the insects counted come from the right place and not from an inappropriate but easier to reach location (e.g. on the roadside) as often in practice. Along with the coordinates, mean values from recordings of temperature and humidity sensors are transmitted in order to be correlated and be used as supportive evidence to assist decision making. Temperature and humidity are registered and transmitted as well, as these environmental parameters are closely correlated to insects’ life cycle and reproduction patterns [
All processing is performed locally on the trap, while the results are transmitted via a text message, aiming to keep power consumption and operational costs to a minimum. The delivery of data can be remotely set on per event, daily or weekly basis and only the power consumption can set the limit. Therefore the present trap has the potential to monitor the spatial distribution and dynamics of pest populations in real-time.
The reported literature on electronic insect traps that employ optical sensors is sparse [
The trap is modified to account for the fact that, a number of
Improved custom-made electronics are developed that are placed on the exterior of the trap as a slim add-on kit and therefore do not alter the internal space of the trap.
We carry out detailed controlled experiments with several flying insects including
In order to produce a usable platform, it is important to balance between the competing needs of accuracy and other priorities such as the cost, real-time performance and power sufficiency. Moreover, considering the platform’s long-term exposure to real-field conditions, the electronic equipment should be simple and robust. Sensor’s efficiency is also of grave importance, but this should be achieved with low-power consumption and using a low-price sensor. Algorithmic accuracy is highly ranked, but it should be achieved with low-complexity algorithms that will allow real time performance and low power consumption. Construction cost is of importance as well, but this work focuses on monitoring (that requires 1 trap per 1000 trees ~ 10 ha) and not on mass-trapping. The electronic trap, cannot penetrate the routine of agricultural work if it does not fulfill a real need that justifies its added cost, i.e. the reliable timing of the spraying process at large scales and an estimation of where and how dense the problem is prior to and after the treatment. The cost in crop loss due to an erroneous estimate of the initiation of the spraying process is very large compared to the cost of the electronic traps. Considering that the number of electronic traps to be deployed for monitoring is small, the additional cost of the trap is justified compared to the possible damage.
Provided that a sufficient number of traps is deployed, the monitoring agency can track the status of infestation from day to day for large cultivated areas, spreading up to country level, and assess the impact of the spraying in a timely manner, as there is zero time-lag between the time insects are captured and the time data are reported. This lack of delay between the reality of the infestation and the delivery of the assessment report allows one to efficiently design policies and actions. A time-lag in data reports would mean that people could be reflecting on a situation that may as well have evolved to another unknown state by the time they decide on an action plan. This level of service could only in theory be achieved by manual means, as it would require an amount of funds that would practically be difficult to secure. The device (see
It attracts insects, either with food-baits or pheromones, as classical McPhail type traps do.
As they fly in the trap, an opto-electronic sensor composed of an array of photoreceptors that acts as a receiver and an array of infrared LEDs on the opposite side of the circular entrance guard the entrance by forming a light gate.
As the insect flies in, its wings interrupt the flow of infrared light from emitter to receiver. The signal of the wingbeat received is of very high signal to noise ratio (SNR) and resolves the fundamental frequency of the wingbeat as well as several harmonics up to 2 kHz.
The analog signal of the wingbeat recording received from the opto-electronic sensors is directed to a microprocessor embedded in the trap that analyses the frequency content (i.e. the spectrum) of the acquired recording. The aim is to extract the fundamental frequency and the way the energy is distributed on the harmonics of the recording. We show that this information, extracted from typical 30–500 ms duration flights of
The insects are counted one at a time while alive and upon their entrance to the trap, and, therefore one avoids to confront the maze of insects that, due to piling and disintegration are difficult to be reliably recognized and counted visually.
The electronic circuit stores insect counts and target counts internally and transmit counts according to a preset timetable using the Global System for Mobile Communications (GSM) network. The detection results of all entering insects are accumulated on per-day basis and a Short Message Service (SMS) message, with the results is emitted from the field straight to base. The SIM card and the GSM antenna are embedded in the microprocessor’s hardware. The time-schedule of data delivery can be reset remotely by the user, by sending a typical short message (SMS) to the trap.
The insect flying in occludes with its flapping wings the path of light from emitter to receiver. The electronics of the trap analyze the light fluctuation of the receiver. Light intensity fluctuations constitute a ‘biometric signature’ directly related to insect’s wingbeat frequency, size and shape of its wings. The signature is compared to pre-embedded patterns from the target pest. Finally, counts of the target pest, temperature, humidity and GPS coordinates are transmitted through the mobile GSM network from the field to the monitoring agency.
It takes a flying
The fundamental frequency of
The time domain plots show 30 ms of data. The spectrum is derived from approximately 2.5 s from the same signals. Note how the flute and violin time-domain plots have the same period but different shapes leading to a different weighting on the importance of the harmonics.
Therefore, during classification we will not only take into account the fundamental frequency, that is the frequency that the insect beats its wings, but also the harmonics produced by the movement of the wings. The harmonics produced are related to the size and shape of the wings. The slightest difference leaves an acoustic imprint. Insects of the same species (e.g.
The electronic McPhail trap, while operating in the field, compares the recording of the flying in insect with pre-stored recording of the pest that act as prototype patterns. Based on this comparison, it decides on the identity of the incoming unknown insect according to a distance measure. These prototype patterns are recorded in the laboratory, by placing sensors (see
As insects fly freely in the cage, some of them randomly pass through the square thus interrupting the light path from emitter to receiver.
The larvae inside the olive fruit are in various instars and they feed upon the pulp until they exit, usually as third instar larvae which pass through the sieve to pupate. Then they are collected and grown in an insectary cage. As larvae turn into adult insects, we supply them with yeast hydrolysate-sugar diet and water to sustain them to life. We keep only first generation insects, as breeding generations of insects in captivity results into degeneration that might affect the flying mechanism. Denoting as day 0 the time we collect olives from trees, then at day 0–12 the 3d instar larvae come out and, from day 8–20 they turn into adults. The same day (few hours later), they can fly. Adult insects are free to fly without induced stress inside the cage and while they fly they pass incidentally through the ‘candlestick-like’ sensors placed inside the cage (see
Each time an insect incidentally flies through the rectangle, a recording of a wing flap is acquired.
We have fabricated two versions of the optoelectronic device; the receiver is either an array of photodiodes or phototransistors (see
Both sensors resolve the fundamental frequency of the wingbeat around 180 Hz. The diodes resolve better the harmonics at multiples of ~180 Hz.
PSD of photodiodes array vs microphone transducer. Both sensors resolve the fundamental frequency of the wingbeat and have good accordance until the 5th harmonic. In this particular controlled setup, the microphone can resolve higher harmonics as well.
The waveforms generated by the flight sounds of
One should not rush to see a benefit of the microphone to this task compared to the optical device:
The optical sensor records an event only for the time that the light from emitter to receiver is interrupted and therefore the wingbeat events are ad-hoc shorter in time than events recorded by a gun microphone in a small cage containing a large number of insects of the same species. Special measures have been taken in order to make possible a microphone recording in the lab and these measures cannot be applied when operating in the unconstrained field: The cage was placed in a quiet chamber in the laboratory providing low-noise conditions to study insect sounds. In normal operational conditions, the microphone would pick up sound from all directions, as the field is exposed to relative high noise levels (due to wind, cicadas, birds and machinery), thus making unpractical to be used for the task at hand.
We observed closely the behavioral mode of
Direct, fast crossing of the sensor either by moving upwards or diving in at an almost straight path. It takes the insect 30–50 ms to cross the field of view of the sensor. In
Slower flying movements like if the insect is strolling. The movement may involve coordinated turns and hovering. This mode of flight can take roughly 50–300 ms to cross the receiver but depending on the angle of the entrance can reach even more.
The electronics can pick up interference from electric lamps and must, therefore, be operated only in daylight or DC-powered light in the laboratory. They also operate in total darkness. It is possible to use electronic configurations that are immune to interferences but this increases the cost of the sensors and we need to keep the cost down in order to be acceptable by the end-users. More sophisticated electronics that send 60 kHz pulses instead of continuous light to the flapping wings of the insect that modulates the amplitude of the high frequency carrier, clean the low frequencies from interferences and demodulate back the wingbeat at acoustical frequencies are immune to low frequency interference variations caused by AC powered electric light [
Light for the sensor was produced by 4 infrared LEDs, operating at 940nm, connected in row (see
Wingbeats are low in energy level. Despite this low power level the electronics provide recordings with very high SNR (see
The embedded platform runs a constantly-looping program (see
Processing is continued to the next stage only in case of a positive event otherwise transmission stage rests in sleep mode
Once the data is normalized according to (1), it is compared to the reference spectra stored in microcontroller’s EEPROM. In its current iteration, six reference recordings is the maximum that the microcontroller’s EEPROM size of 4 kB allows, but this can trivially be expanded in future versions of the hardware platform.
Based on the comparison of the captured signal’s spectrum to the stored references, and depending on if it matches the pre-defined similarity criteria, the investigated event can be classified as a match or not. In the former case, the counter of target species matches is increased by one and auxiliary data is collected by the system’s sensors (namely temperature, humidity and GPS coordinates). The resulting dataset transmitted via SMS to one or more recipients.
The trap carries a photodiodes array sensor identical to the one used to record the reference patterns in the laboratory (as depicted in
Dimensions are in mm.
We have experimented with the applicability of GPRS that is available on the communication chip embedded in the trap and connects the trap to the Internet. It can be used to transmit results as SMS messages, including counts of detected insects and associated metadata (i.e. battery status, GPS coordinates, temperature and humidity) to a web interface. In line with the goal of producing an accessible and cost-effective solution, it was important to implement this web/backend functionality without resorting to any costly and/or hard to setup and operate software platforms. To this end, we exploited the dweet.io platform (
As we want to aggregate and visualize this data, we need to extract it from servers and forward it to another platform. For this purpose, we make use of another free service, namely Freeboard (
A simpler transmission method was also implemented, which involved having the platform send a daily SMS message with target counts and total events to a predefined mobile phone number. This way, the owner of the olive orchards being monitored can get updates, even when no internet access is available and/or when one is not computer inclined and thus not comfortable with using the web interface.
Although we plan to study the case that the traps form a wireless network that collects data from each trap using hop communication and there is only one transmission from the gate to the central station we did not choose this option in this work. The traps must be strategically distributed to cover the whole area of interest and this may include distance of several kilometers, hills and other obstacle that can prevent nodes to contact each other. Moreover, farmers would rather avoid dealing with the complexity of technology and possible problems with incompatibility of diverse technologies.
The objective of the verification module is to examine if the generated optical fluctuations modulated by the wingbeats of the incoming insect belong to
In
One can note by looking at
One can get a variety of features out of a recording but, as analyzed thoroughly in [
The derived spectrum of the incoming insect is compared against the spectrum of four pre-embedded spectrum prototypes. These prototypes are derived by performing K-means clustering on a set of
A threshold derived by the mean of the tracked distances and enlarged by 3 standard deviations is kept as the expected maximum divergence of
The data set of the target insect is composed of 230 recordings of
Various insect species tested against the target species.
Species | No. of records |
---|---|
230 | |
80 | |
205 | |
37 | |
492 |
Recall (
These quantities are also related to the (
High precision relates to a low false positive rate, and high recall relates to a low false negative rate. High scores for both show that the classifier is returning accurate results (high precision), as well as returning a majority of all positive results (high recall) [
First, we verify the presence of
To help interpret system operating characteristics, the analog output from the sensor is recorded before it is sent to the microcontroller, and recordings are also made after the signals are processed by amplifier and filter circuits.
precision | recall | F1 score | #recs | |
---|---|---|---|---|
0.95 | 0.94 | 0.95 | 230 | |
Mosquitoes | 0.84 | 0.86 | 0.85 | 80 |
Avg/total | 0.92 | 0.92 | 0.92 | 310 |
The next experiment deals with a species that is closer to the frequencies of
precision | recall | F1 score | #recs | |
---|---|---|---|---|
0.95 | 0.94 | 0.95 | 230 | |
0.94 | 0.94 | 0.94 | 205 | |
Avg/total | 0.94 | 0.94 | 0.94 | 435 |
We then move to the
The capability of the system to discern among insects is again adequate as shown in
precision | recall | F1 score | #recs | |
---|---|---|---|---|
0.90 | 0.94 | 0.92 | 230 | |
0.48 | 0.32 | 0.39 | 37 | |
Avg/total | 0.84 | 0.86 | 0.85 | 267 |
Then we move to a difficult case:
In this case the decision rule to classify insects based on the absolute difference of the spectra fails to discern the different species. This is due to the fact that the fundamental frequency, that is almost the same in both species, dominates in the calculation of the distance measure. Thus, we examine other ways of classifying the spectrum that do not rely on a single number. First, we resort to off-line, model-based classification that bases the decision to more parameters than a single threshold, and is able to capture variable interactions to a large depth. Off-line pattern recognition may classify data using more computational demanding algorithms, in order to set a rough limit of what can be achieved on the specific dataset, provided we had no memory and power constraints. Since we have fixed our approach to rely exclusively on the spectrum and its transformations, we employed well-established, state of the art, machine learning techniques that are capable of dealing with high-dimensional datasets. Support Vector Machines (SVM), Random Forests (RF), Randomized Trees Classifiers, as well as the Gradient Boosting Classifier (GBC), are known to be able to handle efficiently high dimensional feature sets [
Average scores over 10-fold cross-validation, 20% hold-out.
10-fold average | |
---|---|
Support Vector Machines | 0.738 |
Random Forests | 0.736 |
Extra Trees | 0.725 |
Gradient Boosting Classifier | 0.736 |
For each fold we retain 80% from the shuffled data and accuracy is measured on the unseen 20% of the data.
precision | recall | F1 score | #recs | |
---|---|---|---|---|
0.74 | 0.74 | 0.74 | 43 | |
0.78 | 0.78 | 0.78 | 49 | |
Avg/total | 0.76 | 0.76 | 0.76 | 92 |
precision | recall | F1 score | #recs | |
---|---|---|---|---|
0.61 | 0.38 | 0.47 | 230 | |
0.59 | 0.79 | 0.68 | 262 | |
Avg/total | 0.60 | 0.60 | 0.58 | 492 |
As detailed in the Methods section, we developed a web backend to allow trap monitoring in a user friendly and timely manner. A proof-of-concept of this can be seen in
The figure shows the online web interface that presents detection results of trapped insects in general and target species in particular, based on Freeboard.io and OpenStreetMap. (Figure is similar but not identical to the original image, due to copyright restrictions, and is therefore for representative purposes only).
In the example of
The results show that the current state of the verification module can deliver reliable counts of insects, provided their spectra do not overlap significantly. One should note, that, in the aforementioned experimental results we have chosen a very simple rule based on the absolute distance of spectra. The model-based classification experiments demonstrate that there is room for significant improvement in the classification scores. We have scrutinized the analog recordings prior to entering the microprocessor and after being digitized by it and we see a positive perspective in the following directions: a) increasing the coding depth of the A/D converter to 16 bits (currently restrained to 10 because of the processor used) as coding affects recognition greatly, b) the buffer of monitored samples used to trigger an event is currently discarded which reduces the available data from which the spectrum is derived. Since our data are short-time, this, sometimes leads to very noisy recordings as parts of the events were discarded. A number of misclassification was due to this fact and we will account for this loss of data in future versions, c) studying 2D receptors (i.e. two layers of photoreceptors instead a single array to allow for a longer time-span of flight (at least-double) that is expected to increase spectrum accuracy, d) transmit the recordings to a central agency instead of the decisions in text. Since typical events last only 30–50 ms this means that 1 second can hold about 20–30 events at 4 kHz sampling rate. The transmission using the General Packet Radio Services (GPRS) functionality will allow the more accurate, off-line, model-based techniques to be applied on the recordings while significantly reducing the cost of the trap. The latter choice is expected to shift recognition results by a large margin.
The effort of this work was to make a functional, stand-alone, prototype electronic trap that completes successfully all processing stages starting from attracting
The electronic trap, at this stage, can adequately discern
There are some issues that are not dealt in this work but will be in the near future:
Power sufficiency is something missing from the current analysis of the electronic trap. We are working on low-power electronics, in order to make the trap power sufficient for months and also reduce its actual cost (see Appendix).
A pending task is the experimentation with gender recognition. In the case of mosquitoes, that are dimorphic, classifying sex from the wingbeat is a relatively easy task as females are larger than males and this leaves a trace in the wingflap as demonstrated in [
We are currently experimenting with the idea of wirelessly sending recordings of the pest to be installed automatically in the trap and serve as prototypes i.e. take reference signals from an internet host of insects’ wingbeats measured with optoacoustic systems and download the prototypes remotely straight to the traps. This would augment the utility of the trap as it would be able to change focus on different insects without human intervention.
The real field holds the truth regarding the functionality of the electronic traps and a detailed open-field test is pending. However, in-lab analysis with almost real-field conditions was the necessary step prior to the unconstrained field as it revealed possible problems and permitted the fine-tuning of software and equipment.
We believe that biometrics can be applied in various ways on animals, including insects, in order to realize what happens and where it happens and what is the density of the species counted, allowing the design of reliable policies based on the outcome of these measurements. We suggest that electronic traps that record and analyze insects’ wingflap can open new ways for several other applications such as smart beehives, devices that emit species labels and counts to central agencies to feed with data infestation models updated real-time and make predictions of future outbreaks (see e.g.
We hereinafter include a cost analysis of the electronic trap.
There is no reason nowadays to base monitoring of insects on expensive and fragile
Regarding the electronic version of the McPhail trap presented in the paper, we believe that the quality and value of olive oil and the high-risk due to the pest
Item | Model | Unit Price € | Quantity |
---|---|---|---|
Emitter (infrared led) | TCRT5000 | 0.45 | 4 |
Receiver (diodes) | TEMD5080X01 | 0.68 | 10 |
Microcontroller | ATMega2560 | 12.15 | 1 |
Temperature-humidity | Si7021 | 4.03 | 1 |
GSM/GPS | SIM908 | 15 | 1 |
Electronic Components | Passive, RTC, EEPROM, PCB, Connectors | 15 | 1 |
Plastics | Plastic McPhail trap, add-on | 5 | 1 |
Battery | Lithium 6000mAh | 19 | 1 |
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
Light is provided by 4 infrared LEDs (940nm) connected in row, led by a 2.7 mA current.
(TIF)
The receiver is a linear array of 10 photodiodes connected in parallel. The received light is amplified by the IC1 and is driven to the band-pass filter.
(TIF)
The filtering process is carried out by the high-pass filter IC7 A, B & C and the low-pass filter IC6 A, B & C. Subsequently, the signal is amplified by IC6D and driven to the processor.
(TIF)
The signal coming out of the filters is amplified by the current circuit by the factors 1, 10 and 100. The signals x1, x10 and x100 are driven to the 3 analog inputs of the microcontroller.
(TIF)
The power supply circuit has as input the Lithium battery 3.7 Vdc and supplies voltages 3.3 Vdc & 5 Vdc for the digital and analogue circuits.
(TIF)
The GSM module sends text data through GPRS. It also embeds a GSM. When not in use MOSFET Q2 cuts its power supply so it does not consume energy. It is controlled via the Microcontroller (ATMEGA2560).
(TIF)
In the SD card the microcontroller stores the recordings (we store the FFT but the actual recording of the ADC can be also stored) for further analysis. When not in use MOSFET M7 cuts its power supply so it does not consume energy.
(TIF)
Processor IC1, receives the analog signal from the inputs A0, A1 and A2 and selects the proper gain. IC1 is processing the resulting digital signal. It also controls the GSM module, the humidity and temperature sensors.
(TIF)
Wireless transmission of the recording of the optoelectronic sensor prior to its entrance to the microcontroller.
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(TIF)
(ZIP)
(ZIP)
Sound is the sonification of the optoacoustic sensor played through loudspeakers. Photodiodes array, Laser emitter as in [
(M4V)
Photodiodes array, infrared emitter.
(WMV)
(WMV)
(M4V)
We acknowledge Anastasia Kambouraki, from the Department of Biology, University of Crete, Heraklion, Greece for patiently collecting insect specimens and taking care of insect cultures for the recording experiments and Prof. John Vontas head of the Laboratory of Molecular Entomology in the same department for granting us access to all insectaries and associated equipment. Plastic McPhail type traps and attractants were provided by PHYTOPHYL-N.G. Stavrakis—Insect Attractants & Insect Traps (