Hardware and Software updates to Autonomous Moth Trap

This is the third post in a series:

• Autonomous Moth Trap Project
• Autonomous Moth Trap Hardware Revisions
• Hardware and Software updates to Autonomous Moth Trap
• Alternatives for Autonomous Moth Trap
• Software components for Autonomous Moth Trap
• Time Lapse module for Autonomous Moth Trap
• Autonomous Moth Trap Image Pipeline

This post provides notes on modifications and enhancements to my implementation of the automated moth trap designed and implemented by Kim Bjerge and colleagues (Bjerge, K.; Nielsen, J.B.; Sepstrup, M.V.; Helsing-Nielsen, F.; Høye, T.T. An Automated Light Trap to Monitor Moths (Lepidoptera) Using Computer Vision-Based Tracking and Deep Learning. Sensors 2021, 21, 343).

High-power UV LED tube

The original version used a 15W fluorescent UV tube as the attractant. Unfortunately, these tubes lose brightness with use and therefore add an unnecessary amount of variation (given my desire to standardise as much as possible).

I have therefore built a replacement with nine high-power LEDs (six UV and one each of white, green and blue) in an acrylic tube (and with an aluminium extruded channel for heat dissipation). This seems to work well and is readily powered from the same 240V AC to 12V DC power supply as the Raspberry Pi, ring light and light table.

Rebuild of housing

I have also completely rebuilt the unit in a new housing that includes better controls (soft shutdown of the Raspberry Pi via a push button, external control of the light table brightness, override switch to turn on all lights independently of the Raspberry Pi, and status LED).

Most significantly, it now includes an external temperature and humidity sensor so that local environmental readings can be added to all images.

Image processing software

The goal is to train a machine learning model for automated species recognition. Several stages are planned:

1. Run one or more traps nightly to collect images annotated with date/time, temperature and humidity
2. Segment and organise cropped images for each significant blob in the source images, annotating them with position, size, colour classification and track (series of images for the presumed same blob)
3. Curate the cropped images to validate tracks (including joining or breaking as appropriate) and associate species (or higher taxon) identifications
4. Build reference sets for identified species (annotated with all the above elements)
5. Train a model to identify these species and count associated tracks per night
6. Run the trap continuously to generate time-series data

The Danish team developed Python software (MCC-trap) for the second of these stages., but their focus was on identification of a small set of training species. I chose to refactor all their code to focus on my needs, in particular to export and annotate all blobs and organise these in tracks for my subsequent stages.

MCC-trap derives tracks using two primary measures of similarity/distance between blobs in consecutive images: size of the blob and distance apart. In addition, the code allows for a blob/track to reappear within five frames.

I chose to use additional dimensions to assess similarity/distance. My code now assesses five factors each as costs on a 0.0 – 1.0 scale:

• Distance: Blob distance is calculated as a cost as in MCC-trap, i.e. as a fraction of the diagonal size of the image. However, if the centroids of the blobs are within the central 20% of each dimension of the other image, and if the larger blob is no more than 20% larger than the smaller, they are treated as identical (overriding all other cost calculations).
• Size: Blob size is compared using the ratio of the number of pixels in the larger blob to the number of pixels in the smaller. If the ratio is four or higher, the cost is given as 1.0. Otherwise the cost is measured as (ratio – 1) / 3.
• Bearing: If a track already includes at least two blobs, direction is assessed using the diffrence between the bearing of the line joining the last two blobs and the bearing of the line between the last blob and a candidate blob. The cost is 0.0 if the bearing is unchanged and 1.0 for +180° or -180°.
• Colour: An RGB histogram is generated for the pixels inside the blob and a set of letters is associated with the blob if the proportion of pixels near each of the eight vertices of the histogram exceeds a defined threshold. At present, the histogram is a 2x2x2 cube and the threshold for each of the eight sectors is 2%. This is still under investigation.
• Age: The cost for linking to a blob in the previous frame is 0.0, increasing to 1.0 for the fifth last frame.

These five costs are each assigned a weight, currently 4 each for Distance and Size, 2 each for Bearing and Colour and 1 for Age. Hence, the actual cost estimates are distances in a 4x4x2x2x1 five-dimensional space. As with MCC-trap, these costs are fed into the Hungarian algorithm to solve the problem of the lowest-cost pairing of new blobs to existing tracks.

Results

As shown in this gallery of blobs sorted by track id, the processing outlined above seems to be giving generally good results with few false positive matches between new blobs and existing tracks but rather more cases in which a track is broken into segments.

Two videos have been uploaded to Flickr to illustrate progress so far.

ArabaAMT-20210925 presents 5027 images from an eight-hour period during the night of 24-25 September 2021 as a 10-minute 5-fps video with annotations showing the track ids, blob ids and cost measures. A wide variety of moths (including large numbers of Tortricidae) appear during the video, many of them identifiable.

ArabaAMT-20210613 has been built from 947 images taken during the night of 12-13 June 2021, when many fewer insects were active. This video has been used as a test of annotating tracks with identifications. Insects have been labeled as they appear and include:

• Lepidoptera – Lepidoscia sp. (Psychidae), Lepidocroca sanginolenta (Oecophoridae), at least one other Gelechioidea, Meritastis sp. and other Tortricinae, Chloroclystis filata, Capusa senilis and a Nacophorini (probably Fisera sp.) in flight (Geometridae), and Ectopatria horologa (Noctuidae).
• Neuroptera – Micromus tasmaniae (Hemerobiidae), Chrysopidae.
• Hymenoptera – Ichneumoninae.
• Diptera – Muscoidea plus other flies.