Real‐time insect tracking and monitoring with computer vision and deep learning

Abstract Insects are declining in abundance and diversity, but their population trends remain uncertain as insects are difficult to monitor. Manual methods require substantial time investment in trapping and subsequent species identification. Camera trapping can alleviate some of the manual fieldwork, but the large quantities of image data are challenging to analyse. By embedding the image analyses into the recording process using computer vision techniques, it is possible to focus efforts on the most ecologically relevant image data. Here, we present an intelligent camera system, capable of detecting, tracking, and identifying individual insects in situ . We constructed the system from commercial off‐the‐shelf components and used deep learning open source software to perform species detection and classification. We present the Insect Classification and Tracking algorithm (ICT) that performs real‐time classification and tracking at 0.33 frames per second. The system can upload summary data on the identity and movement track of insects to a server via the internet on a daily basis. We tested our system during the summer 2020 and detected 2994 insect tracks across 98 days. We achieved an average precision of 89% for correctly classified insect tracks of eight different species. This result was based on 504 manually verified tracks observed in videos during 10 days with varying insect activities. Using the track data, we could estimate the mean residence time for individual flower visiting insects within the field of view of the camera, and we were able to show a substantial variation in residence time among insect taxa. For honeybees, which were most abundant, residence time also varied through the season in relation to the plant species in bloom. Our proposed automated system showed promising results in non‐destructive and real‐time monitoring of insects and provides novel information about phenology, abundance, foraging behaviour, and movement ecology of flower visiting insects.