InsectDCT: A generalized pipeline for detection, taxonomic classification, and tracking of insects in camera-trap recordings

Abstract

Automated monitoring of insect pollinators in natural environments with insect camera traps and trained deep learning algorithms can provide large-scale, temporally resolved observations for insect ecological studies. However, efficient and accurate recognition of insects in recorded images or videos is challenging, particularly when small insects appear against complex backgrounds with diverse vegetation communities. In this work, we present InsectDCT, an AI-based pipeline for automated detection, hierarchical classification, and tracking of insects during daylight in footage of natural vegetation, tested in different environments. The InsectDCT pipeline consists of three levels: insect Detection and localization, hierarchical taxonomic Classification, and spatio-temporal Tracking. In the first stage, insects are detected in time-lapse images or video recordings using the You Only Look Once (YOLO11) object detection architecture. Detection performance is improved using motion-enhanced images, which improve robustness in cluttered environments. The detector is trained on an extensive dataset of more than 60,000 images, and the classifier is trained on 90,358 insect images collected using camera traps deployed across a wide range of plant families and floral habitats. It achieves the best F1-score of 0.88 with motion-enhanced images. In the second stage, detected insects are classified using a hierarchical taxonomy-aware classification framework that covers 80 taxonomic groups. Classification is performed at multiple taxonomic ranks, including order, family, and genus/species, achieving F1-scores of 0.90, 0.84, and 0.76 at each respective rank. It allows for coarse and fine-grained ecological analyzes while accounting for varying levels of visual ambiguity. In the third stage, a multi-object tracking module is applied to high temporal-resolution image sequences and video data to associate detections across consecutive frames and generate track-level observations. It achieves a MOTA of 76.1% and IDF1 of 74.2%. InsectDCT code and all datasets are made publicly available.