Zero-shot Learning based Cyclists Detection through Surveillance Systems

Abstract: Active transportation, which includes pedestrians and cyclists, plays a crucial role in the modern transportation system. The 2023 NHTSA report reveals that active transportation users are involved in a significant number of traffic fatalities, with pedestrian and cyclist fatalities accounting for 17% and 2% of all deaths, respectively. These statistics underscore the pressing need for improved safety measures for vulnerable road users. Despite advancements in technology, existing sensing systems lack detailed classifications for active transportation users, unlike those for motorized vehicles. Our study aims to bridge this gap by introducing a zero-shot learning approach for cyclist detection. We leverage pre-trained models to identify humans and bicycles without requiring additional training data. We propose five critical metrics for cyclist identification and extend this to video detection and tracking using YOLO models and the ByteTrack algorithm. The research evaluates the zero-shot constraints against traditional methods using metrics such as Multiple Object Tracking Accuracy (MOTA), Identity F1 score (IDF1), and Higher Order Tracking Accuracy (HOTA). Results demonstrate the robustness of our approach, with MOTA scores reaching 0.941, IDF1 up to 0.538, and HOTA as high as 0.924, signifying a significant improvement in cyclist detection and tracking. In summary, our study enhances cyclist detection and tracking accuracy, contributing to better sensing of active transportation users and improved traffic safety.