A Real-Time Framework for Nighttime Vehicle Tracking Using Optimized Long Short-term Memory Network

Abstract: Traffic surveillance of intelligent transportation systems heavily relies on video data analysis and processing of the camera footage to estimate the motion parameters of the traffic flow, such as the vehicle movement speed, the traffic volume, and the vehicle presence in the geo-fence zone. Existing object detectors and trackers contributing to the camera-based traffic surveillance systems, demonstrate high performance in the lab environment, in the daytime, and in good weather conditions, but struggle to operate in more complex scenarios, such as low-illumination environment, severe weather conditions, and a high-density traffic flow. Light reflections, rain, strong gusts of wind; and the object vehicle overlapping and vehicle occlusion in the road scene may impair the detection and tracking ability of the intelligent transportation system. In this study, we consider the problem of multiple vehicle tracking in different complex road scenarios using a traffic surveillance video feed obtained by a monocular RGB camera in low-illumination conditions. We propose an approach based on the encoder-decoder architecture which analyzes patterns within the past object trajectory and predicts the future trajectory using the Long short-term memory (LSTM) network. The ultimate goal of the developed method is to establish continuous correspondences between vehicle objects across a sequence of frames in the video sequence. The joint use of motion features for the LSTM deep neural model and appearance features produced by the re-identification network promotes the generation of high-quality object trajectories and operation in a constantly changing traffic scene environment, having different traffic flow density and illumination conditions. Consequently, produced vehicle trajectories can be efficiently utilized to estimate the traffic flow parameters, such as vehicle movement speed. To demonstrate the efficiency of the developed approach, the experiments carried out on the specialized traffic scene datasets in nighttime conditions demonstrate significant and robust results.