A practical framework for robust lane detection and tracking in adverse weather

Abstract

Recently, autonomous driving systems are being progressively incorporated into vehicles. In the autonomous driving system, detecting lanes is a critical part that provides feedback on the vehicle’s position and enriches the path planning module with information on the road’s trajectory, high reliability and accuracy are required. Significant advancements in precision and effectiveness have been made through recent deep learning methods. However, lane detection in real-world scenarios still faces various challenges, including extreme lighting conditions, eroded lane markings, occlusions and adverse weather conditions, etc. Heavy rain as a representative of extreme weather conditions, can interfere with the sensory image signal by making it more challenging to detect the lanes. This can compromise the safety of the autonomous driving system. But due to the imbalanced popular datasets, many lane detectors are not evaluated adequately under rainy conditions, raising doubts about their robustness. Moreover, effectiveness has been a problem for many models. For the purpose of enhancing reliability and safety of the AV, lane detection need to perform in real-time to pass road information to other systems, enabling them to respond more readily to handle hazards or obstacles. This project proposed a lane detection and tracking framework combining the DL-based lane detector with a lane tracker. The lane tracking module was introduced as a post-processing method based on Kalman filtering, applied on the detection output of UFLD and can augment the output without touching the detection network. Additionally, to address the shortage of samples in adverse weather conditions, a synthetic rainy dataset named Tusimple-Rain was used as a supplementary dataset. Considering its superior data amount and diversity, after being projected to 2D and converted to TuSimple format, ONCE-3DLanes dataset was used for training and testing in our work as well. The lane detectors and the detection system developed by us were all pre-trained with TuSimple, TuSimple-Rain and ONCE-3DLanes, and were evaluated on the three datasets and various scene categories of ONCE-3DLanes. Results show that our approach outperformed the lane detector models without tracking on ONCE-3DLanes in terms of the accuracy, FP value and FN value under different weather conditions, showing its robustness. With frame skip set to five, the developed system also achieved an increased average FPS.