To implement reliable autonomous driving technology, it's essential to accurately reconstruct the 3D environment around the vehicle in real-time. At 42dot, we are advancing autonomous driving technology by continuously collecting data and tracking model performance to develop high-performance 3D perception models that operate in real-time.
1. The Problem of Duplicate Predictions in Object Detection Models and Its Impact
Autonomous vehicles reconstruct their surrounding environment to identify the location of obstacles, predict the movement of other vehicles and pedestrians, and plan a safe driving route. Therefore, the accuracy of the 3D object detection model, which is the first step in the autonomous driving system, significantly affects the vehicle's operation.
At 42dot, we continuously collect data to evaluate and improve our models. When evaluating the model with recently added data, we found that in cases where 4-5 pedestrians were gathered in a small area, more duplicate predictions occurred than the actual number of pedestrians. These duplicate predictions can ca1use the autonomous vehicle to recognize non-existent objects, potentially leading to abnormal movements. While we can reduce false detections by increasing the prediction score threshold in the post-processing stage, this may increase the number of undetected pedestrians, posing a safety risk.
When the score of predicted pedestrians is low, the probability of the detected object being a duplicate prediction increases.
2. Active Learning Pipeline
To improve the model's weaknesses, 42dot operates an Active Learning Pipeline that continuously collects data from similar cases and includes it in network training. To efficiently incorporate newly collected data into model training with minimal human intervention, we use a high-performance model that operates offline. The knowledge learned from this high-performance model is then distilled into the real-time model operating in the vehicle, improving its performance.
3. Introduction of Transformer Model Structure
Recent object detection models [1, 2] have significantly improved performance by introducing transformer structures. Transformers allow object queries to exchange information between objects through self-attention and utilize the entire information of the feature map, providing richer information than CNN structures. In particular, transformer-based object detection models use one-to-one matching between ground truth and predictions during training, which is highly effective in suppressing duplicate predictions. These characteristics of transformer models are expected to effectively solve the pedestrian duplicate prediction problem and greatly improve pedestrian detection performance.
4. Utilizing Denoising Techniques
However, contrary to expectations, simply introducing a transformer-based object detection model did not result in significant performance improvements. To identify the cause of the problem, we analyzed the number of object queries per class in the model. In densely populated urban areas, we found that the number of object queries generated around pedestrians was very high, causing instability in the learning process when matching the generated object queries with ground truth. To solve this, we applied the Dense Distinct Query [3] technique introduced at CVPR 2023, which helped resolve the issue and secure learning stability.
5. Performance Improvement Results
By introducing these various technologies, we have significantly reduced the number of pedestrian duplicate predictions while greatly improving the model's precision. Additionally, the prediction quality of pedestrian location, size, and direction of movement has been further improved. This enhanced high-performance model is applied to vehicles to improve driving performance.
42dot is committed to continuously developing and researching technology to make autonomous driving safer.
Youngseok Kim, Juhan Cha (Intern) ⎢ AD Algorithm
We are developing technology to help autonomous vehicles understand their surroundings faster and more accurately.
Reference
[1] Nicolas Carion et al., “End-to-End Object Detection with Transformers,” ECCV, 2020.
[2] Xuyang Bai et al., “TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers,” CVPR, 2022.
[3] Shilong Zhang et al., “Dense Distinct Query for End-to-End Object Detection,” CVPR, 2023.