Performance Evaluation of Lightweight YOLOv5n Models in CARLA Simulator for Autonomous Driving Applications

Abstract

Real-time object detection is critical for autonomous driving systems, especially in realistic simulation environments like the CARLA platform. This study aims to evaluate the performance of lightweight models (YOLOv5n) in detecting complex objects, including multi-colored traffic signals, vehicles, and pedestrians, within a CARLA simulation. We trained two models using CARLA-derived datasets to assess the trade-off between accuracy and computational efficiency. Model 1 (YOLOv5n-1864-T): This model was trained on 1,864 training samples. It demonstrated superior accuracy (mAP50 = 0.935) and a faster inference speed (0.022-0.025 seconds per frame). Model 2 (YOLOv5n-1600-TV): This model was trained on 1,600 training samples along with 480 validation and test samples. It achieved better generalization (mAP50 = 0.942) with balanced confidence scores (0.75-0.96), despite a slightly slower inference speed (0.04-0.07 seconds). During inference tests on 8 dynamic CARLA frames, both models achieved high performance (F1-Score: 0.92-0.94). Model 1 excelled at detecting red traffic signals, while Model 2 showed greater adaptability to complex pedestrian angles. The results confirm that lightweight models can compete with heavier models while maintaining high computational efficiency (4.2 GFLOPs). The study recommends integrating validation data with expanded training sets to improve generalization without sacrificing speed, providing a practical framework for deploying such models in simulation-based autonomous driving systems.