Enhancing Traffic Safety with a Multimodal Large Language Model for Real Time Hazard Detection رسالة ماجستير

Abstract

Traffic safety remains a critical global issue, with traditional detection systems often falling short in complex, real-world environments due to limited generalizability and high computational demands. This thesis introduces HazardNet, a lightweight, edge compatible Multimodal Large Language Model (MLLM) fine-tuned from Qwen2-VL-2B using parameter-efficient techniques like Low-Rank Adaptation (LoRA) and Quantized Low-Rank Adaptation (QLoRA). HazardNet is designed for real-time, contextual hazard detection by combining visual and textual inputs, making it suitable for deployment on low-resource devices such as in-vehicle systems without GPUs. To support this model, the study presents HazardQA, a new Vision Question Answering (VQA) dataset derived from driving scenarios in the DRAMA dataset. HazardQA includes over 7,000 annotated question-answer pairs enriched with reasoning chains and safety-specific labels, covering a wide range of hazards and traffic contexts. Experiments show that HazardNet performs competitively on safety-critical tasks such as scene understanding, hazard identification, and action recommendation, rivaling larger models like GPT-4o while maintaining low computational requirements. The research highlights how compact MLLMs, when adapted with domain-specific data and efficient fine-tuning methods, can provide high interpretability and scalability for traffic safety systems. Key contributions include the development of HazardNet, the open-source release of HazardQA, and a demonstration of real-world deployment strategies for AI based hazard detectio