Irregularities Detection in Non-Verbal Cues Using Machine Learning رسالة ماجستير

Abstract

Effective human communication across varied fields such as healthcare, politics, law, business, education, and social interactions It depends on understanding and expressing situations). Detecting facial gestures makes it easy to recognize the communicator’s emotional motivational (happiness, anger, fear, sadness disgust, surprise, and contempt), and health state (neurological weakness, and disorders, and strokes). Since the diagnosis clinically of facial tics disorders involves various complex processes, patient behavior observations and evaluation usually require time and effective cooperation between the doctors and the patients. This study proposed an effective and novel framework for detecting the irregularities in (head position, Eyelid movement detection, iris position, yawning drowsiness, mouth deviation(mouth droopy corners)) and applied as real-time assisting system for on real-time front face laptop camera, and uploaded videos and uploaded images. first, Mediapipe face landmark model is initiated. Preprocessed frames or images using Open CV library, retrieving and extracting and landmarks from the Mediapipe models to identify the specific points or landmarks on a face, find distances between the chosen specific points, detect irregularities depending on the distance between the chosen points, classify the movement among the distance or angles and the predefined thresholds and the number of frames the facial movement lasts. The system was evaluated on a diverse dataset of labeled images. Following preprocessing and comparison with defined thresholds, evaluation metrics (accuracy, precision, recall, and F1) were calculated. Results indicated high accuracy: 100% for head position, 96% for iris position, 86% for eyelid status, 96% for yawning detection, 88% for mouth deviation, 97% for drowsy eye detection, and 100% for mouth movement