Artificial Intelligence System for Arabic Sign Language Recognition to Enhance Education for Deaf and Hard-of-Hearing Students. رسالة ماجستير

Abstract

This study aims to examine the development and effectiveness of the “SignPulse” system, an intelligent educational system designed to support deaf and hard-of-hearing students in learning STEM education using Palestinian Sign language(PSL). The research adopts a mixed-method approach, combining qualitative and quantitative analyses to evaluate the system's impact on student engagement, comprehension, and interaction. Data were collected through a structured video dataset for PSL, along with usability questionnaires completed by experts in sign language, educational technology, and STEM education. This dataset enabled the training and evaluation of a hybrid Siamese vision transformer (Siamese-ViT) model for accurate sign recognition learning, while the system's interactive component, powered by GPT+RAG, provided personalized explanations, contextual questions, and immediate feedback to enhance the learning experience. The findings suggest that integrating AI technology significantly enhances learning accessibility, engagement, and comprehension for students with hearing disabilities. Moreover, the study highlights the pivotal role of specialized guidance and culturally appropriate sign language resources in maximizing the system's effectiveness. It also discusses challenges related to technical limitations, user adaptation, and integration into classroom practices. The study concluded that AI-driven sign language systems, such as “ SignPulse,” have significant potential to improve inclusive education for students with hearing disabilities, support the implementation of adaptive learning strategies, and contribute to providing equitable educational opportunities in Palestine.