Enhancing Mobile Learning Applications with Large Language Models: Design and Evaluation of AIChemApp

Abstract

Traditional mobile learning is constrained by the need to update its content manually and frequently due to rigid structures. Conversely, integrating LLMs with mobile learning applications is expected to provide a catalyst for both developers and teachers on programming and technical perspectives for such applications by considering automatic content generation, dynamic assessments, and reducing technical complexities. This study presents the feasibility and effectiveness of integrating LLMs, specifically ChatGPT, with mobile learning applications developed for chemistry courses. The provided solution leverages LLM’s ability to provide content generation and real-time responses related to queries about the periodic table, electron configuration, chemical equations and game-based assessment, ultimately enhancing the effectiveness and engagement of mobile learning. An experimental evaluation was conducted to assess the accuracy of ChatGPT’s responses within the mobile learning application for chemistry. The results demonstrated that ChatGPT achieves an impressive 93.33% accuracy in providing information about elements and their categories from the periodic table. However, the accuracy rates for the chemical equation balancing and electron configuration tasks were relatively low, at 75.71% and 68.91%, respectively, indicating areas for further optimization and refinement. This research highlights the technical implications of LLM integration with mobile learning applications, including potential challenges and opportunities for developers and educators in terms of API communications, prompt engineering, and the quality and accuracy of AI-generated responses.