Transforming legal text interactions: leveraging natural language processing and large language models for legal support in Palestinian cooperatives

Abstract

In recent years, there has been a remarkable transformation in our interaction with legal texts due to the widespread utilization and adoption of natural language processing technology. This technology has advanced the analysis and enhanced the understanding of complex legal terminology and contexts. Moreover, the emergence of recent generative large language models (LLMs), particularly ChatGPT, has also introduced a revolutionary contribution to the way legal texts can be processed and comprehended. This paper focuses on the development of a cooperative legal question-answering LLM-based chatbot. Our work involves formulating a set of legal questions pertaining to Palestinian cooperatives and their associated regulations. We compare the auto-generated answers provided by the chatbot with correspondences prepared by a legal expert. To assess the chatbot’s performance, we evaluate its responses to 50 queries generated by the legal expert and compare them to their relevance judgments. The results indicate that the chatbot achieved an impressive overall accuracy rate of 82% in answering the queries, with an F1 score equivalent to 79%.