Development of Facebook Data Extraction, Analysis and Visualization Tool رسالة ماجستير

Abstract

Facebook has become an indispensable tool in our everyday lives. It is often used as a platform where people can share their opinions, ideas, special moments, feelings and many other forms of social expressions. It is also used in various fields such as business, education and health care for exchanging and sharing specialized knowledge and experiences. The frequent use of Facebook by a large number of people on a daily basis results in massive amounts of data. For instance, an amount of 4 petabytes of data were generated by an average of 1.23 billion daily active users in December 2016. Facebook is thus seen as a rich data source which is complex to process and analyze. In this respect, big data and data mining are emerging approaches aimed at the analysis of massive datasets to obtain helpful and useful information that can be used in different areas. Along the dominance of Facebook as a social network site where such massive datasets are generated, an increasing number of tools have been developed to process and analyze Facebook generated data. These tools can be classified based on their use into a number of categories. First, data extraction tools which are used to fetch data out of Facebook and store it in different formats. A second category of tools refer to tools that allow data analysis. Finally, data visualization tools which are important to generate a readable and useful output. With the spread use of social network sites, more and more researches started to employ it in many fields. Predicting the future is one of these fields, depending on social network sites content we can predict the real-world outcomes. On the other hand, studying tie strength between friends is another application. Some people use social network sites for commercial purposes, for marketing and studying opponents. Healthcare is another filed addressed by social network sites vii where it helps healthcare organizations, clinicians and patients. Social network sites has used as a communication media within the instructors and students to create online classes. Many of these tools have been studied and investigated in order to develop a single tool that combines extraction, analysis and visualization features and to overcome the existing tools limitations. However, developing such a tool is complex since none of the investigated tools can be used independently. Our goal is to develop a universal tool to extract the data out of Facebook using Facebook graph API and to analyze the collected data based on different algorithms to get time measures as well as data measures. Finally, visualize the analyzed data in form of charts to be used by analysts and decision makers. In this thesis, we implement a new web-based tool based on Spring framework that is capable of fetching data out of Facebook using Facebook API and then analyze these data to get the Facebook page key performance indicators such as time related and data related information. Finally, the tool is capable of visualizing the key performance indicators to the user in the form of charts. The new developed tool allows users to run their own inquires with a few clicks and without any background in datamining or programming. This was achieved via our tool that has its own crawler, analyzer and visualizer. The crawler has been developed on the top of “httpClient” to get the data from Facebook. Currently, it depends on Facebook graph API to fetch the pages-related data since it is the only available API that can be trusted. Earlier studies suggested the design of a new crawler that crawls Facebook DOM. This kind of crawlers faced some issues with Facebook due to access rate limitations. The crawlers will therefore be blocked by Facebook after a period of time. viii The tool was successfully used to fetch, analyze and visualize data out of tens of thousands of posts and comments from different kinds of Facebook pages, including university pages, news pages, and commercial pages. The tool successfully and thoroughly generated professional charts out of these pages that can practically help in the decision making process.