BigData Analysis in Healthcare: Apache Hadoop , Apache spark and Apache Flink
Анализ больших данных в здравоохранении: Apache Hadoop, Apache Spark и Apache Flink
2019-07-27
SCID: 54.1/tjbgvxqv
Discuss with AI
Apache FlinkApache Hadoop MapReduceApache SparkBig data analyticsHealthcare data processing
Figures from the paper
Abstract (AI)
Introduction: Health care data is increasing. The correct analysis of such data will improve the quality of care and reduce costs. This kind of data has certain features such as high volume, variety, high-speed production, etc. It makes it impossible to analyze with ordinary hardware and software platforms. Choosing the right platform for managing this kind of data is very important. The purpose of this study is to introduce and compare the most popular and most widely used platform for processing big data, Apache Hadoop MapReduce, and the two Apache Spark and Apache Flink platforms, which have recently been featured with great prominence.Material and Methods: This study is a survey whose content is based on the subject matter search of the Proquest, PubMed, Google Scholar, Science Direct, Scopus, IranMedex, Irandoc, Magiran, ParsMedline and Scientific Information Database (SID) databases, as well as Web reviews, specialized books with related keywords and standard. Finally, 80 articles related to the subject of the study were reviewed.Results: The findings showed that each of the studied platforms has features, such as data processing, support for different languages, processing speed, computational model, memory management, optimization, delay, error tolerance, scalability, performance, compatibility, Security and so on. Overall, the findings showed that the Apache Hadoop environment has simplicity, error detection, and scalability management based on clusters, but because its processing is based on batch processing, it works for slow complex analyzes and does not support flow processing, Apache Spark is also distributed as a computational platform that can process a big data set in memory with a very fast response time, the Apache Flink allows users to store data in memory and load them multiple times and provide a complex Fault Tolerance mechanism Continuously retrieves data flow status.Conclusion: The application of big data analysis and processing platforms varies according to the needs. In other words, it can be said that each technology is complementary, each of which is applicable in a particular field and cannot be separated from one another and depending on the purpose and the expected expectation, and the platform must be selected for analysis or whether custom tools are designed on these platforms.
Key Findings
1
Apache Flink supports repeated in-memory data access and provides complex fault tolerance by continuously recovering the state of data streams.
2
Apache Hadoop provides simplicity, error detection, and cluster-based scalability management, but its batch-processing model is unsuitable for stream processing and slow for complex analyses.
3
Apache Spark processes large datasets in memory as a distributed computing platform, enabling very fast response times.
4
Healthcare data characteristics—including high volume, variety, and rapid generation—make conventional hardware and software inadequate for effective analysis.
5
The study reviews and compares Apache Hadoop MapReduce, Apache Spark, and Apache Flink for healthcare big-data processing based on 80 related articles.
Research Object
Big data processing platforms in healthcare, specifically Apache Hadoop MapReduce, Apache Spark, and Apache Flink
Research Subject
Comparative processing capabilities and performance characteristics, including processing speed, computational model, memory management, optimization, latency, fault tolerance, scalability, compatibility, and security
Publication Details
Publication Date
2019-07-27
Journal
Publisher
ISSN
Cited by
57
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest