In-Memory Big Data Management and Processing: A Survey

Управление и обработка больших данных в оперативной памяти: обзор
Beng Chin Ooi, Meihui Zhang, Hao Zhang, Gang Chen, Kian‐Lee Tan
2015-04-29

concurrency controlfault tolerancein-memory big data managementin-memory data processinginteractive data analytics
Growing main memory capacity has fueled the development of in-memory big data management and processing. By eliminating disk I/O bottleneck, it is now possible to support interactive data analytics. However, in-memory systems are much more sensitive to other sources of overhead that do not matter in traditional I/O-bounded disk-based systems. Some issues such as fault-tolerance and consistency are also more challenging to handle in in-memory environment. We are witnessing a revolution in the design of database systems that exploits main memory as its data storage layer. Many of these researches have focused along several dimensions: modern CPU and memory hierarchy utilization, time/space efficiency, parallelism, and concurrency control. In this survey, we aim to provide a thorough review of a wide range of in-memory data management and processing proposals and systems, including both data storage systems and data processing frameworks. We also give a comprehensive presentation of important technology in memory management, and some key factors that need to be considered in order to achieve efficient in-memory data management and processing.
1
Efficient in-memory data management requires careful consideration of memory-management technologies and system-design factors, including time-space trade-offs and modern hardware characteristics.
2
Fault tolerance and consistency are more difficult to provide efficiently in in-memory environments.
3
Growing main-memory capacity has enabled in-memory big-data systems that eliminate disk-I/O bottlenecks and support interactive analytics.
4
In-memory systems expose CPU, memory-hierarchy, and other overheads that are less significant in traditional disk-based, I/O-bound systems.
5
The survey reviews a broad range of in-memory storage systems and processing frameworks, covering CPU and memory utilization, efficiency, parallelism, and concurrency control.

In-memory big data management and processing systems (including in-memory data storage systems and data processing frameworks)

design approaches, performance factors, and challenges of in-memory data management and processing, including memory management, CPU/memory hierarchy utilization, efficiency, parallelism, concurrency control, fault tolerance, and consistency

Publication Details
Publication Date
2015-04-29
Journal
Publisher
ISSN
Cited by
425
Access Type
Author Information
Authors
Beng Chin Ooi
Meihui Zhang
Hao Zhang
Gang Chen
Kian‐Lee Tan
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%