Automated data processing and feature engineering for deep learning and big data applications: A survey
Автоматическая обработка данных и конструирование признаков для приложений глубокого обучения и больших данных: обзор
2024-01-08
SCID: 54.1/x54uepyf
Discuss with AI
automated data processingautomated machine learningdata augmentationdeep learning pipelinesfeature engineering
Figures from the paper
Abstract (AI)
Modern approach to artificial intelligence (AI) aims to design algorithms that learn directly from data. This approach has achieved impressive results and has contributed significantly to the progress of AI, particularly in the sphere of supervised deep learning. It has also simplified the design of machine learning systems as the learning process is highly automated. However, not all data processing tasks in conventional deep learning pipelines have been automated. In most cases data has to be manually collected, preprocessed and further extended through data augmentation before they can be effective for training. Recently, special techniques for automating these tasks have emerged. The automation of data processing tasks is driven by the need to utilize large volumes of complex, heterogeneous data for machine learning and big data applications. Today, end-to-end automated data processing systems based on automated machine learning (AutoML) techniques are capable of taking raw data and transforming them into useful features for Big Data tasks by automating all intermediate processing stages. In this work, we present a thorough review of approaches for automating data processing tasks in deep learning pipelines, including automated data preprocessing– e.g., data cleaning, labeling, missing data imputation, and categorical data encoding–as well as data augmentation (including synthetic data generation using generative AI methods) and feature engineering–specifically, automated feature extraction, feature construction and feature selection. In addition to automating specific data processing tasks, we discuss the use of AutoML methods and tools to simultaneously optimize all stages of the machine learning pipeline.
Key Findings
1
AutoML systems can transform raw data into useful features for Big Data applications by jointly optimizing all machine-learning pipeline stages.
2
It organizes automated feature engineering into feature extraction, feature construction, and feature selection.
3
It reviews automated preprocessing methods for data cleaning, labeling, missing-data imputation, and categorical-data encoding.
4
The paper covers data augmentation approaches, including synthetic data generation with generative AI methods, for complex and heterogeneous datasets.
5
The survey examines automation of previously manual deep-learning data-processing stages, including collection, preprocessing, augmentation, and feature engineering.
Research Object
automated data processing systems and deep learning pipelines for Big Data applications
Research Subject
automation and joint optimization of data preprocessing, data augmentation, and feature engineering across machine learning pipelines
Publication Details
Publication Date
2024-01-08
Journal
Publisher
ISSN
Cited by
163
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai17
ImageNet classification with deep convolutional neural networks2017
Exploiting Generative AI to Scale up Intelligent Tutoring Systems2023
Scikit-learn: Machine Learning in Python2012
ImageNet: A large-scale hierarchical image database2009
PyTorch: An Imperative Style, High-Performance Deep Learning Library2019
LLaMA: Open and Efficient Foundation Language Models2023
Representation Learning: A Review and New Perspectives2013
A survey on Image Data Augmentation for Deep Learning2019
Proceedings of the 24th international conference on Machine learning2007
Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining2016
A Survey on Evaluation of Large Language Models2024
mixup: Beyond Empirical Risk Minimization2017
Randaugment: Practical automated data augmentation with a reduced search space2020
Deep learning applications and challenges in big data analytics2015
Automated machine learning: Review of the state-of-the-art and opportunities for healthcare2020
Deep Learning Image Reconstruction for CT: Technical Principles and Clinical Prospects2023
Qualitative Spatial Reasoning over Questions (Short Paper)2022