Big Data Analysis and Data Mining

Deep Learning for Big Data Applications

Typically, training deep neural networks requires large amounts of data that often do not fit in memory. You do not need multiple computers to solve problems using data sets too large to fit in memory. Instead, you can divide your training data into mini-batches that contain a portion of the data set. By iterating over the mini-batches, networks can learn from large data sets without needing to load all data into memory at once. If your data is too large to fit in memory, use a data store to work with mini-batches of data for training and inference. MATLAB® provides many different types of data store tailored for different applications. For more information about data stores for different applications, see Data stores for Deep Learning. augmentedImageDatastore is specifically designed to pre-process and augment batches of image data for machine learning and computer vision applications.