Deep Learning in MATLAB
What Is Deep Learning?
Deep learning is a branch of machine learning that teaches computers to do what comes naturally to humans: learn from experience. Deep learning uses neural networks to learn useful representations of features directly from data. Neural networks combine multiple nonlinear processing layers, using simple elements operating in parallel and inspired by biological nervous systems. Deep learning models can achieve stateoftheart accuracy in object classification, sometimes exceeding humanlevel performance.
Deep Learning Toolbox™ provides simple MATLAB^{®} commands for creating and interconnecting the layers of a deep neural network. Examples and pretrained networks make it easy to use MATLAB for deep learning, even without knowledge of advanced computer vision algorithms or neural networks.
For a free handson introduction to practical deep learning methods, see Deep Learning Onramp. To quickly get started deep learning, see Try Deep Learning in 10 Lines of MATLAB Code.
Start Deep Learning Faster Using Transfer Learning
Transfer learning is commonly used in deep learning applications. You can take a pretrained network and use it as a starting point to learn a new task. Finetuning a network with transfer learning is much faster and easier than training from scratch. You can quickly make the network learn a new task using a smaller number of training images. The advantage of transfer learning is that the pretrained network has already learned a rich set of features that can be applied to a wide range of other similar tasks. For an interactive example, see Transfer Learning with Deep Network Designer. For a programmatic example, see Train Deep Learning Network to Classify New Images.
To choose whether to use a pretrained network or create a new deep network, consider the scenarios in this table.
Use a Pretrained Network for Transfer Learning  Create a New Deep Network  

Training Data  Hundreds to thousands of labeled data (small)  Thousands to millions of labeled data 
Computation  Moderate computation (GPU optional)  Compute intensive (requires GPU for speed) 
Training Time  Seconds to minutes  Days to weeks for real problems 
Model Accuracy  Good, depends on the pretrained model  High, but can overfit to small data sets 
To explore a selection of pretrained networks, use Deep Network Designer.
Deep Learning Workflows
To learn more about deep learning application areas, see Deep Learning Applications.
Domain  Example Workflow  Learn More  

Image classification, regression, and processing 
 Apply deep learning to image data tasks. For example, use deep learning for image classification and regression.  Get Started with Transfer Learning Pretrained Deep Neural Networks Create Simple Deep Learning Network for Classification 
Sequences and time series 
 Apply deep learning to sequence and time series tasks. For example, use deep learning for sequence classification and time series forecasting.  
Computer vision 
 Apply deep learning to computer vision applications. For example, use deep learning for semantic segmentation and object detection.  Getting Started with Semantic Segmentation Using Deep Learning (Computer Vision Toolbox) Recognition, Object Detection, and Semantic Segmentation (Computer Vision Toolbox) 
Audio processing 
 Apply deep learning to audio and speech processing applications. For example, use deep learning for speaker identification, speech command recognition, and acoustic scene recognition.  Audio Processing Using Deep Learning Introduction to Deep Learning for Audio Applications (Audio Toolbox) 
Automated driving 
 Apply deep learning to automated driving applications. For example, use deep learning for vehicle detection and semantic segmentation.  
Signal processing 
 Apply deep learning to signal processing applications. For example, use deep learning for waveform segmentation, signal classification, and denoising speech signals.  Signal Processing Using Deep Learning Classify Time Series Using Wavelet Analysis and Deep Learning 
Wireless communications 
 Apply deep learning to wireless communications systems. For example, use deep learning for positioning, spectrum sensing, autoencoder design, and digital predistortion (DPD).  Wireless Communications Using Deep Learning Spectrum Sensing with Deep Learning to Identify 5G and LTE Signals ThreeDimensional Indoor Positioning with 802.11az Fingerprinting and Deep Learning (WLAN Toolbox) 
Reinforcement learning 
 Train deep neural network agents by interacting with an unknown dynamic environment. For example, use reinforcement learning to train policies to implement controllers and decisionmaking algorithms for complex applications such as resource allocation, robotics, and autonomous systems.  
Computational finance 
 Apply deep learning to financial workflows. For example, use deep learning for applications including instrument pricing, trading, and risk management.  Computational Finance Using Deep Learning Compare Deep Learning Networks for Credit Default Prediction 
Lidar processing 
 Apply deep learning algorithms to process lidar point cloud data. For example, use deep learning for semantic segmentation, object detection on 3D organized lidar point cloud data.  Lidar Processing Using Deep Learning Aerial Lidar Semantic Segmentation Using PointNet++ Deep Learning 
Text analytics 
 Apply deep learning algorithms to text analytics applications. For example, use deep learning for text classification, language translation, and text generation.  
Predictive maintenance 
 Apply deep learning to predictive maintenance applications. For example, use deep learning for fault detection and remaining useful life estimation. 
Deep Learning Apps
Process data, visualize and train networks, track experiments, and quantize networks interactively using apps.
You can process your data before training using apps to label ground truth data. For more information on choosing a labeling app, see Choose an App to Label Ground Truth Data.
Name  Description  Learn More  

Deep Network Designer 
 Build, visualize, edit, and train deep learning networks.  Transfer Learning with Deep Network Designer Train Network for Time Series Forecasting Using Deep Network Designer 
Experiment Manager 
 Create deep learning experiments to train networks under multiple initial conditions and compare the results.  
Deep Network Quantizer 
 Reduce the memory requirement of a deep neural network by quantizing weights, biases, and activations of convolution layers to 8bit scaled integer data types.  
Reinforcement Learning Designer (Reinforcement Learning Toolbox) 
 Design, train, and simulate reinforcement learning agents.  Design and Train Agent Using Reinforcement Learning Designer (Reinforcement Learning Toolbox) 
Image Labeler (Computer Vision Toolbox) 
 Label ground truth data in a collection of images.  Get Started with the Image Labeler (Computer Vision Toolbox) 
Video Labeler (Computer Vision Toolbox) 
 Label ground truth data in a video, in an image sequence, or from a custom data source reader.  Get Started with the Video Labeler (Computer Vision Toolbox) 
Ground Truth Labeler (Automated Driving Toolbox) 
 Label ground truth data in multiple videos, image sequences, or lidar point clouds.  Get Started with the Ground Truth Labeler (Automated Driving Toolbox) 
Lidar Labeler (Lidar Toolbox) 
 Label objects in a point cloud or a point cloud sequence. The app reads point cloud data from PLY, PCAP, LAS, LAZ, ROS and PCD files.  Get Started with the Lidar Labeler (Lidar Toolbox) 
Signal Labeler (Signal Processing Toolbox) 
 Label signals for analysis or for use in machine learning and deep learning applications.  Using Signal Labeler App (Signal Processing Toolbox) 
Train Classifiers Using Features Extracted from Pretrained Networks
Feature extraction allows you to use the power of pretrained networks without
investing time and effort into training. Feature extraction can be the fastest way to
use deep learning. You extract learned features from a pretrained network, and use those
features to train a classifier, for example, a support vector machine (SVM — requires
Statistics and Machine Learning Toolbox™). For example, if an SVM trained using alexnet
can
achieve >90% accuracy on your training and validation set, then finetuning with
transfer learning might not be worth the effort to gain some extra accuracy. If you
perform finetuning on a small dataset, then you also risk overfitting. If the SVM
cannot achieve good enough accuracy for your application, then finetuning is worth the
effort to seek higher accuracy.
For an example, see Extract Image Features Using Pretrained Network.
Deep Learning with Big Data on CPUs, GPUs, in Parallel, and on the Cloud
Training deep networks is computationally intensive and can take many hours of computing time; however, neural networks are inherently parallel algorithms. You can use Parallel Computing Toolbox™ to take advantage of this parallelism by running in parallel using highperformance GPUs and computer clusters. To learn more about deep learning in parallel, in the cloud, or using a GPU, see Scale Up Deep Learning in Parallel, on GPUs, and in the Cloud.
Datastores in MATLAB^{®} are a convenient way of working with and representing collections of data that are too large to fit in memory at one time. To learn more about deep learning with large data sets, see Deep Learning with Big Data.
Deep Learning Using Simulink
Implement deep learning functionality in Simulink^{®} models by using blocks from the Deep Neural Networks block library, included in the Deep Learning Toolbox™, or by using the Deep Learning Object Detector block from the Analysis & Enhancement block library included in the Computer Vision Toolbox™.
For more information, see Deep Learning with Simulink.
Block  Description 

Classify data using a trained deep learning neural network  
Predict responses using a trained deep learning neural network  
Classify data using a trained deep learning recurrent neural network  
Predict responses using a trained recurrent neural network  
Deep Learning Object Detector (Computer Vision Toolbox)  Detect objects using trained deep learning object detector 
Deep Learning Interpretability
Deep learning networks are often described as "black boxes" because the reason that a network makes a certain decision is not always obvious. You can use interpretability techniques to translate network behavior into output that a person can interpret. This interpretable output can then answer questions about the predictions of a network.
The Deep Learning Toolbox provides several deep learning visualization methods to help
you investigate and understand network behaviour. For example, gradCAM
,
occlusionSensitivity
, and imageLIME
.
For more information, see Deep Learning Visualization Methods.
Deep Learning Customization
You can train and customize a deep learning model in various ways. For example, you
can build a network using builtin layers or define custom layers. You can then train
your network using the builtin training function trainNetwork
or define a deep learning model as a function and use a
custom training loop. For help deciding which method to use, consult the following
table.
Method  Use Case  Learn More 

Builtin training and layers  Suitable for most deep learning tasks. 

Custom layers  If Deep Learning Toolbox does not provide the layer you need for your task, then you can create a custom layer.  
Custom training loop  If you need additional customization, you can build and train your network using a custom training loop. 
For more information, see Train Deep Learning Model in MATLAB.
Deep Learning Import and Export
You can import networks and network architectures from TensorFlow™Keras, TensorFlow 2, the ONNX™ (Open Neural Network Exchange) model format, and Caffe. You can also export trained networks to the ONNX model format.
Import Functions
External Deep Learning Platform and Model Format  Import Model as Network  Import Model as Layer Graph 

Network in ONNX model format  importONNXNetwork  importONNXLayers 
TensorFlow network in the saved model format  importTensorFlowNetwork  importTensorFlowLayers 
TensorFlowKeras network in HDF5 or JSON format  importKerasNetwork  importKerasLayers 
Caffe network with architecture in .prototxt file and
weights in .caffemodel file  importCaffeNetwork  importCaffeLayers 
The importTensorFlowNetwork
and
importTensorFlowLayers
functions are recommended over the
importKerasNetwork
and importKerasLayers
functions:
The
importTensorFlowNetwork
andimportTensorFlowLayers
functions support TensorFlow 2. For more information on the supported TensorFlow versions, see Limitations. For more information on the advantages of migrating from TensorFlow 1 to TensorFlow 2, see https://www.tensorflow.org/guide/migrate.The
importTensorFlowNetwork
andimportTensorFlowLayers
functions support the saved model format, whereasimportKerasNetwork
andimportKerasLayers
support the HDF5 format. TensorFlow recommends the saved model format. Consider converting your TensorFlow network to the saved model format before importing the network into MATLAB.The
importTensorFlowNetwork
andimportTensorFlowLayers
functions can automatically generate custom layers when you import custom TensorFlow layers or when the software cannot convert TensorFlow layers into equivalent builtin MATLAB layers. For an example, see Import TensorFlow Network with Autogenerated Custom Layers.
The importONNXNetwork
and importONNXLayers
functions
can also generate custom layers when the software cannot convert ONNX operators into equivalent builtin MATLAB layers. For an example, see Import ONNX Network with Autogenerated Custom Layers. In rare cases, when
importONNXNetwork
and importONNXLayers
cannot
import an ONNX model into layers, you can use importONNXFunction
to import the model as a function. For more information
on how to select an ONNX import function, see Select Function to Import ONNX Pretrained Network.
By using ONNX as an intermediate format, you can interoperate with other deep learning frameworks that support ONNX model export or import, such as TensorFlow, PyTorch, Caffe2, Microsoft^{®} Cognitive Toolkit (CNTK), Core ML, and Apache MXNet™. Export a trained Deep Learning Toolbox network or graph of network layers to the ONNX model format using the exportONNXNetwork
function.