Deep Learning

Neural networks that operate on two or three layers of connected neuron layers are known as shallow neural networks. Deep learning networks can have many layers, even hundreds. Both are machine learning techniques that learn directly from input data.

Deep learning is getting lots of attention, and for good reason. It’s achieving results that were not possible before.

Deep learning is especially well suited to complex identification applications such as face recognition, text translation, and voice recognition. It’s also a key technology used in advanced driver assistance systems and tasks including lane classification and traffic sign recognition.

A neural network combines several processing layers, using simple elements operating in parallel and inspired by biological nervous systems. It consists of an input layer, one or more hidden layers, and an output layer. The layers are interconnected via nodes, or neurons, with each layer using the output of the previous layer as its input.

Techniques Used with Neural Networks

Common machine learning techniques for designing neural network applications include supervised and unsupervised learning, classification, regression, pattern recognition, and clustering.

Supervised Learning

Supervised neural networks are trained to produce desired outputs in response to sample inputs, making them particularly well suited for modeling and controlling dynamic systems, classifying noisy data, and predicting future events. Deep Learning Toolbox™ includes four types of supervised networks: feedforward, radial basis, dynamic, and learning vector quantization.

Classification

Classification is a type of supervised machine learning in which an algorithm “learns” to classify new observations from examples of labeled data.

Regression

Regression models describe the relationship between a response (output) variable and one or more predictor (input) variables.

Pattern Recognition

Pattern recognition is an important component of neural network applications in computer vision, radar processing, speech recognition, and text classification. It works by classifying input data into objects or classes based on key features, using either supervised or unsupervised classification.

For example, in computer vision, supervised pattern recognition techniques are used for optical character recognition (OCR), face detection, face recognition, object detection, and object classification. In image processing and computer vision, unsupervised pattern recognition techniques are used for object detection and image segmentation.

Unsupervised Learning

Unsupervised neural networks are trained by letting the neural network continually adjust itself to new inputs. They are used to draw inferences from data sets consisting of input data without labeled responses. You can use them to discover natural distributions, categories, and category relationships within data.

Deep Learning Toolbox includes two types unsupervised networks: competitive layers and self-organizing maps.

Clustering

Clustering is an unsupervised learning approach in which neural networks can be used for exploratory data analysis to find hidden patterns or groupings in data. This process involves grouping data by similarity. Applications for cluster analysis include gene sequence analysis, market research, and object recognition.

With tools and functions for managing large data sets, MATLAB^® offers specialized toolboxes for working with machine learning, neural networks, deep learning, computer vision, and automated driving.

With just a few lines of code, MATLAB lets you develop neural networks without being an expert. Get started quickly, create and visualize models, and deploy models to servers and embedded devices.

With MATLAB, you can integrate results into your existing applications. MATLAB automates deploying your neural network models on enterprise systems, clusters, clouds, and embedded devices.

Typical Workflow for Designing Neural Networks

Each neural network application is unique, but developing the network typically follows these steps:

1. Access and prepare your data
2. Create the neural network
3. Configure the network’s inputs and outputs
4. Tune the network parameters (the weights and biases) to optimize performance
5. Train the network
6. Validate the network’s results
7. Integrate the network into a production system

Classification and Clustering of Shallow Networks

MATLAB and Deep Learning Toolbox provide command-line functions and apps for creating, training, and simulating shallow neural networks. The apps make it easy to develop neural networks for tasks such as classification, regression (including time-series regression), and clustering. After creating your networks in these tools, you can automatically generate MATLAB code to capture your work and automate tasks.

Preprocessing, Postprocessing, and Improving Your Network

Preprocessing the network inputs and targets improves the efficiency of shallow neural network training. Postprocessing enables detailed analysis of network performance. MATLAB and Simulink^® provide tools to help you:

• Reduce the dimensions of input vectors using principal component analysis
• Perform regression analysis between the network response and the corresponding targets
• Scale inputs and targets so they fall in the range [-1,1]
• Normalize the mean and standard deviation of the training data set
• Use automated data preprocessing and data division when creating your networks

Improving the network’s ability to generalize helps prevent overfitting, a common problem in neural network design. Overfitting occurs when a network has memorized the training set but has not learned to generalize to new inputs. Overfitting produces a relatively small error on the training set but a much larger error when new data is presented to the network.

Two solutions to improve generalization include:

• Regularization modifies the network’s perforance function (the measure of error that the training process minimizes). By including the sizes of the weights and biases, regularization produces a network that performs well with the training data and exhibits smoother behavior when presented with new data.
• Early stopping uses two different data sets: the training set, to update the weights and biases, and the validation set, to stop training when the network begins to overfit the data

Code Generation and Deployment

Using Deep Learning Toolbox with MATLAB Coder™, GPU Coder™, and MATLAB Compiler™, you can deploy trained networks to embedded systems, or integrate them with a wide range of production environments. You can use MATLAB Coder to generate C and C++ code for your trained network, which enables you to simulate a trained network on PC hardware and then deploy the network to embedded systems. You can use MATLAB Compiler and MATLAB Compiler SDK™ to deploy trained networks as C/C++ shared libraries, Microsoft^®. NET assemblies, Java^® classes, and Python^® packages from MATLAB programs. You can also train a network model in the deployed application or a component.

Simulink Support

Deep Learning Toolbox provides a set of blocks for building shallow neural networks in Simulink. All blocks are compatible with Simulink Coder™. These blocks are divided into four libraries:

• Transfer function blocks, which take a net input vector and generate a corresponding output vector
• Net input function blocks, which take any number of weighted input vectors, weight-layer output vectors, and bias vectors, and return a net input vector
• Weight function blocks, which apply a neuron's weight vector to an input vector (or a layer output vector) to get a weighted input value for a neuron
• Data preprocessing blocks, which map input and output data into the ranges best suited for the neural network to handle directly

Alternatively, you can create and train your networks in the MATLAB environment and automatically generate network simulation blocks for use with Simulink. This approach also enables you to view your networks graphically.