Main Content

Deep Learning Custom Training Loops

Customize deep learning training loops and loss functions

If the trainingOptions function does not provide the training options that you need for your task, or custom output layers do not support the loss functions that you need, then you can define a custom training loop. For networks that cannot be created using layer graphs, you can define custom networks as a function. To learn more, see Define Custom Training Loops, Loss Functions, and Networks.

Functions

expand all

dlnetworkDeep learning network for custom training loops
resetStateReset state parameters of neural network
plotPlot neural network architecture
addInputLayerAdd input layer to network
addLayersAdd layers to layer graph or network
removeLayersRemove layers from layer graph or network
connectLayersConnect layers in layer graph or network
disconnectLayersDisconnect layers in layer graph or network
replaceLayerReplace layer in layer graph or network
summaryPrint network summary
initializeInitialize learnable and state parameters of a dlnetwork
networkDataLayoutDeep learning network data layout for learnable parameter initialization
forwardCompute deep learning network output for training
predictCompute deep learning network output for inference
adamupdateUpdate parameters using adaptive moment estimation (Adam)
rmspropupdate Update parameters using root mean squared propagation (RMSProp)
sgdmupdate Update parameters using stochastic gradient descent with momentum (SGDM)
dlupdate Update parameters using custom function
minibatchqueueCreate mini-batches for deep learning
onehotencodeEncode data labels into one-hot vectors
onehotdecodeDecode probability vectors into class labels
padsequencesPad or truncate sequence data to same length
trainingProgressMonitorMonitor and plot training progress for deep learning custom training loops
dlarrayDeep learning array for custom training loops
dlgradientCompute gradients for custom training loops using automatic differentiation
dlfevalEvaluate deep learning model for custom training loops
dimsDimension labels of dlarray
finddimFind dimensions with specified label
stripdimsRemove dlarray data format
extractdataExtract data from dlarray
isdlarrayCheck if object is dlarray
functionToLayerGraphConvert deep learning model function to a layer graph
dlconvDeep learning convolution
dltranspconvDeep learning transposed convolution
lstmLong short-term memory
gruGated recurrent unit
attentionDot-product attention
embedEmbed discrete data
fullyconnectSum all weighted input data and apply a bias
dlode45Deep learning solution of nonstiff ordinary differential equation (ODE)
reluApply rectified linear unit activation
leakyreluApply leaky rectified linear unit activation
geluApply Gaussian error linear unit (GELU) activation
batchnormNormalize data across all observations for each channel independently
crosschannelnormCross channel square-normalize using local responses
groupnormNormalize data across grouped subsets of channels for each observation independently
instancenormNormalize across each channel for each observation independently
layernormNormalize data across all channels for each observation independently
avgpoolPool data to average values over spatial dimensions
maxpoolPool data to maximum value
maxunpoolUnpool the output of a maximum pooling operation
softmaxApply softmax activation to channel dimension
sigmoidApply sigmoid activation
sigmoidApply sigmoid activation
crossentropyCross-entropy loss for classification tasks
l1lossL1 loss for regression tasks
l2lossL2 loss for regression tasks
huberHuber loss for regression tasks
mseHalf mean squared error
ctcConnectionist temporal classification (CTC) loss for unaligned sequence classification
dlaccelerateAccelerate deep learning function for custom training loops
AcceleratedFunctionAccelerated deep learning function
clearCacheClear accelerated deep learning function trace cache

Topics

Custom Training Loops

Model Functions

Automatic Differentiation

Deep Learning Function Acceleration