neuralODELayer
R2026bDescription
A neural ODE layer outputs the solution of an ODE.
Creation
Description
layer = neuralODELayer(net,tspan)Network and
TimeInterval
properties.
layer = neuralODELayer(net,tspan,Name=Value)neuralODELayer(net,tspan,Solver="ode1") creates a neural ODE layer
that uses a solver for nonstiff differential equations using Euler method.
Input Arguments
Neural network characterizing neural ODE function, specified as a dlnetwork object.
If the network has one input, then predict(net,Y) defines the ODE system, where net is the network. If the network has two inputs, then predict(net,T,Y) defines the ODE system, where T is a time step repeated over the batch dimension.
The size and format of the network inputs and outputs must match.
When GradientMode is "adjoint", the network State property must be empty. To use a network with a nonempty State property, set GradientMode to "direct".
This argument sets the Network
property.
Interval of integration, specified as a numeric vector with two or more elements. The elements in the vector must be all increasing or all decreasing.
The solver imposes the initial conditions given by Y0 at the
initial time tspan(1), then integrates the ODE function from
tspan(1) to tspan(end).
If
tspanhas two elements,[t0 tf], then the solver returns the solution evaluated at pointtf.If
tspanhas more than two elements,[t0 t1 ... tf], then the solver returns the solution evaluated at the given points[t1 ... tf]. The solver does not step precisely to each point specified intspan. Instead, the solver uses its own internal steps to compute the solution, then evaluates the solution at the points specified intspan. The solutions produced at the specified points are of the same order of accuracy as the solutions computed at each internal step.Specifying several intermediate points has little effect on the efficiency of computation, but for large systems it can negatively affect memory management.
This argument sets the TimeInterval property.
Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64
Name-Value Arguments
Specify optional pairs of arguments as
Name1=Value1,...,NameN=ValueN, where Name is
the argument name and Value is the corresponding value.
Name-value arguments must appear after other arguments, but the order of the
pairs does not matter.
Example: neuralODELayer(net,tspan,Solver="ode1") creates a neural
ODE layer that uses a solver for nonstiff differential equations using Euler
method.
Since R2025a
Solver for neural ODE operation, specified as one of these values:
"ode45"— Solver for nonstiff differential equations using explicit Runge-Kutta (4,5) formula, the Dormand-Prince pair. The"ode45"solver is well suited for most tasks and can be faster and more accurate than other solvers."ode1"— Solver for nonstiff differential equations using Euler method. The"ode1"solver uses a fixed step size and can be better suited for code generation and Simulink tasks.
If Solver is "ode1", then the RelativeTolerance and AbsoluteTolerance properties have no effect.
If you specify the SolverOptions name-value argument, then
the default value for Solver is the corresponding solver.
Otherwise, the default value is "ode45".
This argument sets the Solver
property.
Since R2025a
Solver options object, specified as a deep.ode.options.ODE45 or a deep.ode.options.ODE1 object.
In most cases, you do not need to create the options object directly. Create the
layer first, then set the solver options of the layer, using dot notation. For
example, to set the relative tolerance to 1e-4, use
layer.SolverOptions.RelativeTolerance = 1e-4,
where layer is an instance of the neural ODE layer.
If you specify a solver options object, then you must not specify the
RelativeTolerance,
AbsoluteTolerance, and
GradientMode argument values.
To see which options the "ode45" and "ode1" solvers support, see deep.ode.options.ODE45 and deep.ode.options.ODE1, respectively.
The default value is the options object that corresponds to the
Solver argument value. The object has the default property
values.
This argument sets the SolverOptions property.
Method to compute gradients with respect to the initial conditions and
parameters when using the dlgradient function, specified as one of these values:
GradientMode | Description | Notes |
|---|---|---|
"direct" | Compute gradients by backpropagating through the operations
undertaken by the numerical solver. This option best suits large
mini-batch sizes or when tspan contains many
values. | The |
"adjoint" | Compute gradients by solving the associated adjoint ODE system. This
option best suits small mini-batch sizes or when
tspan contains a small number of values. | The
Warning All layers in The software traces the ODE function input to determine the computation graph used for automatic differentiation. This tracing process can take some time and can end up recomputing the same trace. By optimizing, caching, and reusing the traces, the software can speed up the gradient computation. For more information on deep learning function acceleration, see Deep Learning Function Acceleration.
|
"adjoint-seminorm" | Compute gradients by solving the associated adjoint ODE system and use a seminorm for the adaptive solver error control [1]. Using a seminorm reduces the number of terms in the step-size error computation, which can speed up training. | The This option is not
supported when Before R2026b: To compute gradients using
adjoint seminorm computations, use the
Warning All layers in For more information on deep learning function acceleration, see Deep Learning Function Acceleration.
|
Tip
To customize the neural ODE solver options, use the Solver
name-value argument and the SolverOptions property. This approach is recommended because it
provides additional control over the neural ODE solver.
The default value is the GradientMode property value
of the SolverOptions argument value.
This argument sets the GradientMode property.
Relative error tolerance, specified as a positive scalar. This tolerance measures the error relative to the magnitude of each solution component. Roughly speaking, it controls the number of correct digits in all solution components, except those smaller than the absolute tolerance AbsoluteTolerance.
At each step, the ODE solver estimates the local error e in the ith component of the solution. To be successful, the step must have acceptable error, as determined by both the relative and absolute error tolerances:
|e(i)| <= max(RelativeTolerance*abs(y(i)),AbsoluteTolerance(i))
If the Solver
argument value is "ode1", then the
RelativeTolerance argument has no effect.
Tip
To customize the neural ODE solver options, use the Solver and SolverOptions properties. These properties are recommended because they provide additional control over the neural ODE solver.
This argument sets the RelativeTolerance property.
Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64
Absolute error tolerance, specified as a positive scalar or vector. This tolerance is a threshold below which the value of the solution becomes unimportant. If the solution |y| is smaller than AbsoluteTolerance, then the solver does not need to obtain any correct digits in |y|. For this reason, the value of AbsoluteTolerance should take into account the scale of the solution components.
If AbsoluteTolerance is a vector, then it must be the same length as the solution. If AbsoluteTolerance is a scalar, then the value applies to all solution components.
At each step, the ODE solver estimates the local error e in the ith component of the solution. To be successful, the step must have acceptable error, as determined by both the relative and absolute error tolerances:
|e(i)| <= max(RelativeTolerance*abs(y(i)),AbsoluteTolerance(i))
If the Solver
argument value is "ode1", then the
AbsoluteTolerance argument value has no effect.
Tip
To customize the neural ODE solver options, use the Solver and SolverOptions properties. These properties are recommended because they provide additional control over the neural ODE solver.
This argument sets the AbsoluteTolerance property.
Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64
Properties
Neural ODE
Neural network characterizing neural ODE function, specified as a dlnetwork object.
If the network has one input, then predict(net,Y) defines the ODE system, where net is the network. If the network has two inputs, then predict(net,T,Y) defines the ODE system, where T is a time step repeated over the batch dimension.
The size and format of the network inputs and outputs must match.
When GradientMode is "adjoint", the network State property must be empty. To use a network with a nonempty State property, set GradientMode to "direct".
Interval of integration, specified as a numeric vector with two or more elements. The elements in the vector must be all increasing or all decreasing.
The solver imposes the initial conditions given by Y0 at the
initial time TimeInterval(1), then integrates the ODE function from
TimeInterval(1) to TimeInterval(end).
If
TimeIntervalhas two elements,[t0 tf], then the solver returns the solution evaluated at pointtf.If
TimeIntervalhas more than two elements,[t0 t1 ... tf], then the solver returns the solution evaluated at the given points[t1 ... tf]. The solver does not step precisely to each point specified inTimeInterval. Instead, the solver uses its own internal steps to compute the solution, then evaluates the solution at the points specified inTimeInterval. The solutions produced at the specified points are of the same order of accuracy as the solutions computed at each internal step.Specifying several intermediate points has little effect on the efficiency of computation, but for large systems it can negatively affect memory management.
Data Types: double
Since R2025a
Solver for neural ODE operation, specified as one of these values:
"ode45"— Solver for nonstiff differential equations using explicit Runge-Kutta (4,5) formula, the Dormand-Prince pair. The"ode45"solver is well suited for most tasks and can be faster and more accurate than other solvers."ode1"— Solver for nonstiff differential equations using Euler method. The"ode1"solver uses a fixed step size and can be better suited for code generation and Simulink tasks.
If Solver is "ode1", then the RelativeTolerance and AbsoluteTolerance properties have no effect.
If you change the Solver property value, then the software
automatically updates the SolverOptions property value to the
corresponding solver options object. The object has the default property
values.
Since R2025a
Solver options object, specified as a deep.ode.options.ODE45 or a deep.ode.options.ODE1 object.
In most cases, you do not need to create the options object directly. Create the
layer first, then set the solver options of the layer, using dot notation. For
example, to set the relative tolerance to 1e-4, use
layer.SolverOptions.RelativeTolerance = 1e-4,
where layer is an instance of the neural ODE layer.
If you specify a solver options object, then you must not specify the
RelativeTolerance,
AbsoluteTolerance, and
GradientMode argument values.
To see which options the "ode45" and "ode1" solvers support, see deep.ode.options.ODE45 and deep.ode.options.ODE1, respectively.
If you change the SolverOptions property value type, then the
software automatically updates the Solver property value to the
corresponding solver.
Method to compute gradients with respect to the initial conditions and parameters
when using the dlgradient
function, specified as one of these values:
GradientMode | Description | Notes |
|---|---|---|
'direct' | Compute gradients by backpropagating through the operations undertaken by the numerical solver. This option can compute
more accurate gradients but requires more memory and can be faster for
models that have components that the | The |
'adjoint' | Compute gradients by solving the associated adjoint ODE system. This option requires less memory for large networks
and batch sizes, but can converge to suboptimal results. This option can
be faster for models that the | The
Warning All layers in the The software traces the ODE function input to determine the computation graph used for automatic differentiation. This tracing process can take some time and can end up recomputing the same trace. By optimizing, caching, and reusing the traces, the software can speed up the gradient computation. For more information on deep learning function acceleration, see Deep Learning Function Acceleration.
|
'adjoint-seminorm' | Compute gradients by solving the associated adjoint ODE system and use a seminorm for the adaptive solver error control [1]. Using a seminorm reduces the number of terms in the step-size error computation, which can speed up training. | The This option is not
supported when To compute gradients using adjoint
seminorm computations, use the
Warning All layers in the For more information on deep learning function acceleration, see Deep Learning Function Acceleration.
|
Tip
To customize the neural ODE solver options, use the Solver
argument and the SolverOptions property. This approach is recommended because it
provides additional control over the neural ODE solver.
The default value is the GradientMode property value
of the SolverOptions property value.
When GradientMode is "adjoint", the network State property must be empty. To use a network with a nonempty State property, set GradientMode to "direct".
Relative error tolerance, specified as a positive scalar. This tolerance measures the error relative to the magnitude of each solution component. Roughly speaking, it controls the number of correct digits in all solution components, except those smaller than the absolute tolerance AbsoluteTolerance.
At each step, the ODE solver estimates the local error e in the ith component of the solution. To be successful, the step must have acceptable error, as determined by both the relative and absolute error tolerances:
|e(i)| <= max(RelativeTolerance*abs(y(i)),AbsoluteTolerance(i))
If Solver is "ode1", then the
RelativeTolerance property has no effect.
Tip
To customize the neural ODE solver options, use the Solver and SolverOptions properties. These properties are recommended because they provide additional control over the neural ODE solver.
Data Types: double
Absolute error tolerance, specified as a positive scalar or vector. This tolerance is a threshold below which the value of the solution becomes unimportant. If the solution |y| is smaller than AbsoluteTolerance, then the solver does not need to obtain any correct digits in |y|. For this reason, the value of AbsoluteTolerance should take into account the scale of the solution components.
If AbsoluteTolerance is a vector, then it must be the same length as the solution. If AbsoluteTolerance is a scalar, then the value applies to all solution components.
At each step, the ODE solver estimates the local error e in the ith component of the solution. To be successful, the step must have acceptable error, as determined by both the relative and absolute error tolerances:
|e(i)| <= max(RelativeTolerance*abs(y(i)),AbsoluteTolerance(i))
If Solver is "ode1", then the
AbsoluteTolerance property has no effect.
Tip
To customize the neural ODE solver options, use the Solver and SolverOptions properties. These properties are recommended because they provide additional control over the neural ODE solver.
Data Types: double
Layer
This property is read-only.
Number of inputs to the layer, represented as 1. This layer has a
single input only.
Data Types: double
This property is read-only.
Input name, represented as {'in'}. This layer has a single input
only.
This property is read-only.
Number of outputs from the layer, represented as 1. This layer has
a single output only.
Data Types: double
This property is read-only.
Output name, represented as {'out'}. This layer has a single output
only.
Examples
Create a neural ODE layer. Specify an ODE network containing a convolution layer followed by a tanh layer. Specify a time interval of [0, 1].
inputSize = [14 14 8];
layersODE = [
imageInputLayer(inputSize)
convolution2dLayer(3,8,Padding="same")
tanhLayer];
netODE = dlnetwork(layersODE);
tspan = [0 1];
layer = neuralODELayer(netODE,tspan)layer =
NeuralODELayer with properties:
Name: ''
TimeInterval: [0 1]
Learnable Parameters
Network: [1×1 dlnetwork]
Solver properties
GradientMode: 'direct'
RelativeTolerance: 1.0000e-03
AbsoluteTolerance: 1.0000e-06
Solver: ode45
Show all properties
Create a neural network containing a neural ODE layer.
layers = [
imageInputLayer([28 28 1])
convolution2dLayer([3 3],8,Padding="same",Stride=2)
reluLayer
neuralODELayer(netODE,tspan)
fullyConnectedLayer(10)
softmaxLayer];
net = dlnetwork(layers)net =
dlnetwork with properties:
Layers: [6×1 nnet.cnn.layer.Layer]
Connections: [5×2 table]
Learnables: [6×3 table]
State: [0×3 table]
InputNames: {'imageinput'}
OutputNames: {'softmax'}
Initialized: 1
View summary with summary.
Since R2025a
Create a neural ODE layer. Specify an ODE network containing a convolution layer followed by a tanh layer. Specify a time interval of [0, 1].
inputSize = [14 14 8];
layersODE = [
imageInputLayer(inputSize)
convolution2dLayer(3,8,Padding="same")
tanhLayer];
netODE = dlnetwork(layersODE);
tspan = [0 1];
layer = neuralODELayer(netODE,tspan)layer =
NeuralODELayer with properties:
Name: ''
TimeInterval: [0 1]
Learnable Parameters
Network: [1×1 dlnetwork]
Solver properties
GradientMode: 'direct'
RelativeTolerance: 1.0000e-03
AbsoluteTolerance: 1.0000e-06
Solver: ode45
Show all properties
Specify an initial step size of 1e-3 and a maximum step size of 1e-2.
layer.SolverOptions.InitialStep = 1e-3; layer.SolverOptions.MaxStep = 1e-2;
View the solver options.
layer.SolverOptions
ans =
ODE45 with properties:
InitialStep: 1.0000e-03
MaxStep: 0.0100
RelativeTolerance: 1.0000e-03
AbsoluteTolerance: 1.0000e-06
GradientMode: 'direct'
Tips
To apply the neural ODE operation in deep learning models defined as functions or in custom layer functions, use
dlode45.
Algorithms
The neural ordinary differential equation (ODE) operation returns the solution of a specified ODE. In particular, given an input, a neural ODE operation outputs the numerical solution of the ODE for the time horizon (t0,t1) and with the initial condition y(t0) = y0, where t and y denote the ODE function inputs and θ is a set of learnable parameters. Typically, the initial condition y0 is either the network input or the output of another deep learning operation.
To apply the operation, NeuralODELayer uses the ode45 function, which is based on an explicit Runge-Kutta (4,5) formula, the
Dormand-Prince pair. It is a single-step solver—in computing
y(tn), it needs only the solution at the
immediately preceding time point, y(tn-1)
[2]
[3].
Most layers in a layer array or layer graph pass data to subsequent layers as formatted
dlarray objects.
The format of a dlarray object is a string of characters in which each
character describes the corresponding dimension of the data. The format consists of one or
more of these characters:
"S"— Spatial"C"— Channel"B"— Batch"T"— Time"U"— Unspecified
For example, you can describe 2-D image data that is represented as a 4-D array, where the
first two dimensions correspond to the spatial dimensions of the images, the third
dimension corresponds to the channels of the images, and the fourth dimension
corresponds to the batch dimension, as having the format "SSCB"
(spatial, spatial, channel, batch).
You can interact with these dlarray objects in automatic differentiation
workflows, such as those for:
developing a custom layer
using a
functionLayerobjectusing the
forwardandpredictfunctions withdlnetworkobjects
This table shows the supported input formats of NeuralODELayer objects and the
corresponding output format. If the software passes the output of the layer to a custom
layer that does not inherit from the nnet.layer.Formattable class, or to
a FunctionLayer object with the Formattable property set
to 0 (false), then the layer receives an unformatted
dlarray object with dimensions ordered according to the formats in this
table. The formats listed here are only a subset of the formats that the layer supports. The
layer might support additional formats, such as formats with additional
"S" (spatial) or "U" (unspecified)
dimensions.
If TimeInterval contains more than two elements, then the layer
outputs data with a "T" (time) dimension.
| Input Format | TimeInterval | Output Format |
|---|---|---|
| [t0 tf] |
|
[t0 t1 ... tf] |
| |
| [t0 tf] |
|
[t0 t1 ... tf] |
| |
| [t0 tf] |
|
[t0 t1 ... tf] |
| |
| [t0 tf] |
|
[t0 t1 ... tf] |
| |
| [t0 tf] |
|
[t0 t1 ... tf] |
| |
| [t0 tf] |
|
[t0 t1 ... tf] |
| |
| [t0 tf] |
|
[t0 t1 ... tf] |
| |
"SB" (spatial, batch) | [t0 tf] | "SB" (spatial, batch) |
[t0 t1 ... tf] | "SBT" (spatial, batch, time) | |
"SSB" (spatial, spatial, batch) | [t0 tf] | "SSB" (spatial, spatial, batch) |
[t0 t1 ... tf] | "SSBT" (spatial, spatial, batch, time) | |
"SSSB" (spatial, spatial, spatial,
batch) | [t0 tf] | "SSSB" (spatial, spatial, spatial, batch) |
[t0 t1 ... tf] | "SSSBT" (spatial, spatial, spatial, batch, time) | |
"SS" (spatial, spatial) | [t0 tf] | "SS" (spatial, spatial) |
[t0 t1 ... tf] | "SST" (spatial, spatial, time) | |
"SSS" (spatial, spatial, spatial) | [t0 tf] | "SSS" (spatial, spatial, spatial) |
[t0 t1 ... tf] | "SSST" (spatial, spatial, spatial, time) |
References
[1] Kidger, Patrick, Ricky T. Q. Chen, and Terry Lyons. “‘Hey, That’s Not an ODE’: Faster ODE Adjoints via Seminorms.” arXiv, May 10, 2021. https://doi.org/10.48550/arXiv.2009.09457.
Version History
Introduced in R2023bSpecify the solver using the Solver property. To further customize
the "ode45" or "ode1" solvers, use the
SolverOptions argument.
See Also
ODE45 | trainnet | trainingOptions | dlnetwork | dlode45
MATLAB Command
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)