File Exchange

image thumbnail

Deep Learning Toolbox Converter for ONNX Model Format

by MathWorks Deep Learning Toolbox Team
Import and export ONNX™ models within MATLAB for interoperability with other deep learning frameworks
2.1
21 Ratings

73 Downloads

Updated 26 Apr 2021

Import and export ONNX™ (Open Neural Network Exchange) models within MATLAB for interoperability with other deep learning frameworks. ONNX enables models to be trained in one framework and transferred to another for inference.

Opening the onnxconverter.mlpkginstall file from your operating system or from within MATLAB will initiate the installation process for the release you have.
This mlpkginstall file is functional for R2018a and beyond.

Usage example:
%% Export to ONNX model format
net = squeezenet; % Pretrained Model to be exported
filename = 'squeezenet.onnx';
exportONNXNetwork(net,filename);

%% Import the network that was exported
net2 = importONNXNetwork('squeezenet.onnx', 'OutputLayerType', 'classification');

% Compare the predictions of the two networks on a random input image
img = rand(net.Layers(1).InputSize);
y = predict(net, img);
y2 = predict(net2,img);

max(abs(y-y2))

To import an ONNX network in MATLAB, please refer:
https://www.mathworks.com/help/deeplearning/ref/importonnxnetwork.html

To export an ONNX network from MATLAB, please refer:
https://www.mathworks.com/help/nnet/ref/exportonnxnetwork.html

Comments and Ratings (78)

cui

@David please share your exported .onnx file ,Otherwise we can't help you debug

David

That all SEEMS fine, but now show it working in ONNXRuntime. I have been unable to, or find anyone else that can, get remotely the same answers between Matlab and ONNXRuntime. I'm not sure if it's my python code or the exported .onnx file but so far I haven't seen a single example of it working in e.g. python with ONNXRuntime. The only example I saw from Mathworks was using a random array that gave AN answer here https://stackoverflow.com/questions/62372645/run-inference-using-onnx-model-in-python-input-incompatibility-problem
Has anyone else been able to export their trained model from Matlab to ONNX and use it in another environment?

Raar A

Ting Su

@Jin, You can use "importONNXLayers" to importer a layerGraph, add output layers and use "assembleNetwork" to convert a layerGraph to a DAGNetwork.

Jon Cherrie

@David -- If you are using Windows, then you might be hitting this bug: https://www.mathworks.com/support/bugreports/2465087 . There is a workaround listed in that bug report.

David

Worked in 2020b, but crashes in 2021a.

jin louis

OK，now,my matlab version 2018b,I found a problem:
错误使用 nnet.internal.cnn.onnx.translateONNX (line 35)
Models with multiple outputs are not supported yet.

出错 nnet.internal.cnn.onnx.importONNXNetwork (line 8)
LayersOrGraph = nnet.internal.cnn.onnx.translateONNX(modelProto, OutputLayerType,
UserImageInputSize, true);

出错 importONNXNetwork (line 53)
Network = nnet.internal.cnn.onnx.importONNXNetwork(ModelFile, varargin{:});

出错 test_network_01 (line 4)
net = importONNXNetwork('best.onnx','OutputLayerType','classification','Classes',classes)
How to solve it？

cui

Dear Ting Su:
After copying onnxpb.dll, it can work now, thank you very much!

Hiroyuki Hishida

Dear Ting Su,
I also met this issue on R2021a ( 9.10.0.1602886 (R2021a) ) and tried following and it worked.
copyfile C:\ProgramData\MATLAB\SupportPackages\R2021a\bin\win64\onnxpb.dll C:\ProgramData\MATLAB\SupportPackages\R2021a\toolbox\nnet\supportpackages\onnx\+nnet\+internal\+cnn\+onnx\

Ting Su

Cui,
Our engineer found the cause and workaround. Can you try the following command?
mv C:\ProgramData\MATLAB\SupportPackages\R2021a\bin\win64\onnxpb.dll C:\ProgramData\MATLAB\SupportPackages\R2021a\toolbox\nnet\supportpackages\onnx\+nnet\+internal\+cnn\+onnx\

cui

Dear Ting Su:
I'm using matlab2021a,Converter for ONNX Model Format(version21.1.0)

cui

Similar questions have not been answered correctly：https://www.mathworks.com/matlabcentral/answers/494277-onnx-export-yields-error

Ting Su

Hi Cui,
Can you tell us which release of MATLAB you are using?

cui

2021.3.15 update: Run bug report
No matter importing or exporting common onnx models(resnet50,resnet18,etc), errors occurred
---------------------------------import yolov5.onnx---------------------------------------
Error using nnet.internal.cnn.onnx.onnxmex
Invalid MEX-file 'C:\ProgramData\MATLAB\SupportPackages\R2021a\toolbox\nnet\supportpackages\onnx\+nnet\+internal\+cnn\+onnx\onnxmex.mexw64': 找不到指定的模块。
Error in nnet.internal.cnn.onnx.ModelProto (line 32)
ModelPtr = onnxmex(int32(FuncName.EdeserializeFromFile), filename);
Error in nnet.internal.cnn.onnx.importONNXFunction (line 8)
modelProto = nnet.internal.cnn.onnx.ModelProto(inputFilename);
Error in importONNXFunction (line 39)
params = nnet.internal.cnn.onnx.importONNXFunction(modelfile, outputFunctionName);
Error in onnxtest (line 3)
params = importONNXFunction(onnxpath,'yolov5Fcn');
---------------------------------export resnet50.onnx---------------------------------------
Error using nnet.internal.cnn.onnx.onnxmex
Invalid MEX-file 'C:\ProgramData\MATLAB\SupportPackages\R2021a\toolbox\nnet\supportpackages\onnx\+nnet\+internal\+cnn\+onnx\onnxmex.mexw64': 找不到指定的模块。
Error in nnet.internal.cnn.onnx.ModelProto/writeToFile (line 52)
ModelPtr = onnxmex(int32(FuncName.EnewModelProto));
Error in nnet.internal.cnn.onnx.exportONNXNetwork (line 38)
writeToFile(modelProto, Filename);
Error in exportONNXNetwork (line 38)
nnet.internal.cnn.onnx.exportONNXNetwork(Network, filename, varargin{:});

cui

2021.3.3 update: can't import resnet3D model???
https://www.mathworks.com/matlabcentral/answers/761086-importonnxlayers-importonnxnetwork-and-importonnxfunction-can-t-import-my-onnx-model

cui

can't import yolov3.onnx ???
https://www.mathworks.com/matlabcentral/answers/640365-importonnxfunction-cannot-import-yolov3-onnx

cui

The "importONNXFunction" function is introduced in R2020b, which is very good, but the imported function is poorly readable and not easy to modify manually; in addition, does mathworks plan to introduce the "exportONNXFunction" function? I personally suggest that mathworks should strengthen the flexibility of the custom layer, and the readability will be greatly enhanced!

cui

when does support Opset version 10 calibrated quantized-onnx to import? such as quantized resnet50?
链接：https://pan.baidu.com/s/1oMcm2w4r5bUFU-RAqV8AUg
提取码：afba

cui

Updated 14 Oct 2020 ? release notes?

Markus

Why does this feature not support Shallow Neural Network?

How can I use my neural network e.g. as a model in the Azure? Are there any tips?

Markus

r nawaz

How to download Deep Learning Toolbox Converter for ONNX Model Format and install it offline (not connected to the Internet)

Jean-Nicola Blanchet

The documentation says tanhLayer is supported, but when I try exporting it, I get instead:
Placeholder2 Placeholder operator:com.mathworks

What I am doing wrong?

Jan Drgona

Hi,

Does this code work for other than CNN architectures?

Thanks,
Jan

cui

For onnx, It should be onnx fine-grained to Operators, not rough "layer",For example, I customized a network layer yolov3Layer, how to export to Onnx?

Jakob Nikolas Kather

currently does not support shufflenet models

Saad Al-Baddi

Dear MathWorks Deep Learning Toolbox Team:
Hi cu

Do you have a new version which can support 'nnet.cnn.layer.RegionProposalLayer', nnet.cnn.layer.RPNClassificationLayer and nnet.cnn.layer.RPNSoftmaxLayer?

hamed bastami

Dear MathWorks Deep Learning Toolbox Team:
Hi, I tried to use exportONNXNetwork, I ran this part of code, but i saw this error, could you help me pls?!
Usage example:
%% Export to ONNX model format
net = squeezenet; % Pretrained Model to be exported
filename = 'squeezenet.onnx';
exportONNXNetwork(net,filename);
-------------------------------------
this error appear:
Error using onnxmex
Opening file 'squeezenet.onnx' failed.

Error in nnet.internal.cnn.onnx.ModelProto/writeToFile (line 55)
onnxmex(int32(FuncName.EserializeToFile), ModelPtr, filename);

Error in nnet.internal.cnn.onnx.exportONNXNetwork (line 37)
writeToFile(modelProto, Filename);

Error in exportONNXNetwork (line 40)
nnet.internal.cnn.onnx.exportONNXNetwork(Network, filename, varargin{:});

cui

Dear MathWorks DeepLearning Toolbox Team:
When I used another simple 3d cnn network import test, there was an error with the error ???
my 3d network is here: https://drive.google.com/open?id=1vrX44WV1yWIsoNdbW5sTBQHurdMhf0c_
Error using assembleNetwork (line 47)
Invalid network.
Error in nnet.internal.cnn.onnx.importONNXNetwork (line 11)
Network = assembleNetwork(LayersOrGraph);
Error in importONNXNetwork (line 52)
Network = nnet.internal.cnn.onnx.importONNXNetwork(modelfile, varargin{:});
Caused by:
Layer 'node_22': Layer validation failed. Error using 'predict' in Layer nnet.onnx.layer.FlattenLayer. The function threw an error and could not be
executed.
Error using permute
ORDER must have at least N elements for an N-D array.
Layer 'node_22': Layer validation failed. Error using 'predict' in Layer nnet.onnx.layer.FlattenLayer. The function threw an error and could not be
executed.
Error using permute
ORDER must have at least N elements for an N-D array.
Layer 'node_22': Input size mismatch. Size of input to this layer is different from the expected input size.
Inputs to this layer:
from layer 'node_21' (1×4×4×512 output)

cui

hi, @Kristina Mikolaichuk
I am sorry to tell you that according to warning, the current version does not support certain layers，'nnet.cnn.layer.RegionProposalLayer',...etc，Only the weight of the support layer is exported.

Kristina Mikolaichuk

Hello! Could you help me? When I try to export Faster RCNN Network, there are some warnings:

Warning: ONNX does not support layer 'nnet.cnn.layer.RegionProposalLayer'. Exporting to ONNX operator
'com.MathWorks.Placeholder'.
Warning: ONNX does not support layer 'nnet.cnn.layer.RPNSoftmaxLayer'. Exporting to ONNX operator
'com.MathWorks.Placeholder'.
Warning: ONNX does not support layer 'nnet.cnn.layer.RPNClassificationLayer'. Exporting to ONNX
operator 'com.MathWorks.Placeholder'.

Will these layers be supported in future updates?

cui

The version of November 15, 2019 still cannot import and export yolov2 network！

cui

This version now supports importing 3d convolutional neural networks, but the "activations" function will fail when using the imported 3d convolutional network?
my simple 3d CNN is here:https://drive.google.com/file/d/15i0IEiNWAqqexairNFv1eITtqwMA5rYi/view?usp=sharing

Mathias Held

Still not working with biLSTM Layers with regression outputlayer

liangcheng jin

Hi, When I Import the layers from a Keras network , an error occur:
'Importing 'LSTM' layers in Keras models built with the functional API is not yet supported'
So,what should i do to overcome this problem?

cui

Dear MathWorks Deep Learning Toolbox Team:
I hope that feature versions will support ONNX operators more abundantly, not just the current 28 operators.
希望以后版本更丰富的支持ONNX operators，不仅仅只是目前的28种operators.
So far the "importCaffeNetwork" function has performed very poorly.
到目前为止“importCaffeNetwork”这个函数表现的很差！
Come on！
加油！

Andreas Herzog

Hi @cui

we pretty much followed the examples in

https://github.com/microsoft/onnxruntime/blob/master/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests.Capi/CXX_Api_Sample.cpp

and the network seem to work. I have not figured out how to reset the internal state of the LSTM layers for a new sequence. We just reload the model.

Best wishes

Andreas

cui

Dear Ting Su,
I can import and export the mobilenetv2 model that comes with matlab very freely and conveniently, but when I import mobilenetv2.onnx saved in the pytorch-onnx framework, the last layer of averagePooling can't be imported correctly. Why?
https://github.com/tonylins/pytorch-mobilenet-v2 ,onnx liabray can import and export this responsibily,matlab can't...

警告: Unable to import some ONNX operators, because they are not supported yet. They have been replaced by placeholder layers. To find
these layers, call the function findPlaceholderLayers on the returned object.
> In nnet.internal.cnn.onnx.importONNXLayers (line 13)
In importONNXLayers (line 48)

cui

Hi @Andreas Herzog:
Can I ask how you used the LSTM onnx model in a C++ interface function? Is also a sequence feature with matlab, there is such an example code can refer to it, thank you!

Andreas Herzog

Dear Matlab Team,

exporting and load the LSTM model now works fine, also scoring works using the C++ interface.

One minor thing I noticed, the output tensor ist not called like the laster layer in the network but as combination of layer name and last computer graph operation.

So my last layer is named "fc_2" (a standard name from deep learning toolbox) but the outputtensor has to be retrieved in the C++ interface using "fc_2_Add" which is also displayed when you load the onnx file with Netron App.

Is this naming necessary? We save in our model description a serialized onnx file and the name of a certain layer as output tensor to control which compute graph node acts as output. Since for certain types this not necessarly needs to be the laster layer (coding with autoencoder for example).

So could this be set back to the previouse behaviour or can we read from the layer structure, what the correct tensor name should look like?

Best wishes and thanks a lot for the effort.

Andreas

Andreas Herzog

Dear Matlab Team,

the new version (1st Aug) seem to resolve our problems with the LSTM export.

Test with the given example on github with python onnxruntime 0.5.

Many thanks!

Andreas

Andreas Herzog

Dear Ting Su,

any word on the new version that writes LSTM compatible to ONNX runtime?

Sorry for being a pain, but we need that piece of functionality to deliver a model for our customer.

Best wishes

Andreas

saikishore reddipalli

I was able to point the installation location to a folder with enough space to get this done. matlabshared.supportpkg.setSupportPackageRoot()

Thanks.

saikishore reddipalli

I am hitting an issue with the installation for onnx. Not even able to download the file. I am using RHEL 7.5. Any idea on the issue ?

Wei Zhuang Lim

Hi,
i am trying to export a model to use it in tensorflow. It is the basically the same as this
https://de.mathworks.com/help/deeplearning/examples/cocktail-party-source-separation-using-deep-learning-networks.html
i get the warning: " Warning: ONNX does not support layer 'BiasedSigmoidLayer'. Exporting to ONNX operator 'com.MathWorks.Placeholder'. " because one of the layer is a custom sigmoid layer.
i failed to import into tensorflow getting the error
ValidationError: No Schema registered for Placeholder with domain_version of 1
==> Context: Bad node spec: input: "fc_1" output: "layer_1" name: "layer_1" op_type: "Placeholder" doc_string: "Placeholder operator" domain: "com.mathworks"
Is there anyway i can solve this?
my Onnx Modell: https://drive.google.com/open?id=1c5ItcPoYU2xkmOZNiUgrIetLsixEewYK

Thanks in advance

Andreas Herzog

Dear Ting Su,

excellent! :D

Best wishes

Andreas

Ting Su

Hi Andreas,
The new version will be released soon.

Kevin Chng

Does it work with yoloV2?

Andreas Herzog

Dear Ting Su,

any word on a new version that can resolve the issue with the LSTM (see github ticket). We would like to deploy some models into an application with the onnxruntime.

Best wishes

Andreas

cui

Dear Ting Su,
The onnx model exported by exportONNXNetwork() is not the same as the result of running in opencv and Matlab? I posted my issue also here:
https://ww2.mathworks.cn/matlabcentral/answers/464550-the-onnx-model-exported-by-exportonnxnetwork-is-not-the-same-as-the-result-of-running-in-opencv-an

cui

Gabriel Mittag

Hi Ting Su,
I noticed there was a recent update of the converter but LSTMs still don't seem to work properly. I posted my issue also here:
https://de.mathworks.com/matlabcentral/answers/457176-onnx-export-yields-error-in-windows-ml?s_tid=prof_contriblnk

cui

Dear Ting Su,
Does the current onnx version support the export of target detection networks, such as the Yolov2 network（export to yolov2.onnx）?

Andreas Herzog

Dear Ting Su,

yes, thats the issue I opened on Github.

https://github.com/microsoft/onnxruntime/issues/1016

Best wishes

Andreas

Ting Su

Hi Andreas,
We noticed that some LSTM models exported by MATLAB ONNX Converter don't work well with ONNX Runtime, although they could be loaded into other frameworks, as ONNX Runtime strictly follows ONNX spec for the shape requirement. A new release of MATLAB ONNX converter will be released soon and it will work with ONNX Runtime better.

Ting Su

Hi Andreas,
Thanks for the question. Is this the same issue reported in the following link?
https://github.com/microsoft/onnxruntime/issues/1016
We are looking into this and will get back to you soon.

Andreas Herzog

Dear Matlab Team,

we are exporting an lstm Model (basically build as descripted in the sequene-to-sequence regression example with the turbofan engine example data.

We get an error message when importing it in the onnxruntime (build from source 0.4.0 Release):

Load model from temp.onx failed:Node:fc_2 Output:fc_2 [ShapeInferenceError] Mismatch between number of source and target dimensions. Source=2 Target=3

We can load the onnx File in Netron just fine and have an fc_2 output with somewhat odd <1x1x1> dimension. Could there be a confusion in expected output dimensions?

Could we send the onnx-File / Matlab nnet to you for some help.

Would be much appreciated.

Alessandro Gherardi

Exporting models from matlab to other runtime engines doesn't work apart from trivial examples. I've seen strange shape flipping on output ONNX network layers which causes failures when importing to python frameworks or c#.

michael scheinfeild

when I import the model to c++ I don't have same result as output layer in matlab can you supply example in c++ opencv or tensorflow which get layer out to be same as matlab
conv layer for example

Hong Wang

thanks to Jihang Wang, with you help I setup this tool.

Tommaso Martire

Gabriel Mittag

Hi Jihang, thanks for sharing this information, unfortunately it didn't resolve the problem in my case.

Jihang Wang

Hi everyone, I found the reason why it doesn't work under the help of MathWorks Technical support team. I just want to share my experience here. Basically there is a function on my path which is shadowing one of the built-in MATLAB functions. I reset my MATLAB path using the code below:
>> restoredefaultpath
>> rehash toolboxcache
>> savepath % note: this command will overwrite my current path preferences.

After that, I downloaded and reinstalled the converter app from this page and rerunning the export code. Problem solved :) Hope this helps.

Gabriel Mittag

Hi Andreas, I just used a custom CNN and checked it with WinMLRunner, I didn't try any pretrained models though.

Andreas Wirth

Hi Gabriel
Could you tell me which CNN did you use?
As mentioned before i tryed the basic googlenet and i couldn't use it with Microsoft ML.
It would be very helpful if i could use the onnx file exchange.
Thanks in advance

Gabriel Mittag

Hi Ting, thanks a lot for the Opset update. However, now I obtain the same error as Andreas for LSTM networks: "First input does not have rank 2". If I have more than one LSTM-layer in the network the error messages somehow changes to: "First input tensor must have rank 3", CNNs seem to work though.

Ting Su

Hi Andreas and Jihang, Can you reach our technical support and send model to us?

Andreas Wirth

Hi Ting, I ran into the same issue with C#. I can export the Network in different versions. If I try to load the Model into windows.ml I get an "ShapeInferenceError" the First Input does not have rank 2. With Opset v6 it is possible to load the File but it can't be used. I tested googlenet and tried to compare the onnx models with a program called "Netron". The difference I found was that the first layer “Sub” changed from [3x244x244] to [1x3x244x244] but I’m not sure if this is the Problem. A second thing is that with onnx v6 Visual Studio can generate a model class automatically but not with v7 or higher. It seems that it is not recognized as an onnx model. Can you give an advice how to use Matlab trained model's in C#?

Jihang Wang

Hi Ting, I have the same issue when loading the ONNX model in C#. I tried to save the model to different Opset versions but none of them works. Please advise.

Ting Su

Hi Gabriel,
We recently added support for ONNX Opset 7, 8 and 9. One can specify which Opset to use via an optional input argument 'OpsetVersion' during the export. You should be able to download it if you have a R2018b MATLAB.

Ting Su

Hi Kennth,
We saw a similar issue and the fix will be released soon. It will be great if you could send us your MATLAB model to allow us to test it.

Gabriel Mittag

It would be great if the export could be updated to version 7 or 8 to allow the use with windows ml.

Kenneth Post

exportONNXNetwork does not work properly using CNTK and Python. The conversion produces a ValueError: Gemm: Invalid shape, input A and B are expected to be rank=2 matrices.

Hui Yin Lee

Hi, Is the code or toolbox available for Faster R-CNN model to be exported? As i get the error mentioning the model is not DAGnetwork. Hopefully can get some feedback or help here

Ryan Mukherjee

Do you guys know when support for the constant operator will get added?

Error using importONNXNetwork (line 39)
Node 'node_20': Constant operator is not supported yet.

umit kacar

I worked this code:) It is very good. Thank you.

Ting Su

Hi Trihn,
We would like to hear more details on the problem of importONNXNetwork(). Have you installed an old version of this converter before?

Trinh Pham

The function importONNXNetwork() doesn't work when I use example above!

MATLAB Release Compatibility
Created with R2018a
Compatible with R2018a to R2021a
Platform Compatibility
Windows macOS Linux

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!