How to export INT8 quantized weight of deep neural network?
8 views (last 30 days)
Show older comments
I trained neural network using Deep Learning Toolbox, and quantized it.
Below code is what I used to INT8 quantize network model.
% Create a dlquantizer object for quantization
quantObj = dlquantizer(net);
% quantOpts = dlquantizationOptions(target='host');
calibrate(quantObj,imdsTrain);
% valResults = validate(quantObj, imdsValidation, quantOpts);
% valResults.Statistics
% Perform quantization
quantObj = quantize(quantObj);
qDetailsQuantized = quantizationDetails(quantObj)
% Save the quantized network
save('quantizedNet.mat', 'quantObj');
exportONNXNetwork(quantObj,'quantizedNet.onnx')
After quantization, I got quantized network quantObj .
However, I cannot access weight and bias which coverted to INT8 format.
When I display quantized networks' weight and bias using bwloe code,
>> disp(quantObj.Layers(2).Bias(:,:,1))
-6.9011793e-12
It still shows float type value.
Even I tried to export network as ONNX, MATLAB shows below warning,
>> exportONNXNetwork(quantObj,'quantizedNet.onnx')
Warning: Exported weights are not quantized when exporting quantized networks.
How can I access INT8 quantized weight and bias value?
0 Comments
Accepted Answer
Angelo Yeo
on 30 May 2024
You should inspect your qDetailsQuantized which was extracted with quantizationDetails. Would you look up the qDetailsQuantized.QuantizedLearnables?
The following example can be helpful for you.
3 Comments
Angelo Yeo
on 30 May 2024
Yes, exactly. Thanks for the feedback. It's great to know it worked for you.
More Answers (0)
See Also
Categories
Find more on Quantization, Projection, and Pruning in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!