Main Content

Image Category Classification by Using Deep Learning

This example shows you how to create, compile, and deploy a dlhdl.Workflow object with alexnet as the network object by using the Deep Learning HDL Toolbox™ Support Package for Intel FPGA and SoC. Use MATLAB® to retrieve the prediction results from the target device. Alexnet is a pretrained convolutional neural network that has been trained on over a million images and can classify images into 1000 object categories (such as keyboard, coffee, mug, pencil,and many animals). You can also use VGG-19 and Darknet-19 as the network objects.

Prerequisites

  • Xilinx ZCU102 SoC development kit

  • Deep Learning HDL Toolbox™ Support Package for Xilinx FPGA and SoC

  • Deep Learning Toolbox™ Model for Alexnet

  • Deep Learning Toolbox™

  • Deep Learning HDL Toolbox™

Load the Pretrained Series Network

To load the pretrained series network alexnet, enter:

snet = alexnet;

To load the pretrained series network vgg19, enter:

% snet = vgg19;

To load the pretrained series network darkent 19, enter:

% snet = darknet19;

To view the layers of the pretrained series network, enter:

analyzeNetwork(snet)
% The saved network contains 25 layers including input, convolution, ReLU, cross channel normalization,
% max pool, fully connected, and the softmax output layers.

Create Target Object

Use the dlhdl.Target class to create a target object with a custom name for your target device and an interface to connect your target device to the host computer. Interface options are JTAG and Ethernet. To use JTAG,Install Xilinx™ Vivado™ Design Suite 2019.2. To set the Xilinx Vivado toolpath, enter:

% hdlsetuptoolpath('ToolName', 'Xilinx Vivado', 'ToolPath', 'C:\Xilinx\Vivado\2019.2\bin\vivado.bat');
hTarget = dlhdl.Target('Xilinx','Interface','Ethernet');

Create WorkFlow Object

Use the dlhdl.Workflow class to create an object. When you create the object, specify the network and the bitstream name. Specify the saved pretrained alexnet neural network as the network. Make sure that the bitstream name matches the data type and the FPGA board that you are targeting. In this example, the target FPGA board is the Xilinx ZCU102 SoC board. The bitstream uses a single data type.

hW = dlhdl.Workflow('Network', snet, 'Bitstream', 'zcu102_single','Target',hTarget);

Compile the Alexnet Series network

To compile the Alexnet series network, run the compile method of the dlhdl.Workflow object. You can optionally specify the maximum number of input frames.

dn = hW.compile('InputFrameNumberLimit',15)
          offset_name          offset_address     allocated_space 
    _______________________    ______________    _________________

    "InputDataOffset"           "0x00000000"     "12.0 MB"        
    "OutputResultOffset"        "0x00c00000"     "4.0 MB"         
    "SystemBufferOffset"        "0x01000000"     "28.0 MB"        
    "InstructionDataOffset"     "0x02c00000"     "4.0 MB"         
    "ConvWeightDataOffset"      "0x03000000"     "16.0 MB"        
    "FCWeightDataOffset"        "0x04000000"     "224.0 MB"       
    "EndOffset"                 "0x12000000"     "Total: 288.0 MB"
dn = struct with fields:
       Operators: [1×1 struct]
    LayerConfigs: [1×1 struct]
      NetConfigs: [1×1 struct]

Program Bitstream onto FPGA and Download Network Weights

To deploy the network on the Intel Arria 10 SoC hardware, run the deploy function of the dlhdl.Workflow object. This function uses the output of the compile function to program the FPGA board by using the programming file. It also downloads the network weights and biases. The deploy function starts programming the FPGA device, displays progress messages, and the time it takes to deploy the network.

hW.deploy
### FPGA bitstream programming has been skipped as the same bitstream is already loaded on the target FPGA.
### Deep learning network programming has been skipped as the same network is already loaded on the target FPGA.

Load Image for Prediction

Load the example image.

imgFile = 'espressomaker.jpg';
inputImg = imresize(imread(imgFile), [227,227]);
imshow(inputImg)

Run Prediction for One Image

Execute the predict method on the dlhdl.Workflow object and then show the label in the MATLAB command window.

[prediction, speed] = hW.predict(single(inputImg),'Profile','on');
### Finished writing input activations.
### Running single input activations.
              Deep Learning Processor Profiler Performance Results

                   LastLayerLatency(cycles)   LastLayerLatency(seconds)       FramesNum      Total Latency     Frames/s
                         -------------             -------------              ---------        ---------       ---------
Network                   33531964                  0.15242                       1           33531979              6.6
    conv_module            8965629                  0.04075 
        conv1              1396567                  0.00635 
        norm1               622836                  0.00283 
        pool1               226593                  0.00103 
        conv2              3409730                  0.01550 
        norm2               378491                  0.00172 
        pool2               233223                  0.00106 
        conv3              1139273                  0.00518 
        conv4               892869                  0.00406 
        conv5               615895                  0.00280 
        pool5                50267                  0.00023 
    fc_module             24566335                  0.11167 
        fc6               15819119                  0.07191 
        fc7                7030644                  0.03196 
        fc8                1716570                  0.00780 
 * The clock frequency of the DL processor is: 220MHz
[val, idx] = max(prediction);
snet.Layers(end).ClassNames{idx}
ans = 
'espresso maker'

Run Prediction for Multiple Images

Load multiple images and retrieve their prediction reults by using the mulltiple frame support feature. For more information, see Multiple Frame Support.

The demoOnImage function loads multiple images and retrieves their prediction results. The annotateresults function displays the image prediction result on top of the images which are assembed into a 3-by-5 array.

imshow(inputImg)

 demoOnImage; 
### Finished writing input activations.
### Running single input activations.
FPGA PREDICTION: envelope 
FPGA PREDICTION: file 
FPGA PREDICTION: folding chair 
FPGA PREDICTION: mixing bowl 
FPGA PREDICTION: toilet seat 
FPGA PREDICTION: dining table 
FPGA PREDICTION: envelope 
FPGA PREDICTION: espresso maker 
FPGA PREDICTION: computer keyboard 
FPGA PREDICTION: monitor 
FPGA PREDICTION: mouse 
FPGA PREDICTION: ballpoint 
FPGA PREDICTION: letter opener 
FPGA PREDICTION: analog clock 
FPGA PREDICTION: ashcan