Main Content

Classify Static Image Using Deep Learning on Raspberry Pi

This example shows you how to generate and deploy code for an image classification algorithm using MATLAB® Support Package for Raspberry Pi® Hardware. The algorithm uses the ResNet-50 neural network to identify the image that is passed as an input using the command line of Raspberry Pi.

When you generate code for prediction the Raspberry Pi support package builds the executable on the hardware. Using ResNet-50, the executable classifies images into 1000 object categories, such as keyboard, coffee mug, pencil, and many animals. The executable predicts the object and outputs a label for it along with the associated probability of object category.

Note: You cannot generate and deploy deep learning code on Raspberry Pi hardware using macOS.

Required Hardware

  • Raspberry Pi hardware (Model 4 recommended)

  • USB cable

  • Ethernet cable

  • A monitor connected to the Raspberry Pi hardware and a HDMI cable (optional)

Prerequisite

Configure the Raspberry Pi network using the Hardware Setup screen. During the hardware setup process, ensure that you download the MathWorks® Raspbian image for deep learning. Instead of the MathWorks Raspbian image, if you choose to customize an existing image on your hardware, ensure that you select the option to install the ARM Compute Library.

Step 1: Connect the Raspberry Pi Hardware

Tip: Before you start this example, we recommend you to complete the Getting Started with MATLAB Support Package for Raspberry Pi Hardware and Send Inputs to MATLAB Function from Command Line of Raspberry Pi examples.

Connect the micro end of the USB cable to the Raspberry Pi and the regular end of the USB cable to the computer. Wait until the PWR LED on the hardware starts blinking.

In the MATLAB® Command Window, create a connection to the Raspberry Pi hardware.

r = raspi
r = 
  raspi with properties:

         DeviceAddress: '172.31.164.186'              
                  Port: 18734                         
             BoardName: 'Raspberry Pi Model B+'       
         AvailableLEDs: {'led0'}                      
  AvailableDigitalPins: [4,5,6,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27]
  AvailableSPIChannels: {'CE0','CE1'}                 
     AvailableI2CBuses: {'i2c-1'}                     
      AvailableWebcams: {}                            
           I2CBusSpeed: 100000                        

  Supported peripherals

Step 2: Open the Object Classification Algorithm

In this example, the raspi_fileRead_resnet function loads the ResNet-50 deep learning neural network to classify the input image. The function resizes the image using OpenCV to fit the size expected by the network. It then classifies the image into object categories and outputs a label for object predictions along with the probabilities for each of the object categories.

For more information on the function, enter this command in the MATLAB Command Window.

type raspi_fileRead_resnet
function raspi_fileRead_resnet(fileName)
%#codegen
% Copyright 2020 The MathWorks, Inc.

assert(all(size(fileName) <= 50));
assert(isa(fileName,'char'));

%Initialize DNN and the input size
net        = coder.loadDeepLearningNetwork('resnet50');
inputSize  = [224, 224,3]; %net.Layers(1).InputSize;

%Read the image
imgSizeAdjusted = readResizedImg(fileName,inputSize);

%Classify the input image
[label,score] = net.classify(imgSizeAdjusted);

labelStr  = cellstr(label);
maxScore  = max(score);
outputStr = sprintf('Label : %s \nScore : %f',labelStr{:},maxScore);

%Print label and score to stdout
fprintf('%s\n',outputStr);
end

You can modify the function by using the edit command.

edit raspi_fileRead_resnet

Step 3: Generate C++ Code for Image Classification on Raspberry Pi

Create a hardware configuration object by using the targetHardware function in the MATLAB Command Window.

board = targetHardware('Raspberry Pi');

Verify the DeviceAddress, Username, and Password properties listed in the output. If required, change the value of the properties by using the dot notation syntax. For example, to change the device address to 172.18.186.123, enter,

board.DeviceAddress = '172.18.186.123'

Set the CoderConfig.GenCodeOnly property of the board object to true. This enables the support package to generate and run the image classification algorithm code only on the host computer.

board.CoderConfig.TargetLang = 'C++';
board.CoderConfig.GenCodeOnly = true;

Create an arm-compute deep learning configuration object. Specify the architecture of Raspberry Pi.

dlcfg = coder.DeepLearningConfig('arm-compute');
dlcfg.ArmArchitecture = 'armv7';

Execute this command to display the version of ARM Compute Library on Raspberry Pi.

r.system('strings $ARM_COMPUTELIB/lib/libarm_compute.so | grep arm_compute_versio | cut -d\  -f 1')
ans = 
    'arm_compute_version=v20.02.1
     '

Copy the version number and assign it to the ArmComputeVersion property of the deep learning configuration object.

dlcfg.ArmComputeVersion = '20.02.1'
dlcfg = 
  ARMNEONConfig with properties:

        ArmComputeVersion: '20.02.1'
          ArmArchitecture: 'armv7'
                 DataType: 'fp32'
    CalibrationResultFile: ''
                TargetLib: 'arm-compute'

Set the DeepLearningConfig property of the code generation configuration object to the deep learning configuration object.

board.CoderConfig.DeepLearningConfig = dlcfg
board = 
targetHardware with properties:

               Name: 'Raspberry Pi'
      DeviceAddress: '172.31.164.186'
           Username: 'pi'
           Password: '*********'
           BuildDir: '/home/pi'
    EnableRunOnBoot: 0
        BuildAction: 'Build, load, and run'
        CoderConfig: [1×1 coder.CodeConfig]

Deploy the function on the hardware by using the deploy function. The deploy function initiates code generation of the raspi_fileRead_resnet function. Once code generation is complete, MATLAB generates a code generation report. Use this report to debug the function for any errors and warnings in the generated code.

After successfully generating the code, update the main.cpp file to accept command line arguments.

deploy(board,'raspi_fileRead_resnet')
Code generation successful: View report

Step 4: Edit C++ Main File to Accept Input from Command Line of Raspberry Pi

This section explains how to modify the main.cpp file to provide inputs using the command line of Raspberry Pi. A premodified main.cpp file is available for you to use. You can choose to use the premodified file or to update the file manually using the steps in this section.

To use the premodified main.cpp file, enter this command in the MATLAB Command Window.

copyfile('main.cpp',[pwd,'\codegen\exe\raspi_fileRead_resnet'])
copyfile([pwd,'\codegen\exe\raspi_fileRead_resnet\examples\main.h'],[pwd,'\codegen\exe\raspi_fileRead_resnet'])

To update the file manually, copy the generated main.cpp and main.h files to one level higher than the current level in the file hierarchy. Open the main.cpp file from the GENERATED CODE section of the code generation report and update the file by adding code at the points labeled in this code.

Step 5: Deploy the Function as a Standalone Executable on the Raspberry Pi

To generate and deploy the code on the hardware, set the CoderConfig.GenCodeOnly property of the board object to false and then use the deploy function. Since the deploy function overwrites the main.cpp file that you edited in the previous section, disable the creation of the file before deployment.

board.CoderConfig.CustomSource = ['codegen',filesep,'exe',filesep,'raspi_fileRead_resnet',filesep,'main.cpp'];
board.CoderConfig.GenerateExampleMain = 'DoNotGenerate';
board.CoderConfig.GenCodeOnly = false
board = 
targetHardware with properties:

               Name: 'Raspberry Pi'
      DeviceAddress: '172.31.164.186'
           Username: 'pi'
           Password: '*********'
           BuildDir: '/home/pi'
    EnableRunOnBoot: 0
        BuildAction: 'Build, load, and run'
        CoderConfig: [1×1 coder.CodeConfig]

deploy(board,'raspi_fileRead_resnet')
 Deploying code. This may take a few minutes. 
Code generation successful: View report

The deploy function initiates code generation of the raspi_fileRead_resnet function.

After successfully generating the code, the support package loads the object classification algorithm as a standalone executable on the hardware. The location of the executable is displayed in the MATLAB Command Window. Note the location for later use.

Step 6: Copy Image File to the Raspberry Pi

  1. After the code generation report has been successfully generated, click View report on the MATLAB Command Window.

  2. On the MATLAB Coder Report Viewer window, on the Build Logs tab, copy the location of the generated elf.codegen_report_elf_location.png

  3. Copy the image file to the hardware for prediction, using putFile.

In this example, coffeemug.png is the image input to the executable and the location of the executable is same as the one you noted in the previous step.

coffeemug.png

4. You can provide the input image of your choice and copy it to the location of the generated elf using the following syntax:

r.putfile('<input image>','<target location of the generated elf copied from the code generation report>')

In this example, coffeemug.png is the image input to the executable and the location of the executable is /home/pi/MATLAB_ws_R2021a/C/Users/Documents/MATLAB/Examples/raspberrypiio-ex31623069.

r.putFile('coffeemug.png','/home/pi/MATLAB_ws/R2021a/C/Users/Documents/MATLAB/Examples/raspberrypiio-ex31623069')

Step 7: Run the Executable Program to Get Prediction Scores

To open an SSH terminal with Raspberry Pi, use this command in the MATLAB Command Window.

openShell(r)

In the terminal, log in by entering the user name and password for the Raspberry Pi.

Change the current directory to the directory where the executable was saved.

cd /home/pi/MATLAB_ws/R2021a/C/Users/Documents/MATLAB/Examples/raspberrypiio-ex31623069

Run the executable on the Raspberry Pi by providing the coffee.png image as an input to the executable.

sudo ./raspi_fileRead_resnet.elf coffeemug.png

The executable starts classifying the object in the image file, then it predicts the object and displays its label and the probability associated with the object category.

raspberrypi_ssh_resnet50_output.png

See Also