This example shows you how to capture and process images from a Raspberry Pi Camera Module V2 connected to the NVIDIA® Jetson Nano using the GPU Coder™ Support Package for NVIDIA GPUs. The GPU Coder Support Package for NVIDIA GPUs allows you to capture images from the Camera Module V2 and bring them into the MATLAB® environment for processing. In this example you learn how to develop a Sobel edge detection algorithm by using this capability.
Target Board Requirements
NVIDIA Jetson Nano embedded platform.
Raspberry Pi Camera Module V2 connected to the CSI host port of the target.
Ethernet crossover cable to connect the target board and host PC (if you cannot connect the target board to a local network).
NVIDIA CUDA toolkit installed on the board.
V4L2 and SDL (v1.2) libraries on the board.
GStreamer libraries on the board.
Environment variables on the target for the compilers and libraries. For more information, see Install and Setup Prerequisites for NVIDIA Boards (GPU Coder Support Package for NVIDIA GPUs).
Development Host Requirements
GPU Coder for CUDA code generation. For a tutorial, see Get Started with GPU Coder.
NVIDIA CUDA toolkit on the host.
GPU Coder Interface for Deep Learning Libraries support package. To install this support package, use the MATLAB
The following line of code creates a folder in your current working folder on the host and copies all the relevant files into this folder. If you cannot generate files in this folder, before running this command, change your current working folder.
The GPU Coder Support Package for NVIDIA GPUs uses an SSH connection over TCP/IP to execute commands while building and running the generated CUDA code on the Jetson Nano platforms. Connect the target platform to the same network as the host computer or use an Ethernet crossover cable to connect the board directly to the host computer. For information on how to set up and configure your board, see NVIDIA documentation.
To communicate with the NVIDIA hardware, create a live hardware connection object by using the
jetson (GPU Coder Support Package for NVIDIA GPUs) function. You must know the host name or IP address, user name, and password of the target board to create a live hardware connection object. For example, when connecting to the target board for the first time, create a live object for Jetson hardware by using the command:
hwobj = jetson('jetson-nano-name','ubuntu','ubuntu');
During the hardware live object creation, the support package performs hardware and software checks, IO server installation, and gathers peripheral information on target. This information is displayed in the Command Window.
getCameraList function of the
hwobj object to find the available cameras. If this function outputs an empty table, then try re-connecting the camera and execute the function again.
camlist = getCameraList(hwobj);
To verify that the compilers and libraries necessary for running this example are set up correctly, use the
envCfg = coder.gpuEnvConfig('jetson'); envCfg.BasicCodegen = 1; envCfg.Quiet = 1; envCfg.HardwareObject = hwobj; coder.checkGpuInstall(envCfg);
Create a camera object by using the name from the
camObj = camera(hwobj,"vi-output, imx219 6-0010",[640 480]);
camObj is a handle to a camera object. To display the images captured from the Camera Module V2 in MATLAB, use these commands:
for i = 1:100 img = snapshot(camObj); imagesc(img); drawnow; end
This camera object captures RGB and 3-channel grayscale images.
To create a display object, use the
imageDisplay function. This object is a system object that uses
imshow function to display the images in MATLAB.
dispObj = imageDisplay(hwobj); img = snapshot(camObj); image(dispObj,img);
The Sobel edge detection algorithm is a 2-D spatial gradient operation on a grayscale image. This operation emphasizes the high spatial frequency regions pf the image that corresponds to edges.
Find horizontal gradient(h) and vertical gradient (v) of the input image with respective Sobel kernels. These two Sobel kernels are orthogonal to each other. Before processing live image data from the camera, test the algorithm on a sample image.
kern = [1 2 1; 0 0 0; -1 -2 -1]; img = imread('peppers.png'); imagesc(img); h = conv2(img(:,:,2),kern,'same'); v = conv2(img(:,:,2),kern','same');
Calculate Gradient Magnitude
Find the gradient magnitude from the horizontal and vertical gradients (h and v).
e = sqrt(h.*h + v.*v);
Threshold the Edge Image
Threshold the image to find the regions of image that are edges.
edgeImg = uint8((e > 100) * 240); imagesc(edgeImg);
Create a MATLAB entry-point function,
sobelEdgeDetectionAlg.m, out of the MATLAB code developed in the previous sections of this example. View the code in MATLAB editor.
sobelEdgeDetectionAlg() takes image and threshold input for edge detection and returns the results of edge detection algorithm. Call this function on the images captured from inside a loop. You can vary the threshold variable
thresh to get a proper edge image. This way you can use the camera access capability of the support package to tune the algorithm suitable for the specified camera.
for i = 1:200 img = snapshot(camObj); thresh = 100; edgeImage = sobelEdgeDetectionAlg(img, thresh); image(dispObj,edgeImage); end
To deploy this example as a standalone application on the target board, see Deploy and Run Sobel Edge Detection with I/O on NVIDIA Jetson Nano (GPU Coder Support Package for NVIDIA GPUs).
To remove the example files and return to the original folder, call the
cleanup function. cleanup