Accelerate Pixel-Streaming Designs Using MATLAB Coder

This example shows how to accelerate a pixel-stream video processing algorithm in MATLAB® by using MATLAB Coder™.

You must have a MATLAB Coder license to run this example.

Acceleration with MATLAB Coder enables you to simulate large frame sizes, such as 1080p video, at practical speeds. Use this acceleration workflow after you have debugged the algorithm using a small frame size. Testing a design with a small image is demonstrated in the Pixel-Streaming Design in MATLAB (Vision HDL Toolbox) example.

How MATLAB Coder Works

MATLAB Coder generates C code from MATLAB code. Code generation accelerates simulation by locking-down the sizes and data types of variables. This process removes the overhead of the interpreted language checking for size and data type in every line of code. This example compiles both the test bench file DesignAccelerationHDLTestBench.m and the design file DesignAccelerationHDLDesign.m into a MEX function, and uses the resulting MEX file to speed up the simulation.

The directive (or pragma) %#codegen beneath the function signature indicates that you intend to generate code for the MATLAB algorithm. Adding this directive instructs the MATLAB code analyzer to help you diagnose and fix violations that would result in errors during code generation. The directive %#codegen does not affect interpreted simulation.

Best Practices

Debugging simulations with large frame sizes is impractical in interpreted mode due to long simulation time. However, debugging a MEX simulation is challenging due to lack of debug access into the code.

To avoid these scenarios, a best practice is to develop and verify the algorithm and test bench using a thumbnail frame size. In most cases, the HDL-targeted design can be implemented with no dependence on frame size. Once you are confident that the design and test bench are working correctly, then increase the frame size in the test bench, and use MATLAB Coder to accelerate the simulation. To increase the frame size, test bench only requires minor changes, as you can see by comparing DesignAccelerationHDLTestBench.m with the PixelStreamingDesignHDLTestBench.m in Pixel-Streaming Design in MATLAB (Vision HDL Toolbox).

Test Bench

In the test bench DesignAccelerationHDLTestBench.m, the videoIn object reads each frame from a video source which is converted from RGB to grayscale, and the imresize function interpolates this frame from 240p to 1080p. This 1080p image is passed to the frm2pix object, which converts the full image frame to a stream of pixels and control structures. The function DesignAccelerationHDLDesign is then called to process one pixel (and its associated control structure) at a time. After we process the entire pixel-stream and collect the output stream, the pix2frm object converts the output stream to full-frame video. The DesignAccelerationHDLViewer function displays the output and original images side-by-side.

The workflow above is implemented in the following lines of DesignAccelerationHDLTestBench.m.

for f = 1:numFrm
    frmFull = rgb2gray(readFrame(videoIn));            % Get a new frame
    frmIn = imresize(frmFull,[actLine actPixPerLine]); % Enlarge the frame

     [pixInVec,ctrlInVec] = frm2pix(frmIn);
     for p = 1:numPixPerFrm
         [pixOutVec(p),ctrlOutVec(p)] = DesignAccelerationHDLDesign(pixInVec(p),ctrlInVec(p));
     end
     frmOut = pix2frm(pixOutVec,ctrlOutVec);

     DesignAccelerationHDLViewer(actPixPerLine,actLine,[frmIn uint8(255*frmOut)]);
 end

The data type of frmIn is uint8 while that of frmOut, the edge detection output, is logical. Matrices of different data types cannot be concatenated, so uint8(255*frmOut) maps logical false and true to uint8(0) and uint8(255), respectively.

Both frm2pix and pix2frm are used to convert between full-frame and pixel-stream domains. The inner for-loop performs pixel-stream processing. The rest of the test bench performs full-frame processing (i.e., videoIn, scaler, and viewer inside the DesignAccelerationHDLViewer function).

Before the test bench terminates, frame rate is displayed to illustrate the simulation speed.

Not all functions used in the test bench support C code generation. For those that do not, such as tic, toc, fprintf, use coder.extrinsic to declare them as extrinsic functions. Extrinsic functions are excluded from MEX generation. The simulation executes them in the regular interpreted mode.

Pixel-Stream Design

The function defined in DesignAccelerationHDLDesign.m accepts a pixel stream and five control signals, and returns a modified pixel stream and control signals. For more information on the streaming pixel protocol used by System objects from the Vision HDL Toolbox™, see Streaming Pixel Interface (Vision HDL Toolbox).

In this example, the function contains the Edge Detector System object.

The focus of this example is the workflow, not the algorithm design itself. Therefore, the design code is quite simple. Once you are familiar with the workflow, it is straightforward to implement advanced video algorithms by taking advantage of the functionality provided by the System objects from Vision HDL Toolbox.

Create MEX File and Simulate the Design

Generate and execute the MEX file.

codegen('DesignAccelerationHDLTestBench');
DesignAccelerationHDLTestBench_mex;

Code generation successful.


10 frames have been processed in 5.34 seconds.
Average frame rate is 1.87 frames/second.

The viewer displays the original video on the left, and the output on the right.

HDL Code Generation

Enter the following command to create a new HDL Coder™ project in the temporary folder

coder -hdlcoder -new DesignAccelerationProject

Then, add the file DesignAccelerationHDLDesign.m to the project as the MATLAB Function and DesignAccelerationHDLTestBench.m as the MATLAB Test Bench.

Refer to Get Started with MATLAB to HDL Workflow for a tutorial on creating and populating MATLAB HDL Coder projects.

Launch the Workflow Advisor. In the Workflow Advisor, right-click the 'Code Generation' step. Choose the option 'Run to selected task' to run all the steps from the beginning through HDL code generation.

Examine the generated HDL code by clicking the links in the log window.