Image acquisition in simulink (Framerate problem)

9 views (last 30 days)
Timo
Timo on 13 Apr 2011
Hello,
I'm using simulink to process live images. But I have problems with the speed/framerate of the model.
I reduced the model to only this blocks:
From Video Device
Frame Rate Display
To Video Display
Even in this model the Framerate is below 3fps. If I also remove the "To Video Display" Block the fps climbs to 8fps. Also to slow.
I made the settings of the shutter and so. In the Preview in the "From Video Device" settings the video is displayed smoothly.
I tryed out the sampling time attribute of the "From Video Device" set it to 1/30 and 1/120. But no change. I'm not really comfortable with the solver and sampling time.
How can I get to a faster processing? Can you give me a clue?
Thank you for your help!
I will appreciate it.
Yours sincerely, Timo
I'm using:
Matlab 2010b
Simulink
Image Acquisition Toolbox
Video and Image Processing Blockset
  2 Comments
Shankar Subramanian
Shankar Subramanian on 13 Apr 2011
Hi Timo,
Changing the sample time attribute from 1/30 to 1/120 affects only the timing of execution of the block in relation to the simulation time.
1. What camera are you using? Do you have vendor application in which you are able to acquire faster? What is the rate that you are expecting from the camera?
2. Some cameras do have a "Frame Rate" property when you get to the source properties. Can you double click on "From Video Device" block in your model and click the "Edit properties..." button to check if it exists. If so, what is the value that it is showing?
3. Also, what is the ROI that you have for this particular acquisition. Higher the ROI, the lower the frame rate. Can you also try lowering the ROI and see if your acquisition rate increases?
Thanks
Shankar
Timo
Timo on 14 Apr 2011
Hallo Shankar,
thank you for your answer.
1. An ICube Camera, which model I don't know. Yes in a vendor application the framerate is high.
The max Framerate of the Cam is 30fps. For my tracking application a higher framerate is an increase of precision. I would be lucky with a framerate above 15fps. I want to embed this tracking in a closed-loop control.
2. No this "Framerate Property" doesn't exist.
3.It is a RGB24_1280x1024 Image. In the Preview it works well(framerate is high). But with the 3 blocks model, which doesn't do anything but displaying the video and the framerate, the framerate is low (3 fps with/8fps without - displaying the video).
I also reduced the ROI: framerate increased with displaying the video to 5fps for an 8*640*480. A lower ROI is a loose of precision, I will probably need.
Thanks & Greeting
Timo

Sign in to comment.

Answers (10)

Lukas
Lukas on 19 Jan 2012
We've probably found a simple solution for Simulink - to create your own simulink block based on m-functions from Image acquisition toolbox. Bellow you can find the code for Simulink->User Defined Functions->S-function block. It works well at 30 FPS. Just one known lack is to overflow the memory after tens seconds of run, so it is suitable just for short experiments or you need to make periodical flush of the image buffer (e.g. function getdata(), but it causes a few frames delay).
function camera_pike_s_function(block)
setup(block);
%end camera_pike_s_function
function setup(block)
block.SetPreCompPortInfoToDefaults;
block.NumInputPorts = 0;
block.NumOutputPorts = 1;
block.OutputPort(1).Dimensions = [480 640]; % TODO - automatic setting from camera properties
block.OutputPort(1).DatatypeID = 3; % uint8
block.OutputPort(1).Complexity = 'Real';
block.NumDialogPrms = 0;
block.SampleTimes = [-1 0]; %Inherited sample time
block.SimStateCompliance = 'DefaultSimState';
block.RegBlockMethod('PostPropagationSetup', @DoPostPropSetup);
block.RegBlockMethod('Start', @Start);
block.RegBlockMethod('Outputs', @Outputs);
block.RegBlockMethod('Update', @Update);
block.RegBlockMethod('Terminate', @Terminate);
%end setup
function DoPostPropSetup(block)
% TODO - get camera properties form block parametres
device = videoinput('avtmatlabadaptor_r2009b', 1);
set_param(block.BlockHandle,'UserData',device);
src = getselectedsource(device);
set(src, 'FrameRate', '30');
block.NumDworks = 1;
block.Dwork(1).name = 'image';
block.Dwork(1).Dimensions = block.OutputPort(1).Dimensions(1) * block.OutputPort(1).Dimensions(2);
block.Dwork(1).DatatypeID = 3; %uint8
block.Dwork(1).Complexity = 'Real';
%end DoPostPropSetup
function Start(block)
device = get_param(block.BlockHandle,'UserData');
set(device, 'FramesPerTrigger', inf)
start(device);
%end Start
function Outputs(block)
block.OutputPort(1).Data = reshape(block.Dwork(1).data,block.OutputPort(1).Dimensions);
%end Outputs
function Update(block)
device = get_param(block.BlockHandle,'UserData');
%disp(device.FramesAvailable);
numAvailable = device.FramesAvailable;
if (numAvailable > 0)
%snapshot = getdata(device, 1); %remove it from buffer is too slow, don't use it
snapshot = peekdata(device, 1);
snapshot = snapshot(:);
if (length(snapshot) == length(block.Dwork(1).data))
block.Dwork(1).data = snapshot;
end
end
%end Update
function Terminate(block)
device = get_param(block.BlockHandle,'UserData');
stop(device);
%get recorded imageData and export it into workspace (if needed)
%imageData = getdata(device, device.FramesAvailable);
%evalin('base','imageData = 0;');
%assignin('base','imageData',imageData);
%destroy video object
delete(device);
clear device
%end Terminate

Shankar Subramanian
Shankar Subramanian on 18 Apr 2011
Hi Timo,
I do not know which model of ICube Camera you have but I had checked their product pages where they specify frame rate along with ROI. I might not be seeing same camera, but some cameras of ICube specify 30fps or 15fps at lower ROIs and it also changes depending on whether you have USB2.0 or lower.
The output of From Video Device block is single by default. Can you change it to uint8 and determine the frame rate? Also do check the frame rate without the Display block in the model.
Thanks.

Shankar Subramanian
Shankar Subramanian on 22 Apr 2011
Hi Timo,
Can you run the following code in MATLAB and check the frames rate (fps) value. This calculates the rate at which images are acquired in MATLAB.
% Select the right adaptor, deviceID and format.
vid = videoinput('winvideo', 1, 'RGB24_1280x1024');
vid.FramesPerTrigger = 20;
start( vid );
wait( vid, Inf );
[d t] = getdata( vid, vid.FramesAvailable );
fps = 1 / mean( diff( t ) );
Shankar

Timo
Timo on 18 Apr 2011
Hello Shankar,
Thank you again for your reply. I actual get higher framerates outside of the Simulink model. I have USB 2. I removed also the display block and got up to 7fps, still to slow. I will try your uint8 tip tomorrow and post the result.
Thanks Timo

Timo
Timo on 19 Apr 2011
Hallo Shankar,
I tried uint8 but it didn't change anything. Perhaps the performance will get better if I exported the model with the real time workshop.
Thanks Timo

Timo
Timo on 26 Apr 2011
Hallo Shankar,
thanks for continuing helping me.
I executed your code and got 7.1 fps, but the light isn't so good. So I adjusted the exposure like I did it in Simulink.
If I excecute the code below I get fps=21.4
vid = videoinput('winvideo', 1, 'RGB24_1280x1024');
set(vid,'FramesPerTrigger',20);
src = getselectedsource(vid)
set(src, 'ExposureMode', 'manual');
set(src, 'Exposure', -8);
start( vid );
wait( vid, Inf );
[d t] = getdata( vid, vid.FramesAvailable );
fps = 1 / mean( diff( t ) );
Thanks again, Timo

Gilmore
Gilmore on 19 Nov 2011
Hello Everyone, I am trying to modify the exposure rate of my webcam for maximum fps. I cannot find it listed to the edit properties of my video device block. Any suggestions pls. I am using the 2010A

Lukas
Lukas on 11 Jan 2012
Hello everyone,
we got the exact problem as Timo described. We just use another camera, namely Pike Camera made by Allied Vision Technologies. Everything works well with it except the simulation run.
Win XP SP3 32bit
Matlab R2010b
AVT Adaptor for Matlab Image Acquisition v 1.0.3.
Driver: Intek
Smartview (vendor software): OK (30 FPS)
Imaqtool: OK (30 FPS, mode F0M5)
Simulink - block properties - preview: OK (30 FPS, mode F0M5)
Simulink - simulation - mode F0M5: just 1 FPS (30 FPS set in Edit properties)
ROI settings: no effect
We have contacted the Allied Vision technical support, but they suppose that the problem is in Matlab or Image Acquisition toolbox.
Please, did anyone of you find any solution? It would be really helpful. We've tried possibly all combinations of drivers and setting, but with no result.
Thank you very much in advance.
Lukas

Timo
Timo on 12 Jan 2012
Hello,
we changed to C# and OpenCV, because we didn't find a solution.
Timo

Shankar Subramanian
Shankar Subramanian on 25 Jan 2012
Hi Lukas,
This is an issue with Allied Vision Tech adaptor and they have an updated version for the fix as well. Please download the latest from the following link. Please copy the library into your adaptor directory path for AVT.
Also, do update if this fixed your performance problem.
Thanks
Shankar

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!