How to synchronise video to matlab plot

I have a video of chopping a potato wherein the chopping activites are captured by a pressure sensor. The data is intepretated and plotted with matlab.

How can I synchronise the video to the plot as to know how the pressure values changes as the video plays.

Answer 1

[Saumik Kumar Dey]

According to my understanding, you want to play the video and plot pressure variation with time along with that. You can achive that by two subplots one for playing the video feed and the other for plotting the pressure data. You can use the matlab class "VideoReader" for reading the frames of you video file.

Consider the following example as a guideline to achieve your goal. Make sure the initial and final experiment times of the video matches with that of pressure data.

%% Setup the subplots
ax1 = subplot(2,1,1); % For video
ax2 = subplot(2,1,2); % For pressure plot

%% Setup VideoReader object
filename = 'SomeVideoFileName';
v = VideoReader(filename);
nFrames = v.Duration*v.FrameRate; % Number of frames

% Display the first frame in the top subplot
vidFrame = readFrame(v);
image(vidFrame, 'Parent', ax1);
ax1.Visible = 'off';

%% Load the pressure data
t = 0:0.01:v.Duration; % Cooked up for this example, use your actual data
y = sin(t);

nDataPoints = length(t); % Number of data points

step = round((nDataPoints/nFrames));
index = 1:step:nDataPoints;
i = 2;

% Diplay the plot corresponds to the first frame in the bottom subplot
h = plot(ax2,t(1:index(i)),y(1:index(i)),'-k');
% Fix the axes
ax2.XLim = [t(1) t(end)];
ax2.YLim = [min(y) max(y)];

%% Animate
while hasFrame(v)
    pause(1/v.FrameRate);
    
    vidFrame = readFrame(v);
    image(vidFrame, 'Parent', ax1);
    ax1.Visible = 'off';
    
    i = i + 1;
    set(h,'YData',y(1:index(i)), 'XData', t(1:index(i)))
end

Note: This example considers "nDataPoints > nFrames". The other condition can be handled similarly.

[Sujan Ponnappa]

I tired this code , but I'm getting the following error

Error using image
Cannot set property to a deleted object

Error in Untitled5 (line 13)
image(vidFrame, 'Parent', ax1);

[Saumik Kumar Dey]

You are getting that error because the axis object "ax1" is deleted for some reason.

Make sure you are running the whole code at once just by replacing "SomeVideoFileName" with an existing video filename.

[Sujan Ponnappa]

Thanks , I got it running.

But I'm stuck with another error now.

Index exceeds the number of array elements (0).

Error in Untitled5 (line 22)
h = plot(ax2,t(1:index(i)),y(1:index(i)),'-k');

[Sujan Ponnappa]

Ok I got it working.

Becuase in my case, the no of frames> no of data points.

But the plot is running fast, how can I sync the speed with the video?

[Sujan Ponnappa]

I tried using pause(0.1) in the while loop, but it not slow enough to sync

[Sujan Ponnappa]

Its working now.

The pause(0.1) command was too fast. I changed it to pause(0.7) so as to match.

How can i skip first few frames of the video?

[Saumik Kumar Dey]

First, make sure the "pause" value is always set to "1/v.FrameRate" which will play the video in normal speed.

In case "nDataPoints < nFrames" you can try out one of the following:

• Loop through all the datapoints and skip some of the video frames. You can use the "read" command to extract a video frame with a particular index.
• You can make "nDataPoints" more than or equal to "nFrames" by repeating each value in the array "y" by "round(nFrames/nDatapoints)" times. In this case you have to add additional time values in the array "t" by setting a step size of "1/v.FrameRate" seconds.

To skip first few frames of the video, you can set up a counter and find out the frame number from which you want to start. Then use "read" command to read frames from a specific frame.

[Sujan Ponnappa]

Thank you for your reply.

How do I slow down only the plotting rate and not the video speed. When I use pause, it slows up both the graph and the video speed.

Kindly help.

[Saumik Kumar Dey]

One way to slow down the plotting rate (without changing the pause value) is plotting more number of data points in a single frame. Consider the example below.

t = 0:0.01:10;
x = sin(t);
nDataPoints = length(t);

step = 5; % Increase this value to get a higher plot rate
index = 1:step:nDataPoints;

h = plot(t(1:index(2)), x(1:index(2)), '-k');
axis([0 10 -1.1 1.1])

tic
for i = 3:length(index)
    set(h, 'XData', t(1:index(i)), 'YData', x(1:index(i)));
    pause(0.01)
end
toc

In the above example, keep the pause value constant and change the "step" value to any posoitve integer and see the total time elapsed. Here "step" controls the number of data points plotted in a single execution of the for loop. You can use a similar stategy in your code.

[Sujan Ponnappa]

I have attached the excel file of the pressure data to this. I am not able to sync the plotting rate to the video . How can make sure that they are in sync?

%% Setup the subplots
ax1 = subplot(2,1,1); % For video
ax2 = subplot(2,1,2); % For pressure plot
%%% Setup VideoReader object
filename = 'slice1.avi';
v = VideoReader(filename);
frameratevideo=v.FrameRate;
nFrames = v.Duration*v.FrameRate; % Number of frames
% Display the first frame in the top subplot
vidFrame = readFrame(v);
v.CurrentTime = 16;
image(vidFrame, 'Parent', ax1);
ax1.Visible = 'off';
%%% Load the pressure data
t = ts; % Cooked up for this example, use your actual data
y = ys;
nDataPoints = length(t); % Number of data points
%step = round((nFrames/nDataPoints));
index = 1:0.44:nDataPoints;
i = 2;
% Diplay the plot corresponds to the first frame in the bottom subplot

h = plot(ax2,t(1:index(i)),y(1:index(i)),'-k');
    
% Fix the axes
ax2.XLim = [t(1) t(end)];
ax2.YLim = [min(y) max(y)];
%%% Animate
while hasFrame(v)
    pause(1/v.FrameRate);
   %pause(0.01)
    vidFrame = readFrame(v);
    image(vidFrame, 'Parent', ax1);
    ax1.Visible = 'off';
    
    i = i + 1;
    set(h,'YData',y(1:index(i)), 'XData', t(1:index(i)))
    
end

[sensor]

@Sujan Ponnappa hi did you find the solution ?

Answer 2

[youjarr]

Hey guys,

perfect code, thank you very much.

I have two questions:

How can I save the fig to reopen the file with the video?

Because when I do savefig it is not opening.

How can I replay within the fig?

Thank you very much.

[youjarr]

And I am getting an Error:

Index exceeds the number of array elements. Index must not exceed 361.

Error in VideoSubplot (line 50)

set(h1,'YData',y1(1:index1(j)), 'XData', t02(1:index1(j)))

I thought the code handles the different length of video and measured data?

My nFrames is smaller then my nDataPoints