Incorrectly getting a blank video when using videowriter function to display data

Hello, I am having a difficulty with writing and displaying a video for data that is meant to represent multiple frames that have been corrected for motion. I am essentially stabilizing the motion of a "shaky" video. I am still learning how to use Matlab within my research so this could be lack of coding experience on my end.

I am not receiving any error message but rather getting a video output of a blank video. In the first section, I am correcting the frame offsets. "Stabilized" should be the data used that I want to represent in a movie. When calling on it as a singular and averaged frames figure, I receive what I expect. The "Stabilized" file contains Y axis, X axis, RGB, and number of frames. The second section is my attempt to write a movie of my data which is where I believe I might be doing it incorrectly. I followed the examples provided in youtube and under "Help: Videowriter" in the command row, but nothing has worked. If more information is needed, please do not hesistate to ask. I greatly appreciate the help.

Answer 1

[Walter Roberson]

You are not changing the display between getframe() so all of the frames would be the same.

If you have pre-computed frames stored in stabilized then write them directly, like

for i = 1 : size(stabilized,3)
       currFrame = stabilized(:,:,:,i);
       writeFrame(vidObj, currFrame);
end

And... you should be sure to initialize stabilized to the appropriate data type, such as

stabilized = zeros([2 2 1 1].*size(video), 'like', video);

which I suspect was the major source of your problem with the output being white.

[Walter Roberson]

Actually you can skip the loop and just use

writeFrame(vidObj, stabilized);

without any loop. 4D arrays are understood as a series of RBG images.

[Diego]

Hi Walter. Thank you so much for your reply. This is how I wrote your suggestions. For my output, I received a blacked out video this time for the duration of 14 seconds.

[Diego]

This is the output I am receiving in the command line. I noticed that height, width, and frame count are not being considered. I also attempted to do it with the for loop and receive the same output. Thank you so much for helping me. What does the 'like" variable do?

[Image Analyst]

What is the class of the stabilized variable? Is it uint8 or double?

size(currFrame)
whos currFrame
size(stabilized)
whos stabilized

[Walter Roberson]

It is as-if video is an empty variable.

With respect to like -- like is an option in several of the routines that generate arrays of data, such as zeros() and ones() and rand() and randi() -- even true() and false(). The option is always followed by a variable or expression. MATLAB examines the class() of what is passed, and the output array is set to be that class (provided it is compatible)

u8 = uint8(123);
i16 = int16(123);
sing = single(123);
logi = logical(123);

ru8 = randi(9, 1, 2, 'like', u8)

ru8 = 1×2

6 7

ri16 = randi(9, 1, 2, 'like', i16)

ri16 = 1×2

4 5

rsing = randi(9, 1, 2, 'like', sing)

rsing = 1×2

6 9

rfal = false(1, 2, 'like', logi)

rfal = 1×2 logical array
   0   0

whos

  Name        Size            Bytes  Class      Attributes

  cmdout      1x33               66  char                 
  i16         1x1                 2  int16                
  logi        1x1                 1  logical              
  rfal        1x2                 2  logical              
  ri16        1x2                 4  int16                
  rsing       1x2                 8  single               
  ru8         1x2                 2  uint8                
  sing        1x1                 4  single               
  u8          1x1                 1  uint8                

Notice how the datatype of the variables is the same as the datatype of the expression passed after the 'like' option.

There is a very similar option of just passing in a class name directly, such as

rcl = randi(1, 2, 'uint16')

rcl = 2×2

1 1 1 1

whos rcl

  Name      Size            Bytes  Class     Attributes

  rcl       2x2                 8  uint16              

I used to think that using 'like' was just an alias for taking class() of an expression and passing that in, but it turns out that 'like' also notices whether the prototypes are complex are not -- that and a series of bug fixes and release changes early on in 'like' support made it clear that they are internally handled different ways, even if I cannot explain how so or why. (There were some early bug reports about using 'like' was producing values of the wrong type, and the wording of those indicated that they have different backing implementation methods than just passing a class name.)

[Diego]

To Mr. Walter Robinson, thank you for that easy to follow explanation. That makes sense now.

To Image Analyst,

'stabilized' size dimensions are 2160 x 3840 x 3 x 467 and a double class. Does that make a difference in how I wrote the code?

[Diego]

Whenever, I create a figure as imagesc, I am able to view the average of all of the frames overalyed on each other.

[Walter Roberson]

I suspect that video (and then now stabilized ) contains double precision values outside of the range 0.0 to 1.0 . When you ask to imwrite() or writeVideo() and you pass double precision data, then MATLAB assumes that the valid range of data is 0.0 to 1.0 . If that is not the case, then you need to either change the datatype of the data or you need to rescale the data before writing it.

If you have uint8 data you should not double() it to convert it to double precision -- not unless you uint8() the processed results later. Instead you should im2double() which will rescale the uint8 to the 0 to 1 range.

If you need to fix-up a double precision value afterwards, rescale(), and you should probably use the inputmin and inputmax options. In this particular case where the double precision is 0.0 to 255.0 range you could just uint8() instead of rescale()

img = imread('flamingos.jpg');
imd1 = double(img);
imd2 = im2double(img);
imr = uint8(rescale(imd1, 0, 255, 'inputmin', 0, 'inputmax', 255));

imshow(imd1); title('converted to double without rescaling')

That is an all-white image because MATLAB expected double() to be in the range 0 to 1

imshow(imd2); title('converted to double properly')

imshow(imr); title('rescaled to uint8')

[Diego]

Your logic makes complete sense. Originally the video is in a 4-D uint8 but becomes a double once running the stabilization code. I believe I followed your instructions correctly in converting it to a proper double and then to the uint8. I did not rescale and was unsure if you wanted me to include (imd1, 0, 255, 'inputmin', 0, 'inputmax', 255)) because I would not need to convert it to double precision as you mentioned and therefore would not have imd1. Let me know if my understanding is flawed. I still got the blacked out 14 second video with same video properties as no height, width, or frame count.

%% Create Movie (Loop)
stabilized= zeros([2 2 1 1].*size(video), 'like', video);

imd2= im2double(stabilized); %convert to double properly
imr=uint8(stabilized); %rescale the data before writing it 

vidObj = VideoWriter('Stabilization.mp4','MPEG-4'); %any name can be given to this 
vidObj.FrameRate = 30
open(vidObj);
writeVideo(vidObj,imr);
close (vidObj)

[Walter Roberson]

If your values in stabilized is double precision data in the range of 0 to 255, then just uint8() it and write that.

If video is uint8 then

stabilized= zeros([2 2 1 1].*size(video), 'like', video);

would make stabilized into uint8, not into double.

The code you do inside the loop that copies parts of video into stabilized will not change the datatype if you initialize stabilized the way I showed.

It is not uncommon for people to write code that looks something like

img = double(imread('flamingos.jpg'));
img = img + (randn(size(img)) * noise_std + noise_median);

The problem with that is that using double() keeps the data range 0 to 255, and then when you try to image() or writeVideo() MATLAB is going to see the double() data and believe that anything outside the range 0 to 1 is out of range. The much better way to deal with such situations would be to use

img = im2double(imread('flamingos.jpg'));
img = img + (randn(size(img)) * noise_std + noise_median);

and later after all of the processing is done, im2uint8() the results.

[Walter Roberson]

With regards to the 'like' option, see https://www.mathworks.com/help/matlab/matlab_oop/class-support-for-array-creation-functions.html for more details on how like is implemented for classes.

Answer 2

[Diego]

The stabilized file was in double precision in the range of 0 to 255. I followed your instructions for simply writing it as a uint8() and it worked. I commented out

stabilized= zeros([2 2 1 1].*size(video), 'like', video);

My code for reference to anyone who experiences the same issue in the future is

img= uint8(stabilized; %rescale the data back to uint8 before writing it 

vidObj = VideoWriter('Stabilization.mp4','MPEG-4');%any name can be given to this 
vidObj.FrameRate = 15 %set this your own preference 
open(vidObj);
writeVideo(vidObj,img);
close (vidObj)

Thank you so much Mr. Roberson. I am extremely grateful for your help. Your explanations were phenomenal and learned more than I anticipated learning. Not only did I get the outcome I wanted, but I also learned how to inherit the right logic for solving this MatLab issue.