How to extract/obtain motion vectors from .mp4 videos in MATLAB?

Hello, Is there any code (or inbuilt function) for extracting motion vectors (frame-to-frame) from MPEG-4 (H.264 compressed videos)? For my work, the input video is a scene of people moving through streets/malls.

[Walter Roberson]

If you need the vectors as stored in the file then you need to call upon some third party tool to process the file and return the information. The tools provided by Mathworks call upon operating system provided routines to decode frames.

[Tintumon]

Thanks for the answer. Did you mean the "ffmpeg" ?? Also, could please mention the tools provided by matlab to decode the frames??

[Walter Roberson]

ffmpeg would be an example of a library you could adapt to return the vectors as stored in the file.

The tools used internally by MATLAB are a mix of custom DLLs (all OS), and Microsoft Direct Show (Windows), and Windows Media Foundation (Windows), and Quicktime (Mac), and gstreamer (Linux)

For example for the random mp4 I had sitting around, VideoReader choose to use libmwvideofileavfreaderplugin.dylib (on Mac) to read the file, and that in turn calls upon /System/Library/Frameworks/CoreVideo.framework which is a Mac operating-system provided multimedia framework.

[Shahbaz Khan]

i have downloaded ffmpeg, how to use it to extract motion vectors? can you please guide. the file have several libraries and codes

[Tintumon]

Note: The answer is out of scope of this thread.

But, I will explain it.

1) Build the example programs (located in "doc\examples") by using the following command (run the command while you are in the source FFmpeg directory):

make examples

2) Execute the "extract_mvs" binary file, generated as a result of Step 1, to obtain the motion vectors from an h.264 compressed video (navigate to "doc\examples" or provide the absolute path while executing). Write the motion vectors to a text-file (outputFile.txt).

.\extract_mvs inputVideoName.extension > outputFile.txt

This ".txt" file can be read in MATLAB and used for further processing!

Answer 1

[Tintumon]

Thanks again for a detailed answer.

In the case of "VideoReader": 1) What if I had multiple codecs? I guess the routine chooses the default done. Can we manually change the default setting? 2) Also, Will it produce motion vectors as one of its output?

Answer 2

[Jishnu P]

I am not sure about MATLAB. But you can have a look at MV-Tractus.