Run VMAF Library in FFMPEG

[ecomte]

After reading documentation, I saw it is able to run VMAF via a plugin on ffmpeg. I have yet to figure out how to though.

ffmpeg says "simply" install libvmaf, and run ./configure --enable-libvmaf. This makes sense, but there are no ./configure files to be found and after scrounging everywhere to figure out how this is possible, I have come up empty. I have significant experience using VMAF outside of ffmpeg, but I would very much like to run VMAF through the libvmaf on ffmpeg. Please let me know how this is possible.

Thanks

[li-zhi]

@ashk43712

[ashishps1]

Hi @ecomte , configure file is there inside FFmpeg/ so if you want to run vmaf using FFmpeg first install libvmaf library using make install and then from the root directory of FFmpeg (which is FFmpeg/) do ./configure --enable-libvmaf . After that do make install. Run vmaf through FFmpeg using the following command:
ffmpeg -i main -i ref -lavfi libvmaf -f null -
Also, libvmaf was changed recently so I updated the patch on FFmpeg which is not yet applied so you may get some warnings and errors on installing and running libvmaf. These two patches are yet to be applied: https://patchwork.ffmpeg.org/patch/4610/ and https://patchwork.ffmpeg.org/patch/4575/.
We will update soon.
You can follow this updated file on my personal repo:
https://github.com/ashk43712/FFmpeg/blob/libvmaf/libavfilter/vf_libvmaf.c
Let me know if you still face some issues.
Thanks.

[ecomte]

Hi. Thank you for your quick response, I greatly appreciate it. Will I have to do make install libvmaf from an external source, or is the library included in the FFmpeg development package?

Thanks.

[ashishps1]

Hi @ecomte, FFmpeg only links to libvmaf, the library itself is not included in the FFmpeg development package. You can install libvmaf from Netflix's VMAF package (this repo) using make and then make install.

[ecomte]

Hi @ashk43712. I was able to successfully install libvmaf. After installing, I "cd'ed" into the root FFmpeg directory and ran "./configure --enable-libvmaf". It gave the following error: "ERROR: libvmaf not found using pkg-config". Do you happen to have any idea what this could be? I was wondering if it could be an issue with how I have my CentOS machine set up, or if it is a problem with FFmpeg and the newly installed libvmaf.

Thanks.

[chuanY]

@i have same issues too.
hopes you can fixed.

and my vmaf version is 1.3.1

thanks

[ashishps1]

Hi @chuanY , can you mention your issues?

[chuanY]

when i run "./configure --enable-libvmaf" , i get "ERROR: libvmaf not found using pkg-config" issues

[ashishps1]

Hi, @ecomte and @chuanY . Can you check if the pkg-config file (libvmaf.pc) is copied inside /usr/local/lib/pkgconfig/ after running make install. Can you also check if the library (libvmaf.a) and the
library header (libvmaf.h) are copied to usr/local/lib/ and usr/local/include/ respectively. If these are not their then mostly the issue is with your system, maybe permissions , not sure though. Can you also try running them using sudo (sudo make install). FFmpeg should detect these files if it's there.
Thanks

[ecomte]

I checked all of those directories and they are containing the correct files. I tried running sudo make install, and then './configure --enable-libvmaf' and still got the same results: "ERROR: libvmaf not found using pkg-config".

@chuanY, let me know what you find out, I'm curious to see. Maybe it is an issue with my machine, but I will wait for an update to see if you're still having the same issues.
Thanks

[ashishps1]

Hi @ecomte and @chuanY, can you check the default path of your system's pkg-config files (using "pkg-config --variable pc_path pkg-config" or similar commands). Can you try moving libvmaf.pc at one of these paths and see if it works.
I think it makes sense to make the install location customizable to fix system specific issues.
Thanks.

[chuanY]

hi @ashk43712 @ecomte i fixed by i use latest master branch code, not 1.3.1 release tag, and i success

[nandakd]

Was able to configure ffmpeg with vmaf. Check ffbuild/config.log - you may need to set PKG_CONFIG_PATH.

[slhck]

So, just to summarize, for VMAF:

1. Clone https://github.com/Netflix/vmaf
2. Run the installation according to https://github.com/Netflix/vmaf/blob/master/resource/doc/libvmaf.md#use-libvmaf-with-ffmpeg

For FFmpeg:

1. Clone https://git.ffmpeg.org/ffmpeg.git
2. Install the prerequisites (see the Compilation Guides)
3. Run ./configure --enable-libvmaf (including other configuration switches if needed)
4. Run make, and optionally make install

[nandakd]

Has anyone tried executing the vmaf command itself through ffmpeg?

I've been trying based on the description https://ffmpeg.org/ffmpeg-filters.html#libvmaf, but not successful so far.

[slhck]

@nandakd If you're having issues, you need to show exactly what you've been doing and what output / errors you are getting. When you say that you were "not successful", that's not really a proper problem description.

[nandakd]

I tried the exact examples mentioned on that ffmpeg page, and got an ffmpeg segmentation fault.

I then tried using ffmpeg2vmaf, which worked, but does not support specifying a log file - need to use stdout/stderr redirecting . ffmpeg2vmaf also exits with this weird tty error, where echo is turned off (needs a reset).

[slhck]

got an ffmpeg segmentation fault

Same here – I've reported it on the FFmpeg bug tracker

[ashishps1]

Hi everyone, I have started looking into the issues. I'll try to resolve them as soon as possible.

[rbultje]

The segfault from ffmpeg bug #6691 is now fixed since 4c38aa79ece4fa0ea7533774197d6408d7d3b9e5.

[GyanD]

I've compiled libvmaf on Windows for integration with FFmpeg, so to clarify on what @slhck said above , python and its packages are not required in order to compile either the static libvmaf.a library or the standalone vmafossexec tool.

[xshen1122]

@ashk43712
I have successfully run ffmpeg + libvmaf for mpg file like: ffmpeg -i main.mpg -i ref.mpg -lavfi libvmaf -f null -
But I want to compare yuv file, "ffmpeg -i 1.yuv -i 2.yuv -lavfi libvmaf -f null - " , ffmpeg cannot regonize yuv:
[IMGUTILS @ 0x7ffc653935c0] Picture size 0x0 is invalid
1.yuv: Invalid argument

Any idea? How to compare 2 yuv files?

[GyanD]

That's an ffmpeg issue, not VMAF. The short answer is that since the inputs are raw, there's no metadata to inform ffmpeg about the properties of the input. ffmpeg needs that to initialize the decoder. For raw YUV, you'll have to specify the resolution, frame rate and pixel format for each input e.g. -video_size 1280x720 -pixel_format yuv420p -framerate 30 -i 1.yuv

[xshen1122]

@GyanD 👍
Thanks a lot. It's working now.

[chuanY]

still get segmentation fault at centos run ffmpeg @rbultje

[rbultje]

The patch isn't in ffmpeg yet, so you're probably running the old version.

[chuanY]

@rbultje what's time patch can be include as ffmpeg main branch

[rbultje]

This was merged mid-december:

http://git.videolan.org/?p=ffmpeg.git;a=commit;h=df3222d4bb18cbceda443def17b1b29067ed6e3f

[soulhdb]

my server has 16 cores,but when run libvmaf(do vmaf or psnr or ssim) filter only use one,is there a problem?

#131
and just as the issues describe, libavfilter/vf_libvmaf.c line:320 “pthread_join(s->vmaf_thread, NULL);“ cause segment fault

CentOS + ffmpeg4.0 + glibc2.17

[KevinJW]

http://ffmpeg.org/pipermail/ffmpeg-devel/2018-April/229205.html contains a suggested fix for the pthread_join() issue in ffmpeg.

Kevin

[GyanD]

Above patch available at https://patchwork.ffmpeg.org/patch/8699/

[MukeshA1]

Hi,
Is VMAF library now bundled with ffmpeg or do we still need to build it separately? I am trying to use VMAF on windows OS. Is there a way I can directly download the executable/library and integrate with ffmpeg on windows ( in case it is not bundled with ffmpeg)?
Thanks!

[GyanD]

FFmpeg contains a wrapper for VMAF, but the library is separate and has to be linked. Pass --enable-libvmaf to ffmpeg's configure.

However, at the present time, I believe ffmpeg build with libvmaf HEAD will fail as the VMAF API has changed and the wrapper hasn't been updated.

[MukeshA1]

Hi Gyan,

At https://github.com/Netflix/vmaf , I see the instructions for getting the vmaf only for linux and mac os. Any idea what I need to do to get the vmaf tool on Windows. Is there any repository where the compiled version of the tool is present for Windows OS? ( As no instructions are mentioned for Windows OS, I am just worried whether there will be quite a few prerequisites for building it on windows..)

Thanks

[slhck]

There is no pre-built version that I know of. Compiling ffmpeg under Windows is a little more complicated.

You may be able to use https://github.com/jb-alvarado/media-autobuild_suite which should enable libvmaf.

[li-zhi]

@GyanD @MukeshA1 release v1.3.5 or v1.3.4 should still work with current FFmpeg build.

[cosmin]

Latest snapshot of ffmpeg (as of 8/7 I believe) will compile against latest libvmaf (1.3.9). I think this issue can be closed.

[yayaMissie]

So, just to summarize, for VMAF:

1. Clone https://github.com/Netflix/vmaf
2. Install the prerequisites and the required pip commands
3. Run make, edit your PYTHONPATH, and run make install

For FFmpeg:

1. Clone https://git.ffmpeg.org/ffmpeg.git
2. Install the prerequisites (see the Compilation Guides)
3. Run ./configure --enable-libvmaf (including other configuration switches if needed)
4. Run make, and optionally make install

my question is:

1. How to edit PYTHONPATH and Why should we do the step?
2. I have followed the steps mentioned above(except doing editing PYTHONPATH and not using make install cmd). When using libvmaf, ffmpeg throw exception as:
   ~/ffmpeg$ ./ffmpeg -i /mnt/ssd/videos_qaperf/videos_input/qaperf_30fps_1min_720p_2M_264_gop10.mp4 -i /mnt/ssd/videos_qaperf/videos_input/qaperf_30fps_1min_720p_2M_264_gop10.mp4 -libvmaf -f null - ffmpeg version N-93397-gf8075b2 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 4.9.2 (Debian 4.9.2-10+deb8u1) configuration: --enable-libvmaf --enable-version3 libavutil 56. 26.100 / 56. 26.100 libavcodec 58. 47.103 / 58. 47.103 libavformat 58. 26.101 / 58. 26.101 libavdevice 58. 6.101 / 58. 6.101 libavfilter 7. 48.100 / 7. 48.100 libswscale 5. 4.100 / 5. 4.100 libswresample 3. 4.100 / 3. 4.100 Unrecognized option 'libvmaf'. Error splitting the argument list: Option not found
   Why does this error happen? How can I fix the problem?

[GyanD]

libvmaf is used via a filter, so it's -vf libvmaf.

[slhck]

How to edit PYTHONPATH

Edit your shell configuration file (e.g., ~/.bashrc), see here: https://bic-berkeley.github.io/psych-214-fall-2016/using_pythonpath.html

Why should we do the step?

It's actually just required for running the Python wrapper for VMAF since the program is not installed as a package. For running the libvmaf filter in ffmpeg, this step should not be necessary. (I'll edit my comment above to reflect that.)

The whole procedure is documented here: https://github.com/Netflix/vmaf/blob/master/resource/doc/libvmaf.md#use-libvmaf-with-ffmpeg

[yayaMissie]

Thanks. @slhck @GyanD
There are some problems when using libvmaf via ffmpeg.

1. libvmaf is used as a filter, but "-vf libvmaf" is simple filter graph, which is only for one input and output. So I changed to "-filter_complex/-lavfi"
2. The cmd I used is as follows:
   ./ffmpeg -i input.mp4 -i ref.mp4 -filter_complex "[0:v][1:v]libvmaf" -f null -
   But it doesn't work. The following shows the error I encountered:

Stream mapping:
Stream #0:0 (h264) -> libvmaf:main (graph 0)
Stream #1:0 (h264) -> libvmaf:reference (graph 0)
libvmaf (graph 0) -> Stream #0:0 (wrapped_avframe)
Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
Start calculating VMAF score...
Output #0, null, to 'pipe:':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.27.100
Stream #0:0: Video: wrapped_avframe, yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn, 30 tbc (default)
Metadata:
encoder : Lavc58.48.101 wrapped_avframe
Stream #0:1(und): Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s (default)
Metadata:
handler_name : SoundHandler
encoder : Lavc58.48.101 pcm_s16le
Caught runtime_error: StatVector size is 0.
[Parsed_libvmaf_0 @ 0x3f445c0] libvmaf encountered an error, check log for details
Error while filtering: Invalid argument
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #1:0
Conversion failed!
Start calculating VMAF score...
Caught runtime_error: StatVector size is 0.

[li-zhi]

We have a new section at: https://github.com/Netflix/vmaf/blob/master/resource/doc/libvmaf.md#use-libvmafa-with-ffmpeg
describing a sample cmd using FFmpeg+libvmaf. Hope this helps.

[ngcthuong]

Hello, I aslo has the same problem with

Start calculating VMAF score...
Caught runtime_error: StatVector size is 0.

It is libvmaf with ffmpeg, ubuntu in my case. How did you @yayaMissie resolve it?

The link is not working @li-zhi .
Thanks

[droukd]

mkfifo ref.yuv
ffmpeg -i source.mov -f yuv4mpegpipe ref.yuv
 ⌘T
mkfifo av1.yuv
ffmpeg -i av1.webm -f yuv4mpegpipe av1.yuv
 ⌘T
vmaf --reference ref.yuv --distorted av1.yuv --width 1920 --height 1080 --pixel_format 420 --bitdepth 8

VMAF version 0d84f29
5802 frames ⠍⡁ 4.58 FPSError reading YUV frame data.
Error reading YUV frame data.
5803 frames ⢋⠁ 4.58 FPS
problem reading pictures

problem flushing context

😔
no frame limiting ability.

[hotkidfamily]

mkfifo ref.yuv
ffmpeg -i source.mov -f yuv4mpegpipe ref.yuv
 ⌘T
mkfifo av1.yuv
ffmpeg -i av1.webm -f yuv4mpegpipe av1.yuv
 ⌘T
vmaf --reference ref.yuv --distorted av1.yuv --width 1920 --height 1080 --pixel_format 420 --bitdepth 8

VMAF version 0d84f29
5802 frames ⠍⡁ 4.58 FPSError reading YUV frame data.
Error reading YUV frame data.
5803 frames ⢋⠁ 4.58 FPS
problem reading pictures
problem flushing context

😔
no frame limiting ability.

@droukd did this problem have been solved ? I have the same question.

[avinabsaha]

@droukd @hotkidfamily, how did you solve the issue?

[1480c1]

mkfifo ref.y4m dist.y4m
vmaf --reference ref.y4m --distorted dist.y4m &
(ffmpeg -y -v error -i Football.mp4 ref.y4m & ffmpeg -y -v error -i av1.webm dist.y4m & ) & fg vmaf
65 frames ⠍⠀ 2.52 FPS

for my machine at least, you have to open the files for reading first and then write to it, else it blocks ffmpeg from writing and vmaf from reading.

If you wanted to use libvmaf directly from ffmpeg without all of this, it should be outlined at https://github.com/Netflix/vmaf/blob/master/resource/doc/ffmpeg.md

ffmpeg -i av1.webm -i Football.mp4 -lavfi libvmaf -f null -