VMAF - Video Multi-Method Assessment Fusion

VMAF is a perceptual video quality assessment algorithm developed by Netflix. VMAF Development Kit (VDK) is a software package that contains the VMAF algorithm implementation, as well as a set of tools that allows a user to train and test a custom VMAF model. Read this techblog post for an overview, or this post for the latest updates and tips for best practices.

News

• (7/13/20) We have created a memo to share our thoughts on VMAF's property in the presence of image enhancement operations, its impact on codec evaluation, and our solutions.
• (2/27/20) We have changed VMAF's license from Apache 2.0 to BSD+Patent, a more permissive license compared to Apache that also includes an express patent grant.
• (2/27/20) We made a few changes in a recent refactoring effort:
  • Migrated the build system from makefile to meson.
  • Restructured the code.
  • Introduced a new release candidate API with the associated library libvmaf_rc and executable vmaf_rc, co-existing with the current libvmaf and vmafossexec, all under libvmaf/build. The new release candidate API is designed for better interoperability with encoding optimization. We will deprecate the old API on a future date.

Frequently Asked Questions

Refer to the FAQ page.

Usages

The VDK package offers a number of ways to interact with the VMAF implementation. The core libvmaf library with feature extractors is written in C. The rest of the code including the classes for machine learning regression, training and testing VMAF models and etc., is written in Python. Besides, the libvmaf C library partially replicates the logic in the regression classes, such that the VMAF prediction (excluding training) is fully implemented.

There are a number of ways one can use the package:

• VMAF Python library offers a full array of wrapper classes and scripts to allow running basic VMAF command line, training and testing a VMAF model on video datasets, and visualization tools, etc.
• vmafossexec - a C executable offers running the prediction part of the algorithm in full, such that one can easily deploy VMAF in a production environment without needing to configure the Python dependencies. Additionally, vmafossexec offers a number of exclusive features, such as 1) speed optimization using multi-threading and skipping frames, 2) optionally computing PSNR, SSIM and MS-SSIM metrics in the output.
• libvmaf - a C library offers an interface to incorporate VMAF into your C code.
• VMAF is now included as a filter in FFmpeg and can be configured using: ./configure --enable-libvmaf. See the FFmpeg documentation for usage.
• VMAF Dockerfile generates a VMAF docker image from the VMAF Python library. Refer to this document for detailed usages.
• Build VMAF on Windows: follow instructions on this page.

Datasets

We also provide two sample datasets including the video files and the properly formatted dataset files in Python. They can be used as sample datasets to train and test custom VMAF models.

Models

Besides the default VMAF model which predicts the quality of videos displayed on a 1080p HDTV in a living-room-like environment, VDK also includes a number of additional models, covering phone and 4KTV viewing conditions. Refer to the models page for more details.

Confidence Interval

Since VDK v1.3.7 (June 2018), we have introduced a way to quantify the level of confidence that a VMAF prediction entails. Each VMAF prediction score now can come with a 95% confidence interval (CI), which quantifies the level of confidence that the prediction lies within the interval. Refer to the VMAF confidence interval page for more details.

Matlab Functionality

Besides the Python/C/C++ part of the repository, we also introduced a number of algorithms that are implemented in Matlab. For example, users can calculate ST-RRED, ST-MAD, SpEED-QA, and BRISQUE. For more details, see the Matlab Usage page for more details.

Report of VMAF Bad Cases

Over time, we have received feedbacks on when VMAF's prediction does not reflect the expected perceptual quality of videos, either they are corner cases where VMAF fails to cover, or new application scenarios which VMAF was not initially intended for. In response to that, we have created the Google form to allow users to upload their video samples and describe the scenarios. The bad cases are valuable for improving future versions of VMAF. Users can opt in or out for sharing their sample videos publicly. Please submit the bad cases through this link.

References

Refer to the references page.

External Resources

Refer to this page for a list of 3rd-party tools built on top of VMAF.