A MATLAB implementation of feature fused VIDeo quality EVALuator (VIDEVAL) proposed in [IEEE TIP2021] UGC-VQA: Benchmarking blind video quality assessment for user generated content. IEEEXplore
Check out our BVQA resource list and performance benchmark/leaderboard results in https://github.com/tu184044109/BVQA_Benchmark.
码云Gitee mirror: https://gitee.com/zhengzhong-tu/VIDEVAL_release
The recommended system is Linux, than Windows. Mac is not supported though due to FRIQUEE issue.
Should you find any problems, please feel free to send an issue or email me.
- 🐛 [12-17-2020] Mac system is not supported since there was an error on calling the
mexfiles here. It also means that the FRIQUEE model cannot be run on Mac too. Thanks to @CXMANDTXW for finding this in issue. - ✨ [12-20-2020] A light version
VIDEVAL_lightwas provided as a speed-up version of vanilla VIDEVAL (scales better for high resolution and high fps), albeit the performance may drop. Please check Performances for the performance-speed tradeoff. Check Demos for the running of light VIDEVAL. - 🐛 [04-22-2021] Fixed nan bug by using nanmean(). Thanks to @Sissuire.
VIDEVAL means the original VIDEVAL in demo_compute_VIDEVAL_feats.m. It operates on the original frame resolution sampled at every second frame.
VIDEVAL_light_{res}s_{fps}fps is the light version of VIDEVAL where input video is spatially downscaled to {res} at a frame sampling rate of {fps} fps. Check demo_compute_VIDEVAL_light_feats.m. Note that speed-up parameters [max_reso,frs_per_blk] can be played with for specific application scenarios.
| Methods | KoNViD-1k | LIVE-VQC | YouTube-UGC | All-Combined |
|---|---|---|---|---|
| VIDEVAL | 0.7832 / 0.7803 | 0.7522 / 0.7514 | 0.7787 / 0.7733 | 0.7960 / 0.7939 |
| VIDEVAL_light_720s_6fps | 0.7510 / 0.7510 | 0.7429 / 0.7453 | 0.7514 / 0.7477 | 0.7621 / 0.7689 |
| VIDEVAL_light_720s_3fps | 0.7492 / 0.7508 | 0.7174 / 0.7225 | 0.7445 / 0.7413 | 0.7584 / 0.7666 |
| VIDEVAL_light_480s_3fps | 0.7281 / 0.7338 | 0.7144 / 0.7209 | 0.7140 / 0.7134 | 0.7462 / 0.7537 |
Speed was evaluated on the whole calc_VIDEVAL_feats.m function. The unit is average secs/video.
| Methods | 540p | 720p | 1080p | 4k@60 | scalability |
|---|---|---|---|---|---|
| VIDEVAL | 61.9 | 146.5 | 354.5 | 1716.3 | 🐌 🤬 |
| VIDEVAL_light_720s_6fps | 29.9 | 68.2 | 72.6 | 205.2 | 🚅 😅 |
| VIDEVAL_light_720s_3fps | 15.7 | 33.6 | 40.9 | 115.9 | |
| VIDEVAL_light_480s_3fps | 12.2 | 16.6 | 20.4 | 77.9 | 🚀 😊 |
Note:
- 540p: 540p@30fps@8sec in KoNViD-1k
- 720p: 720p@30fps@10sec in LIVE-VQC
- 1080p: 1080p@30fps@10sec in LIVE-VQC
- 4k@60: 4k@60fps@20s in YouTube-UGC
- Aggressive spatial downsampling will harm the performance on spatially-dominated datasets, KoNViD-1k, YouTube-UGC.
- Increasing frame sampling rate benefits the performance on temporal-distorted or motion-intensive videos, as those in LIVE-VQC.
[Note] Recommended system is Linux. Windows MATLAB users may suffer from
WARNING: You should compile the MEX version of "*.c"and was slower from our tests.
- MATLAB
- FFmpeg
- python3
- sklearn
demo_compute_VIDEVAL_feats.m
or light version:
demo_compute_VIDEVAL_light_feats.m
You need to specify the parameters
This pre-trained model was trained on the combined dataset.
You need first extract features:
demo_compute_VIDEVAL_feats.m
or light version:
demo_compute_VIDEVAL_light_feats.m
Then run:
demo_pred_MOS_pretrained_VIDEVAL.py
or light version:
demo_pred_MOS_pretrained_VIDEVAL_light.py
demo_eval_BVQA_feats_one_dataset.py
You need to specify the parameters
demo_eval_BVQA_feats_all_combined.py
You need to specify the parameters
If you use this code for your research, please cite our papers.
@article{tu2020ugc,
title={UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content},
author={Tu, Zhengzhong and Wang, Yilin and Birkbeck, Neil and Adsumilli, Balu and Bovik, Alan C},
journal={arXiv preprint arXiv:2005.14354},
year={2020}
}
Zhengzhong TU, [email protected]