Tune metric encode quality?

[mikk9]

Intel recently added a tuning option into oneVPL for various video quality metrics. I think this could be interesting how this effects subjective quality and scores.

ONEVPL_EXPERIMENTAL
The TuneQuality enumerator specifies tuning option for encode. Multiple tuning options can be combined using bit mask. 
enum

MFX_ENCODE_TUNE_DEFAULT = 0, The balanced option to keep quality balanced across all metrics.  
MFX_ENCODE_TUNE_PSNR    = 0x1, The encoder optimizes quality according to Peak Signal-to-Noise Ratio (PSNR) metric.
MFX_ENCODE_TUNE_SSIM    = 0x2, The encoder optimizes quality according to Structural Similarity Index Measure (SSIM) metric. 
MFX_ENCODE_TUNE_MS_SSIM = 0x4, The encoder optimizes quality according to Multi-Scale Structural Similarity Index Measure (MS-SSIM) metric. 
MFX_ENCODE_TUNE_VMAF    = 0x8, The encoder optimizes quality according to Video Multi-Method Assessment Fusion (VMAF) metric. 
MFX_ENCODE_TUNE_PERCEPTUAL = 0x10, The encoder makes perceptual quality optimization. 

https://github.com/oneapi-src/oneVPL-intel-gpu/blob/1288f4f5407ce2757c604e406fd1d674b9df4fdf/api/vpl/mfxstructures.h#L4855
intel/vpl-gpu-rt@d74bfb5

[bavdevc]

hey @mikk9
is there actually any change in output format when using that options? I tried the scenario options before and it looks like they are not used yet inside the intel driver software....

kind regards

[mikk9]

scenario-info game_streaming makes a big difference but it only works for CBR/VBR bitrate mode. Some settings can boost PSNR/SSIM. Also LA depth can be varied between 3-100. Without streaming scenario there is only on/off depth, it doesn't matter if it's set to 1 or 100.

[bavdevc]

thanks a ton for that info, was looking hard to get any details so far

[mikk9]

Depth 3-29 works fundamentally different than 30-100. You can try for example dist 8 with depth 8 and dist 8 with depth 50. I would also suggest using --gop-len 250 with this. Depth 8 requires much more bitrate for the desired encoding bitrate. It usually boosts PSNR/SSIM and often also subjective quality.

[bavdevc]

but it only works for CBR/VBR bitrate mode

that explains a lot because all my use-cases and automated test are based on CQP and ICQ atm. I didn't have the time yet to finish the other encoding modes... will follow the next weeks - thank you a lot for your information!

[mikk9]

--scenario-info game_streaming works for AV1 and also h264 CBR/VBR. When I try HEVC I get this error

Failed to get output frame info!: insufficient buffer at input/output

[rigaya]

@mikk9
TuneQuality enumerator seems to be not exposed to oneVPL API. I'll add options to QSVEncC once they are exposed to the API.

[mikk9]

Yes could be, it's a very new entry under the Experimental flag.

[mikk9]

TuneQuality enumerator should be included in API 2.9 and OneVPL v2023.2.0.

https://github.com/oneapi-src/oneVPL/releases

[rigaya]

QSVEnc 7.37 updates to oneVPL 2.9 and adds --tune option.

Please note that the latest 4255 Win driver is still at API 2.8, so not actually tested yet.

[mikk9]

I hope the next driver is getting API 2.9 support.

[bavdevc]

I hope the next driver is getting API 2.9 support.

Windows beta 4311 doesn't have it? Linux should have it right now, but a lot of the other stuff not working there...

[mikk9]

4311 comes with API 2.08.....we need API 2.09. And even with API 2.09 there is no guarantee, we have to try it out.

[rigaya]

WHQL 4314 driver, Beta 4335 driver, unfortunately both still with API 2.8.

[mikk9]

Could someone check driver 4369? It says API 2.09 but nothing is supported. Here from feature check:

[bavdevc]

ffmpeg qsv works fine with that driver version (decoder+encoder):

[AVHWDeviceContext] Use Intel(R) oneVPL to create MFX session, API version is 2.8, the required implementation version is 1.3
[AVHWDeviceContext] Initialize MFX session: implementation version is 2.9
[Parsed_vpp_qsv_0] Use Intel(R) oneVPL to create MFX session with the specified MFX loader
[Parsed_vpp_qsv_0] VPP: input is video memory surface
[Parsed_vpp_qsv_0] VPP: output is video memory surface
[av1_qsv] Using input frames context (format qsv) with av1_qsv encoder.
[av1_qsv] Encoder: input is video memory surface
[av1_qsv] Use Intel(R) oneVPL to create MFX session with the specified MFX loader
[av1_qsv] Using the intelligent constant quality (ICQ) ratecontrol method
...

I guess @rigaya needs to rebuild the windows binaries / 7.36 is the last working version with the new driver

qsvencc64 --check-impl
API 2.09: hw(mfx-gen), Acceleration d3d11
API 2.07: hw(mfx-gen), Acceleration d3d11

InitInput: Success.
InitSession: mfx lib version: 2.09, impl hw,via_d3d11
InitSession: Success.
Detected avaliable features for hw API v2.09, AV1, ICQ (Intelligent Const. Quality)
RC mode x
10bit depth x
Hyper Mode x
Interlace x
VUI info x
Trellis x
BFrame/GopRef x
Adaptive_I x
Adaptive_B x
WeightP x
WeightB x
FadeDetect x
B_Pyramid x
+ManyBframes x
PyramQPOffset x
TuneQuality x
ScenarioInfo x
MBBRC x
ExtBRC x
AdaptiveRef x
AdaptiveLTR x
AdaptiveCQM x
LA Quality x
QP Min/Max x
IntraRefresh x
No Deblock x
No GPB x
Windowed BRC x
PerMBQP(CQP) x
DirectBiasAdj x
MVCostScaling x
SAO x
Max CTU Size x
TSkip x
AV1 encoding is not supported on current platform.

[mikk9]

Yes indeed, I just tried FFmpeg and Handbrake and they work as usual.

[21:41:17] CPU: 12th Gen Intel(R) Core(TM) i5-12500
[21:41:17] - Intel microarchitecture Alder Lake performance hybrid architecture
[21:41:17] - logical processor count: 12
[21:41:17] Intel Quick Sync Video support: yes, gpu list: 0
[21:41:17] Intel Quick Sync Video discrete adapter with index 0
[21:41:17] Impl mfx-gen library path: C:\Windows\System32\DriverStore\FileRepository\iigd_dch_d.inf_amd64_85819d054b1d86c5\libmfx64-gen.dll
[21:41:17] - Intel Media SDK hardware: API 2.9 (minimum: 1.3)
[21:41:17] - Decode support: h264 hevc (8bit: yes, 10bit: yes) av1 (8bit: yes, 10bit: yes)
[21:41:17] - H.264 encoder: yes
[21:41:17] - preferred implementation: hardware (1) via D3D11
[21:41:17] - capabilities (hardware): lowpower breftype icq vsinfo chromalocinfo opt1 opt2+mbbrc+extbrc+trellis+repeatpps+ib_adapt+nmpslice
[21:41:17] - H.265 encoder: yes (8bit: yes, 10bit: yes)
[21:41:17] - preferred implementation: hardware (1) via D3D11
[21:41:17] - capabilities (hardware): lowpower bpyramid icq vsinfo masteringinfo cllinfo opt1
[21:41:17] - AV1 encoder: yes (8bit: yes, 10bit: yes)
[21:41:17] - preferred implementation: hardware (1) via D3D11
[21:41:17] - capabilities (hardware): lowpower bpyramid icq vsinfo opt1 av1bitstream

[rigaya]

QSVEnc 7.41 will fix encode failing with 4369.

However, --tune will still not work. (Not sure, but seems like the driver does not support yet.)

[bavdevc]

Thank you very much for that quick fix / new release - confirmed API 2.9 working now - looks like you are the 1st programmer who uses that API version so I guess the experimental features will work soon -when more people are using it- and intel gets more feedback from all users.

btw. qsvencc ist faster than ever since the 7.39 changes - GPU usage in Task Manager is now a flat line at 100% per used core, no spikes anymore - great job with that multi threading fixes!

[rigaya]

@bavdevc Thank you for confirming QSVEnc 7.41 works file.

btw. qsvencc ist faster than ever since the 7.39 changes

Nice to hear that! Adding multi threading was rather complex work, so I'm glad it works out fine.

[mikk9]

With the latest driver --la-depth no longer works as usual with AV1 and CBR/VBR. Now it requires --extbrc but unfortunately the metric scores are in most cases lower with extbrc+la-depth. There is a workaround: --scenario-info game_streaming with --la-depth 1 (1 or 2 is mandatory).

This is the same:

Old driver: --la-depth 1....100 ( 1 or 100 didn't make a difference)
New driver: --la-depth 1 --scenario-info game_streaming (above 2 differs)

[bavdevc]

yeah @mikk9 - I also noticed some changes between Intel driver revisions lately, worse between windows<>linux on the same stack - Intel software is very volatile atm.

btw. @rigaya is https://github.com/Intel-Media-SDK/MediaSDK/blob/master/tools/tracer/README.md still a thing after the switch from MediaSDK to onevpl? - or what tool would you recommend to trace some onevpl/ex-mediasdk API calls?

[thx4ever]

Hi, i have tested with latest beta driver on Intel DG2 ARC A380 tune quality and get a error about enough buffer, tune quality on this driver are enabled

Codec: H.265/HEVC FF
CBR VBR AVBR QVBR CQP LA LAHRD ICQ LAICQ VCM
RC mode o o x o o x x o x x
10bit depth o o x o o x x o x x
Hyper Mode x x x x x x x x x x
Interlace x x x x o x x x x x
VUI info o o x o o x x o x x
Trellis x x x x x x x x x x
BFrame/GopRef o o x o o x x o x x
Adaptive_I o o x o o x x o x x
Adaptive_B o o x o o x x o x x
WeightP o o x o o x x o x x
WeightB o o x o o x x o x x
FadeDetect o o x o o x x o x x
B_Pyramid o o x o o x x o x x
+ManyBframes o o x o o x x o x x
PyramQPOffset x x x x o x x x x x
TuneQuality o o x o o x x o x x
ScenarioInfo o o x o o x x o x x
MBBRC o o x o x x x o x x
ExtBRC o o x x x x x x x x
AdaptiveRef x x x x x x x x x x
AdaptiveLTR o o x o o x x o x x
AdaptiveCQM x x x x x x x x x x
LA Quality x x x x x x x x x x
QP Min/Max o o x o x x x o x x
IntraRefresh o o x o o x x o x x
No Deblock o o x o o x x o x x
No GPB o o x o o x x o x x
Windowed BRC x x x x x x x x x x
PerMBQP(CQP) x x x x x x x x x x
DirectBiasAdj x x x x x x x x x x
MVCostScaling x x x x x x x x x x
SAO o o x o o x x o x x
Max CTU Size o o x o o x x o x x
TSkip o o x o o x x o x x

MFXENCODE: Failed to get required buffer size for MFXENCODE: invalid video parameters.
Failed to initialize encoder. : invalid video parameters.

My encodings parameters:
./QSVEncc64 -i teste.mkv --avhw --codec hevc --quality best --profile main10 --bframes 16 --b-pyramid --adapt-ltr --open-gop --async-depth 4 --no-repeat-pps --no-tskip --sao none --fixed-func --vbr 2500 --max-bitrate 12000 --psnr --ssim --aud --chapter-copy --colormatrix auto --transfer auto --colorprim auto --chromaloc auto --output-depth 10 --no-hevc-gpb --videoformat ntsc --pic-struct --tune vmaf -o teste_SV.mp4

Regards

[ma3uk]

Hi, i have tested with latest beta driver on Intel DG2 ARC A380 tune quality and get a error about enough buffer, tune quality on this driver are enabled

Codec: H.265/HEVC FF CBR VBR AVBR QVBR CQP LA LAHRD ICQ LAICQ VCM RC mode o o x o o x x o x x 10bit depth o o x o o x x o x x Hyper Mode x x x x x x x x x x Interlace x x x x o x x x x x VUI info o o x o o x x o x x Trellis x x x x x x x x x x BFrame/GopRef o o x o o x x o x x Adaptive_I o o x o o x x o x x Adaptive_B o o x o o x x o x x WeightP o o x o o x x o x x WeightB o o x o o x x o x x FadeDetect o o x o o x x o x x B_Pyramid o o x o o x x o x x +ManyBframes o o x o o x x o x x PyramQPOffset x x x x o x x x x x TuneQuality o o x o o x x o x x ScenarioInfo o o x o o x x o x x MBBRC o o x o x x x o x x ExtBRC o o x x x x x x x x AdaptiveRef x x x x x x x x x x AdaptiveLTR o o x o o x x o x x AdaptiveCQM x x x x x x x x x x LA Quality x x x x x x x x x x QP Min/Max o o x o x x x o x x IntraRefresh o o x o o x x o x x No Deblock o o x o o x x o x x No GPB o o x o o x x o x x Windowed BRC x x x x x x x x x x PerMBQP(CQP) x x x x x x x x x x DirectBiasAdj x x x x x x x x x x MVCostScaling x x x x x x x x x x SAO o o x o o x x o x x Max CTU Size o o x o o x x o x x TSkip o o x o o x x o x x

MFXENCODE: Failed to get required buffer size for MFXENCODE: invalid video parameters. Failed to initialize encoder. : invalid video parameters.

My encodings parameters: ./QSVEncc64 -i teste.mkv --avhw --codec hevc --quality best --profile main10 --bframes 16 --b-pyramid --adapt-ltr --open-gop --async-depth 4 --no-repeat-pps --no-tskip --sao none --fixed-func --vbr 2500 --max-bitrate 12000 --psnr --ssim --aud --chapter-copy --colormatrix auto --transfer auto --colorprim auto --chromaloc auto --output-depth 10 --no-hevc-gpb --videoformat ntsc --pic-struct --tune vmaf -o teste_SV.mp4

Regards

Which driver version are you talking about, I tried version 4382, but TuneQuality doesn't work

[mikk9]

Is it still not supported from the driver? Feature check says it's supported for AV1 and I can see it in the encoding log but it does nothing.

By the way they have changed MFX_ENCODE_TUNE_DEFAULT into MFX_ENCODE_TUNE_OFF in API 2.10, see here: https://intel.github.io/libvpl/latest/VPL_change_log.html#id1

It's listed under experimential, maybe it makes a difference?

By default, experimental API is turned off in the header files. To enable it, need to define ONEVPL_EXPERIMENTAL macro during the application compilation stage.

https://intel.github.io/libvpl/latest/Experimental.html