[CI][microbenchmarks] Add onednn backend to GEMM with transposed matrices

[Egor-Krivov]

Without BENCHMARKING_METHOD="ELAPSED_TIME" I get ~6000TFLOPS GeoMean for onednn measurements.

Closes #2456

[Egor-Krivov]

Test run https://github.com/intel/intel-xpu-backend-for-triton/actions/runs/11258504227

[whitneywhtsang]

Test run https://github.com/intel/intel-xpu-backend-for-triton/actions/runs/11258504227

Please fix the CI failure.

[Egor-Krivov]

f6a4dda removed fast_flush option from triton

[Egor-Krivov]

Test run https://github.com/intel/intel-xpu-backend-for-triton/actions/runs/11258504227

Please fix the CI failure.

Done https://github.com/intel/intel-xpu-backend-for-triton/actions/runs/11273309867/job/31350042844?pr=2445

[whitneywhtsang]

Without BENCHMARKING_METHOD="ELAPSED_TIME" I get ~6000TFLOPS GeoMean for onednn measurements.

Are you saying that the current default timing method IPEX would give incorrect results?

[Egor-Krivov]

Without BENCHMARKING_METHOD="ELAPSED_TIME" I get ~6000TFLOPS GeoMean for onednn measurements.

Are you saying that the current default timing method IPEX would give incorrect results?

Yes, without ELAPSED_TIME, combined with onednn backend gives incorrect results. Probably wrong kernel is picked.
But in this PR there is no issue, since we measure onednn with ELAPSED_TIME

[whitneywhtsang]

Probably wrong kernel is picked.

hmm... kernel_name is not used in do_bench_ipex.

[Egor-Krivov]

Probably wrong kernel is picked.

hmm... kernel_name is not used in do_bench_ipex.

I'm not talking about kernel_name variable. There is some logic extracting kernels on lines 114:120.

[Egor-Krivov]

@whitneywhtsang
We'll probably need to remove remaining usage of fast_flush from benchmark_testing.py in the future.

[whitneywhtsang]

@whitneywhtsang We'll probably need to remove remaining usage of fast_flush from benchmark_testing.py in the future.

Sure, I can add a PR for that.

[Egor-Krivov]

@whitneywhtsang Everything works. Could you approve it now?

[anmyachev]

This will also change the default measurement method even if IPEX is set for xetla and triton.

[Egor-Krivov]

Yes. Otherwise we get wrong measurements for onednn.

[anmyachev]

Yes. Otherwise we get wrong measurements for onednn.

For legacy profiler or upstream profiler?

[Egor-Krivov]

~6000 TFLOPs for legacy
Last time I run onednn with upstream profiler it just failed. No kernel was found

[anmyachev]

I see, btw you could use kernel_name=gemm_kernel for upstream profiler - it should work.

[anmyachev]

~6000 TFLOPs for legacy

As for this, I suggest using the function explicitly do_bench_elapsed_time if the profiler legacy mode is set, with a comment what is workaround is needed, since it gives invalid numbers. Something like this:

    if provider == 'onednn':
        do_bench = benchmark_suit.do_bench
        if BENCHMARKING_MODE == 'PYTORCH_LEGACY_PROFILER_USING_IPEX':
            # comment here
            do_bench = benchmark_suit.do_bench_elapsed_time
        _, min_ms, max_ms, mean_ms, cv = do_bench(lambda: torch.matmul(torch_a, torch_b), warmup=10,
                                                                 rep=10, quantiles=quantiles, kernel_name="gemm_kernel")

This way you won't change the numbers that are used in dashboards.

[Egor-Krivov]

Ok, let me try that

[whitneywhtsang]

@whitneywhtsang We'll probably need to remove remaining usage of fast_flush from benchmark_testing.py in the future.

Sure, I can add a PR for that.

Done in #2460.

[anmyachev]

Please revert this

[anmyachev]

The main idea is that USE_IPEX_OPTION should have priority over BENCHMARKING_METHOD.

[anmyachev]

LGTM! @whitneywhtsang ?

[whitneywhtsang]

LGTM! @whitneywhtsang ?

Thanks for asking! Mostly LGTM, one concern is as we are using oneDNN as baseline to compare with Triton, would using two different benchmarking methods to compare affect the comparison?
I assume we will soon change default to use elapsed time with 2025.0? If so, then the problem I mentioned would be temporary.