Make examples_torch_job faster

[ydshieh]

What does this PR do?

This job sometimes having some test timeout (> 120s.). Even if job passes, the log looks like

115.55s call     examples/pytorch/test_pytorch_examples.py::ExamplesTests::test_run_semantic_segmentation
66.07s call     examples/pytorch/test_pytorch_examples.py::ExamplesTests::test_run_squad
48.24s call     examples/pytorch/test_pytorch_examples.py::ExamplesTests::test_run_speech_recognition_seq2seq

It turns out that setting OMP_NUM_THREADS=1 has a huge impact on this job.

This PR sets OMP_NUM_THREADS=8 to make it run faster. It now looks like

25.78s call     examples/pytorch/test_pytorch_examples.py::ExamplesTests::test_run_semantic_segmentation
22.82s call     examples/pytorch/test_accelerate_examples.py::ExamplesTestsNoTrainer::test_run_swag_no_trainer
20.69s call     examples/pytorch/test_pytorch_examples.py::ExamplesTests::test_run_speech_recognition_seq2seq

Note that setting OMP_NUM_THREADS>1 with pytest -n where n > 1 is going to break things (timeout, blocked etc.).

[ydshieh]

The original value 1 make some test slow and sometimes timeout (> 120 s)

[amyeroberts]

This feels like something which is going to make things very hard to debug and cause flaky tests: can we guarantee the same threads will receive the same jobs each time?

[ydshieh]

This doesn't mean the jobs will be sent to different thread. It's really low level where a python process (that uses some C module that uses OpenMP) dispatches some (sub)tasks to different threads , I believe.

At the test level, that is controlled by pytest -n x , where is the number of workers (processes) to run different tests. And so far we use pytest -n 8 withtout having same worker will receive the same jobs each time.

[amyeroberts]

OK! :)

[ydshieh]

When OMP_NUM_THREADS>1, we should set this to 1.

(well, since we have dist=loadfile in pytest option and we have only 2 files concerned, we won't see the problems. But let's not risk our future life ...)

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[amyeroberts]

Thanks for updating!