tests: WinError 32 on CI again

[jku]

While deleting the temporary directory:

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process:

While my initial reaction is that this some Windows BS again... maybe there are broken tests. Our test setup is in some cases unnecessarily complex and could hide issues.

Full log from https://github.com/theupdateframework/tuf/pull/1337/checks?check_run_id=2296694965:

2021-04-08T12:27:41.4005743Z ======================================================================
2021-04-08T12:27:41.4007002Z ERROR: test_update (test_multiple_repositories_integration.TestMultipleRepositoriesIntegration)
2021-04-08T12:27:41.4008300Z ----------------------------------------------------------------------
2021-04-08T12:27:41.4009113Z Traceback (most recent call last):
2021-04-08T12:27:41.4011194Z   File "D:\a\tuf\tuf\tests\test_multiple_repositories_integration.py", line 175, in tearDown
2021-04-08T12:27:41.4012361Z     shutil.rmtree(self.temporary_directory)
2021-04-08T12:27:41.4013338Z   File "c:\hostedtoolcache\windows\python\3.7.9\x64\lib\shutil.py", line 516, in rmtree
2021-04-08T12:27:41.4014214Z     return _rmtree_unsafe(path, onerror)
2021-04-08T12:27:41.4015136Z   File "c:\hostedtoolcache\windows\python\3.7.9\x64\lib\shutil.py", line 395, in _rmtree_unsafe
2021-04-08T12:27:41.4015989Z     _rmtree_unsafe(fullname, onerror)
2021-04-08T12:27:41.4016900Z   File "c:\hostedtoolcache\windows\python\3.7.9\x64\lib\shutil.py", line 395, in _rmtree_unsafe
2021-04-08T12:27:41.4018012Z     _rmtree_unsafe(fullname, onerror)
2021-04-08T12:27:41.4018950Z   File "c:\hostedtoolcache\windows\python\3.7.9\x64\lib\shutil.py", line 404, in _rmtree_unsafe
2021-04-08T12:27:41.4019860Z     onerror(os.rmdir, path, sys.exc_info())
2021-04-08T12:27:41.4020803Z   File "c:\hostedtoolcache\windows\python\3.7.9\x64\lib\shutil.py", line 402, in _rmtree_unsafe
2021-04-08T12:27:41.4021550Z     os.rmdir(path)
2021-04-08T12:27:41.4023055Z PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'D:\\a\\tuf\\tuf\\tests\\tmplg5mtj4n\\TestMultipleRepositoriesIntegration_ziar2llq\\repository_server2'

In the example case the test creates a tempdir using tempfile, then creates another tempdir inside it using unittest_toolbox, then it starts two server processes that use the inner temp dir...

Then when tearing down it calls unittest_toolbox teardown first, which probably deletes the directory while the process is still running. So:

• Process shutdown should always be the first thing in test teardown
• test setup should not be as complex as it is

The only flaw in this theory is that the error only comes in line 175 in test_multiple_repositories_integration.py -- how did the original unittest_toolbox..TearDown() succeed on line 164?

[jku]

from teodora:

Modified Test Case ignores permission error as a subclass of OSError

This explains why line 164 "succeeds" -- it just quietly fails to remove the directory.