KeyError 'psutil' in some __del__ call inside python shutdown

[yoori]

Summary

• OS: linux, debian, ubuntu
• Architecture: 64bit
• Psutil version: 5.4.3
• Python version: 3.6.8, 3.11.9
• Type: core

Description

psutil, in general case, cannot be used in a some class __del__ method (directly or indirectly),
because __del__ can be called on python shutdown - and here no sys.modules['psutil'] record (it already removed):

import psutil
import os

class Test:
  def __del__(self):
    cur_process = psutil.Process(os.getpid())

t = Test()

Result :

Exception ignored in: <bound method Test.__del__ of <__main__.Test object at 0x7f1297886d68>>
Traceback (most recent call last):
  File "n.py", line 6, in __del__
  File "/usr/lib64/python3.6/site-packages/psutil/__init__.py", line 341, in __init__
  File "/usr/lib64/python3.6/site-packages/psutil/__init__.py", line 363, in _init
  File "/usr/lib64/python3.6/site-packages/psutil/_pslinux.py", line 1417, in __init__
  File "/usr/lib64/python3.6/site-packages/psutil/_pslinux.py", line 214, in get_procfs_path
KeyError: ('psutil',)

In my case I try to use psutil for kill frozen chromedriver child processes for exclude zombies formation - and this psutil behavior is strange for me ...

[giampaolo]

This behavior is related to how cPython shuts down, not to psutil per se. I think you would have a similar problem also with other imports. E.g. even if you move import psutil into the __del__ method you'll get this:

$ python3 foo.py 
Exception ignored in: <function Test.__del__ at 0x78e756911750>
Traceback (most recent call last):
  File "/home/giampaolo/foo.py", line 9, in __del__
ImportError: sys.meta_path is None, Python is likely shutting down

More in general, __del__ is tricky. AFAIK it is not even guaranteed to run, and as such, is not a good place for cleanup code. It's usually better to put cleanup code in a finally clause as in:

try:
    main()
finally:
    cleanup()

...or:

import atexit

@atexit.register
def cleanup():
   ...

Try one of the 2 approaches above and forget about __del__ (at least in this specific scenario).