Fail to import lightning due to missing dependency on setuptools

[cshimmin]

Bug description

Lightning appears to depend on the setuptools package at runtime via an import of the pkg_resources module. However, setuptools is not listed as a dependency, causing import failures in some environments.

In particular, when using the PDM package manager, the virtualenv it uses behind the scenes does not by default include a setuptools distribution (as it is typically only needed in the build environment). This means that installing lightning via pdm leads to a failure at import time. However, the issue can be reproduced in a standard virtualenv by simply uninstalling setuptools.

This bug seems to have been introduced in 2.1.x, and I have confirmed it in python 3.9.18 and 3.11.6.

What version are you seeing the problem on?

v2.1

How to reproduce the bug

python -m venv venv
source venv/bin/activate
pip uninstall -y setuptools
pip install lightning

python -c "import lightning" || python -c "import lightning.pytorch"

Error messages and logs

  File "<string>", line 1, in <module>
  File ".../venv/lib/python3.11/site-packages/lightning/__init__.py", line 18, in <module>
    from lightning.fabric.fabric import Fabric  # noqa: E402
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../venv/lib/python3.11/site-packages/lightning/fabric/__init__.py", line 5, in <module>
    from lightning_utilities.core.imports import package_available
  File ".../venv/lib/python3.11/site-packages/lightning_utilities/__init__.py", line 6, in <module>
    from lightning_utilities.core.apply_func import apply_to_collection
  File ".../venv/lib/python3.11/site-packages/lightning_utilities/core/__init__.py", line 4, in <module>
    from lightning_utilities.core.imports import compare_version, module_available
  File ".../venv/lib/python3.11/site-packages/lightning_utilities/core/imports.py", line 13, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

Environment

Current environment

- PyTorch Lightning Version: 2.1.x
- Python: version 3.9, 3.11 (tested)
- MacOS: 13.5.1 (m1 processor)
- How you installed Lightning: pip, pdm

More info

No response

[awaelchli]

@Borda @carmocca do we need to add setuptools to the package requirements?

[Borda]

As you can see from the trace, it eventually shall be added to utility package...

[Borda]

seems it was added in #155