Custom Python package loader for the plugin subsystem with scie distributions and local devs

[achimnol]

Python has a versatile, highly customizable module loading subsystem.

Example:

Let's use them to auto-detect and load plugin packages:

• Local dev setups: scan a designated directory to get the entrypoints and import paths
  • Currently we are doing a similar thing for plugins directory in pants.toml, but it is just manually parsing(!) setup.cfg and BUILD files found inside it.
  • I'd like to let it use the standard package entrypoint.
• scie-based distributions: scan the current working directory and/or the location of the executable binary to list .whl files and load them with their entrypoints.

This is a prerequisite for the "neo" enterprise TUI installer.

Potential useful references:

• https://docs.python.org/3/library/importlib.html
• https://docs.python.org/3/library/zipimport.html
• https://docs.python.org/3/library/site.html
• https://docs.python.org/3/library/modulefinder.html
• https://packaging.python.org/en/latest/specifications/section-installation-metadata/

(cc: @xyloon @adrysn)