[ci] Remove gcc tests on MacOS

[borchero]

For some time, I've been thinking about simplifying the toolchain setup in this repository by leveraging more of the conda ecosystem. While working on this, I realized that gcc is not available on conda-forge for MacOS. I briefly talked to @xhochy about this earlier and he mentioned that it is very uncommon for gcc to be used for compilation on MacOS.

As a result, I'd propose to get rid of gcc compilation on MacOS in the CI. This opens the avenue for using the conda ecosystem to manage all dependencies required for compilation. While it is obviously possible to compile LightGBM using gcc on MacOS, I don't think it is necessary to officially support it.

---

As a side-effect, this PR also re-enables the MPI jobs on MacOS.

[jameslamb]

Thanks for the write-up and for starting this discussion.

I'm -1 on this change.

I do not support using conda's compilers to build Python wheels or other release artifacts in this repo.

It's important that compiling from source with standard system compiler distributions is tested here, as that's the way the project will be built by repackagers (e.g. MacPorts, nix packages, homebrew, various Linux distributions).

In my experience, conda's compilers are specifically intended to be used by conda-build, to create conda packages, and do things that make them unsuitable for creating more general-purpose artifacts intended for wide distribution.

As I briefly mentioned in #6489 (comment), those compilers do significant other work (like patching RPATH and similar attributes in binaries) to make things that they build all work together in the conda ecosystem. I don't support building artifacts that way which are intended to be used outside of that system (like the lib_lightgbm.dylib that is re-used in lightgbm-ruby, see #6489 (comment)).

For example, see https://docs.conda.io/projects/conda-build/en/stable/resources/compiler-tools.html#anaconda-compilers-implicitly-add-rpath-pointing-to-the-conda-environment.

... the GCC specs are customized so that ... an RPATH pointing to lib/ inside the current enviroment prefix directory ($CONDA_PREFIX/lib) is added. This is done by changing the link_libgcc: section inside GCC specs file...

conda-build knows how to make this automatically relocatable, so that this RPATH will be changed to point to the environment where the package is being installed (at installation time, by conda). But if you only pack this binary in a tarball, it will continue containing this hardcoded RPATH to an environment in your machine. In this case, it is recommended to manually remove the RPATH.

conda's compilers also come with activation scripts which set flags, environment variables, etc. in the build environment, some of which might be specific to the purpose of building conda packages. (see https://docs.conda.io/projects/conda-build/en/stable/resources/compiler-tools.html#using-the-compiler-packages). I'm afraid those may hide issues that users building from source (which is very common for this project) will then see because they aren't caught in CI.

Could you share more details on the benefits you think this project would get from using conda's compilers? Maybe they'll outweigh the costs/risks I've listed above.

I would really really like to hear @StrikerRUS 's opinion on this as well.

[StrikerRUS]

I'm absolutely agree with @jameslamb !
I learned that conda compilers do some additional stuff and are configured in a specific way many years ago and this is still true.
I'm 100% sure that "successful compilation with conda compilers" != "successful compilation with ordinal compilers"!
Also, as James has already mentioned, artifacts compiled with conda compilers will be hard-linked to some conda libraries, which will make those artifacts useless outside of conda ecosystem.

[borchero]

All right, let's get rid of this PR then 😅