[RFC] Problems with deleting config.cmake and .pc files

[barcharcraz]

We know that using find_package() and relying on the CMake behavior to find the dependencies is something that should be avoided in favor of the information provided by the package manager.

problems

• This may be true in some projects, but it completely breaks others. In particular some projects need to be able to build by finding their dependences without conan, in addition to with. For example many cross platform applications need to be able to build without conan dependencies for packaging in linux distributions, but may still want to support conan when compiling on windows.

• Another usecase this could break is when someone wants to compile a library using dependencies from conan, but is not the author of that library. They want to be able to use the cmake_paths generator, but because the package doesn't provide a config file at all, this won't work.

• Further, let's say conan-center packages all generate targets (with the find_package generators
  that have identical names to the ones the packages themselves install. This still breaks projects that want to be buildable without conan, because those generators will apply to ALL packages the projects use, not just the ones from conan-center. I recently ran into this trying to use fmt/6.0.0 from conan-center and cli11/1.6.1@bincrafters/stable. cli11, by default will install targets called CLI11::CLI11, and with find_package multi you get cli11::cli11. I can't pick and choose which packages use which config modules.

• Even using find_package instead (which generates findFoo.cmake instead of fooConfig.cmake files, so I can say find_package(CLI11 CONFIG)). The is because some packages (like CURL) only provide their config files in some situations, and you need to have a find module that looks for them, and falls back to other methods of searching (like cmake's PkgConfig support) if they are not found.

Solutions:

• require packages to include buildsystem generated configuration files, if provided, unless they are hopelessly broken
• prohibit packages from generating their own find modules at package() time (at generation time is fine)
• perhaps make the inclusion of said config files controllable with a package option
• if the user actually wants "pure" conan provided config files then cmake_find_package and cmake_find_package_multi will still take precedence over anything the package provides

It is possible there's some deeper reason for not including the cmake config files that I don't realize. However, if there is it would be nice if the FAQ on the wiki mentioned it.

[danimtb]

Hi @barcharcraz,

This topic has been discussed earlier and of course there is an entry about this in the FAQ section of the wiki explaining the reasons for this. Here you have it https://github.com/conan-io/conan-center-index/wiki/FAQ#why-are-cmake-findconfig-files-and-pkg-config-files-not-packaged

Additionally, we provide a check for package contributors that bans those files in the packages and an explanation about that: https://github.com/conan-io/conan-center-index/wiki/FAQ#why-are-cmake-findconfig-files-and-pkg-config-files-not-packaged

As a summary to answer the points explained above:

• The main goal of this repository is to manage all dependencies of packages explicitly with Conan. No other package managers or system ones are contemplated.

• cmake_paths is used to set the correct module paths to use the CMake files in the packages. We are not promoting this kind of usage for the reasons you explained above and because users should use generators for this purpose such as cmake_find_package or pkg_config.

• Regarding targets, we are working on this and we recently introduced the self.cpp_info.name feature so you can declare a different name for a package. This info can also be used for the name of CMake generators.

• Regarding the custom CMake macros, you can still include CMake files in packages that have additional functionalities. However, we are working on how to improve this use case.

Thanks

[mmha]

I strongly disagree. This isn't about dealing with different package management mechanisms, but about feeding the build system with the information and code it needs.

The main goal of this repository is to manage all dependencies of packages explicitly with Conan. No other package managers or system ones are contemplated.

There is no contradiction. The dependencies are managed with Conan regardless of build system files being present or not.

cmake_paths is used to set the correct module paths to use the CMake files in the packages. We are not promoting this kind of usage for the reasons you explained above and because users should use generators for this purpose such as cmake_find_package or pkg_config.

There is no 1:1 relationship between a Conan package and a CMake package. Qt5 (and soon 6) ships with a sizable amount of CMake configs, e. g. Qt5, Qt5QuickCompiler, Qt5EglFsKmsSupport and those cannot necessarily be split up into separate Conan packages. It also doesn't allow me to inject CMake code the way Config modules to. I therefore consider cmake_find_package too limited for real world use cases and don't use it.

Regarding targets, we are working on this and we recently introduced the self.cpp_info.name feature so you can declare a different name for a package. This info can also be used for the name of CMake generators.

Likewise, there is a n:m relationship between CMake targets and CMake packages:

• ECM for example defines none. ECM is also a hard requirement for all KDE libraries.
• Many define one target per package
• Other projects such as pybind11, Qt and LLVM define multiple ones and for the big frameworks it's very hard to declare the relationships between those targets correctly. Both Qt and LLVM still don't do it properly upstream and I doubt that a package maintainer with no involvement in these projects can do any better.

so an additional field in cpp_info doesn't always help.

Regarding the custom CMake macros, you can still include CMake files in packages that have additional functionalities. However, we are working on how to improve this use case.

It's important to note that CMake Config files are arbitrary code written by the code author and can contain functions as well, not just metadata. The CMake docs contain an example with advanced logic. And this still doesn't answer the question on how a downstream package in conan-center-index gets access to said CMake modules with approved means and without excessive patching. I wouldn't know how to package pybind11 without breaking this piece of upstream documentation.

---

To me it seems that this kind of post-install artifact modification is only creating extra work for both users and package maintainers, forces users to special case for Conan in their build scripts and it opens up an entire category of bugs such as #93. There is no harm keeping these files and the Conan generators take precendence already if a user chooses to use them, so why remove the files?

[danimtb]

As mentioned above, we are working towards integrating the CMake functions that the package ships for consumers. Here you will find information about the issue in Conan conan-io/conan#5798

Regarding the relationship of packages and CMake targets or components, we already evaluated that and the feature is still being developed conan-io/conan#5090

At this moment, we feel there is no need to package additional build system files that can lead to further issues in recipes that are not related to Conan, increasing the effort in maintenance. As you said, some of this config/find files are constantly broken and they could lead users to unpleasant experiences using Conan (for example using cmake_paths). If there is any lack of functionality or information provided by Conan generators, we are open for feedback and we will work on improving them.

Finally, please note that these checks are based on the experience of package contributors and feedback from the community. However, any other approach like packaging build system files or using generators like cmake_paths are totally valid for "close source" projects and recipes can be still customized for enterprise or any other kind of organization.

[barcharcraz]

Could you expand that FAQ entry to provide more background information on the decision at least?

Also a real problem comes up when you have some packages from conan-center and some from other places, you can't choose to use the conan generated information for just the conan-center packages.

[barcharcraz]

At this moment, we feel there is no need to package additional build system files that can lead to further issues in recipes that are not related to Conan, increasing the effort in maintenance. As you said, some of this config/find files are constantly broken and they could lead users to unpleasant experiences using Conan (for example using cmake_paths). If there is any lack of functionality or information provided by Conan generators, we are open for feedback and we will work on improving them.

the current experience using cmake_paths with conan-center-index packages is simply "it doesn't work" Do you think cmake_paths is inherently an unpleasant experience?

[liarokapisv]

Currently we are using semantic versioning by taking advantage of the ConfigVersion.cmake file feature of find_package. This is important for us since it's an extra verification step to ensure compatibility between libraries from the cmake side. So while not that useful when using conan, it's a pretty useful feature for general usage. Banning the version files means that we can no longer specify the version string in the find_package call, since we won't be able to use that feature with conan.

Technically the generator should be able to create such a version file but it should at least support the same semantic versioning functionality of the CMakePackageConfigHelpers module. This would mostly be basic version checking, I don't see conan being able to supply much more information and even if it could the basic denominator would still be the native cmake functionality so I am not sure how useful that would be compared to just unbanning the version files.

[DavidZemon]

I am still floored with this decision and the fact that Conan.io folks are so determined not to reverse it.

In particular some projects need to be able to build by finding their dependencies without Conan, in addition to with.

This was a key aspect of why I liked Conan so much - that it allowed for all this flexibility. Being able to build the same project both with AND without Conan as the dependency manager was crucial to my team. For many teams out there, new to Conan, it's a big stretch to say "Okay, now everyone MUST start using Conan or you won't be able to build this project." Instead, I was able to start adding Conan recipes to projects and slowly get adoption support from the entire team. Because when you introduce a new tool to a team that's been doing the same thing for 20 years, their first response to anything new is going to be "no." Of course, they all saw the benefit of Conan soon enough, but I wouldn't have been able to show it to them without a slow and gradual introduction, without being able to say "hey Joe, check this out, I've been using it for a couple months now, and all you have to do is follow these short instructions; it's already committed to the project and ready for you to use!"

[jgsogo]

Hi! I would like to recap this issue and try to understand the pain and define what can be done and what should be done. First of all, we need to take into account a couple of things: we keep adding new features to Conan trying to improve the experience and trying to improve the C++ ecosystem, sometimes we need to define where we want to be and we need to suffer a little bit while we find time to develop and release the features.

Regarding CMake module files, we know there is too much variability, sometimes I think that they should be considered bugs and some brave effort should be done to fix them: differences in the casing, libraries providing targets without namespaces,... some of these unexpected-things are in core recipes like OpenSSL or Protobuf and we take them seriously and we really try to make the experience as smooth as possible.

Conan implements components that allow the recipe author to mimic any target name and the libraries/includes/defines provided by each target, components can link explicitly with components of other packages, conan-io/conan#7320 will allow defining a different name for the file and the namespace and finally, conan-io/conan#7261 should provide the last-mile needed for a transparent integration.

Some (all?) of the issues mentioned in this thread can be addressed by the features listed above. And I'm pretty sure that the community will take care of them and recipes in Conan Center will satisfy those demands soon (please, be patient or contribute the changes yourselves).

But, anyway, packaging or not .cmake and .pc files is a decision of ConanCenter, it is not something built-in in Conan.

In my personal experience, this is the right thing to do, using Conan generated files many packages (and more every day) can be cross-compiled to Android/Emscripten/... using one single command (and IMHO, this is amazing), something that would fail if we were using the .cmake files provided by the projects themselves (tools required to run in the build machine like protoc or qt-moc).

---

So please, which are the concrete use-cases you think that are not possible with the current (and incoming) features of Conan?

• @liarokapisv if you have a look to the cmake_find_package_multi generator you will find that we are using the Configs approach. We might add the version to the generated files so consumers can check the version. Nevertheless, I would advise not to make these checks in CMake but in Conan, otherwise, you won't be able to use version_ranges or overrides because of the hardcoded version in the CMakeLists.txt file.

• @barcharcraz cmake_paths alone would never work in that way, not every package provides its own .cmake module file (or it is available in the CMake installation), so in the general scenario you still need Conan to generate these files.

Thanks very much for the feedback

[barcharcraz]

@jgsogo Right, the reason I used cmake_paths was because it means that as far as my project's build system is concerned things were the same weather you were using conan, vcpkg, linux system packages, or any other source for dependencies. The dependencies that didn't provide cmake or pc files I just found the usual way (with a FindModule). All of cmake's find_xxx commands respect the paths that cmake_paths sets, so will find things under conan's install directories just fine.

[jgsogo]

This issue, at least the way it's been discussed is mostly stale:

• The underlying issue should be revisited taking into account new generators and toolchains from Conan v2
• ConanCenter has been removing pc/cmake files and so far people is consuming packages (happily 🤞 )