[new] Compatible packages regarding C++ standard

[jgsogo]

This example tries to show how to handle validate and compatible_packages features together.

This is a tiny scenario, but typically dependency graphs will be much bigger and information about libraries won't be as accessible as here.

Disclaimer.- This is a very opinionated dramatization using just two recipes, but it is based on a real story where I played the main character 😅

Alternative 1: recipes 'app' and 'library'

'library' recipe

In order to build your app you know you need library/1.0 and it is available as a Conan package (maybe in ConanCenter, maybe in your own servers) and you need to build it from sources:

conan export library/conanfile.py library/1.0@  # We need this here, it is not in any server
conan install library/1.0@ --build=library

If your compiler doesn't activate C++11 by default, we get a nice error:

ERROR: library/1.0: Invalid configuration: Current cppstd (gnu98) is lower than the required C++ standard (11).

and with a little knowledge of Conan we know how to fix it:

conan install library/1.0@ --build=library -s compiler.cppstd=11
...
Installing package: library/1.0
Requirements
    library/1.0 from local cache - Cache
Packages
    library/1.0:INVALID - Invalid

Installing (downloading, building) binaries...
ERROR: There are invalid packages (packages that cannot exist for this configuration):
library/1.0: Invalid ID: Current cppstd (11) is lower than the required C++ standard (14).

Opps! (We need a better error message here) Again a nice message that tells us we need C++14 to build the library. Easy to fix!

conan install library/1.0@ --build=library -s compiler.cppstd=14

We have our binary available now.

---

'app' recipe

Now we want to use that library in our app. We forgot about the previous steps, maybe our app is upgrading from library/0.1 to library/1.0, maybe different teams take care of the library and the application:

conan export app/conanfile.py app/1.0@  # We need this here, it is not in any server
conan install app/1.0@ --build=app
...
ERROR: library/1.0: Invalid configuration: Current cppstd (gnu98) is lower than the required C++ standard (11).

Conan shows a nice message telling us that using library/1.0 requires at least C++11. app requires plain C++ but the new requirement needs C++11 to be used (API uses C++11 features):

conan install app/1.0@ --build=app -s compiler.cppstd=11
...
library/1.0: Main binary package 'INVALID' missing. Using compatible package 'ccfcfe63664926af39fa1a9f97d2f0dcf252dbf6'
...
app/1.0: Created

It's been easy, we have our application, all the binaries have been generated with the proper profile and can be identified. The binary we are using for library/1.0 belongs to compiler.cppstd=14 and the binary for app/1.0 is being built using a profile with compiler.cppstd=11. This can be important metadata associated with the binaries.

More importantly, this is quite straightforward from the POV of the user.

Alternative 2: recipes 'app' and 'hidden'

'hidden' recipe

The recipe for hidden package contains the typical lines CMAKE_CXX_STANDARD in the CMakeLists.txt file of the project, they are hidden to the consumer of the library. So we, Conan users, run the commands and we are able to compile hidden at the first shot:

conan export hidden/conanfile.py hidden/1.0@  # We need this here, it is not in any server
conan install hidden/1.0@ --build=hidden

This works because our compiler support C++14, otherwise, we would get the typical CMake error at build time. We know the error (not a template metaprogramming one, LOL).

Here we are already getting into troubles, we have generated a binary that uses C++14 but it is associated with the default profile that is using (at least my apple-clang) C++98... but other calls like:

conan install hidden/1.0@ --build=hidden -s compiler.cppstd=11

will succeed too, they will use C++14 even though I'm requesting C++11. This can be a problem from the ABI perspective in some scenarios.

Metadata associated with these binaries will be wrong.

---

'app' recipe (using hidden requirement)

(Let's start only with the default binary again)

conan remove hidden/1.0@
conan export hidden/conanfile.py hidden/1.0@
conan install hidden/1.0@ --build=hidden

We now go to the other team of developers, they retrieve the binary for hidden/1.0, the default profile works and they know the default for their compiler is C++98, their app doesn't use modern features, they can integrate this library. Let's go!

conan export app/conanfile.py app/1.0@ 
conan install app/1.0@ --build=app -o app:use_hidden=True
...

ERRORS!

I get build errors! As an app developer I'm confused, this profile was suitable for the hidden/1.0@ package, but now it fails to build with of C++11 errors... is it my error? Am I missing something?

At least this is a small recipe, I can have a look at the errors, at the sources and at the recipe and I realize that I need C++11 because the API contains some auto declarations. Good, let's build the lib:

conan install app/1.0@ --build=app -o app:use_hidden=True -s compiler.cppstd=11
...

app/1.0: Forced build from source
Installing package: app/1.0
Requirements
    app/1.0 from local cache - Cache
    hidden/1.0 from local cache - Cache
Packages
    app/1.0:f521448e6e11249788f58876ccd50841ecd1a160 - Build
    hidden/1.0:INVALID - Invalid

Installing (downloading, building) binaries...
ERROR: There are invalid packages (packages that cannot exist for this configuration):
hidden/1.0: Invalid ID: Current cppstd (11) is lower than the required C++ standard (14).

WTF! I had a binary, now I need to build from sources the package hidden/1.0 because the new profile with compiler.cppstd=11 computes a new package ID... even more, it tells me that I need C++14, not only C++11. Ok, at least it is easy to fix:

conan install app/1.0@ --build=app -o app:use_hidden=True -s compiler.cppstd=14
...

ERROR: Missing prebuilt package for 'hidden/1.0'
Try to build from sources with '--build=hidden'
Use 'conan search <reference> --table table.html'
Or read 'http://docs.conan.io/en/latest/faq/troubleshooting.html#error-missing-prebuilt-package'

Oh! Of course, let's use --build=missing:

conan install app/1.0@ --build=missing -o app:use_hidden=True -s compiler.cppstd=14
...
OK!

...but I built my app using C++14, and I wanted it just with C++11, one more compilation:

conan install app/1.0@ --build=app -o app:use_hidden=True -s compiler.cppstd=11

And I'm finally there.

---

---

---

Why do I really prefer the first approach, where the C++ standard is managed by Conan and it doesn't appear in CMakeLists.txt at all:

• Information is closer to the user, information is available when building the graph, not via an error from the build-system. This can be very important if the graph becomes big and complex, Conan can report the error after building the graph, no need to start building things to realize we need to change the compiler.cppstd value and start building again.

• Metadata Binaries can be associated with the profile used to create them.

  • If the profile doesn't declare compiler.cppstd we know the binary uses whatever the default is for that compiler.
  • The C++ standard from the profile is the one used in the binary, it is not overridden inside.

  This can be very important when using experimental features from future C++ standards that are not stable yet.

• Better experience regarding ConanCenter packages. The binary available for hidden/1.0@ is not usable at all, we need a profile with compiler.cppstd=11 to be able to consume that package and this extra item in the profile modifies the packageID and it forces us to build from sources!!

Advantages of the second approach:

1. we don't need to patch every CMakeLists.txt file to remove the set(CXX_STANDARD XX) declaration because I know it is everywhere.
2. outside Conan, you don't need to pass the C++ standard in the command line in order to build the library using plain CMake commands.

IMHO, libraries will keep doing (2), but our recipes in ConanCenter (as soon as we start iterating configuration with explicit compiler.cppstd values) should remove those lines from CMakeLists.txt files.