Use development version of PlatformIO

[ivankravets]

We would be thankful if you use PlatformIO Core Dev for CI builds. Marlin is a very important project for the open-source community. We don't want to break it with new changes. So, please report us if PIO Core fails due to our issue.

Thanks in advance!

P.S: PIO Core Dev is very stable. We use it for all our projects in CI.

[thinkyhead]

So, please report us if PIO Core fails due to our issue.

Thanks! We'll keep you posted.

[thinkyhead]

@ivankravets — Looks like we already have a breakage in atmelsam / adafruit_grandcentral_m4:

 #include "SdFat.h"
          ^~~~~~~~~
compilation terminated.
*** [.pio/build/SAMD51_grandcentral_m4/lib545/Adafruit_SPIFlash/Adafruit_FlashCache.cpp.o] Error 1
In file included from .pio/libdeps/SAMD51_grandcentral_m4/Adafruit_SPIFlash/src/Adafruit_SPIFlash.cpp:1:0:
.pio/libdeps/SAMD51_grandcentral_m4/Adafruit_SPIFlash/src/Adafruit_SPIFlash.h:34:10: fatal error: SdFat.h: No such file or directory

No problem compiling with the latest PIO release version.

[thinkyhead]

@ivankravets —
Oops, nevermind. I see Adafruit_SPIFlash got a new release 12 hours ago. I'll specify the previous version.

[rhapsodyv]

@ivankravets —
Oops, nevermind. I see Adafruit_SPIFlash got a new release 12 hours ago. I'll specify the previous version.

We could lock all the libs to a specific version, at least in a time frame right before a release. After the release, the libs could be unlocked again. This way, it could help avoid breaks and yet keep everything cutting edge.

[thinkyhead]

Actually, there does seem to be a dependency problem. Specifying an earlier version of Adafruit_SPIFlash doesn't make any difference. The SdFat.h dependency is in older versions too; it just isn't getting downloaded. The specific version it wants is id=6609 "SdFat - Adafruit Fork." It does properly sense the dependency and download it when I build with PIO release. But I had to explicitly add it before PIO devel would download and build it.

[ivankravets]

We have already deployed the new registry (currently, only backend part). It will allow declaring dependencies based on the package owner and library name. This pair will be unique. Also, all packages in the new registry are Semantic Versioning compliant. We will notify the developer to avoid using direct IDs of packages because they can change. There will be support in the upcoming core where people can use other registries (own-hosted).

So, could we switch Marling from tar.gz URL to the registry packages and use SemVer declaration? For example, allow updating ONLY patched version. For example, ~1.2.3, that means any version in 1.2.X.

If some libraries are not published into the registry, now everyone can do this. We updated our docs, see https://docs.platformio.org/en/latest/librarymanager/creating.html

[thinkyhead]

Thanks for the extra information. I read the new docs and took note that we added a library.json manifest to U8glib-HAL so it appears in the registry. We might create some other custom forks depending on how responsive the main projects are.

So, could we switch Marling from tar.gz URL to the registry packages and use SemVer declaration? For example, allow updating ONLY patched version. For example, ~1.2.3, that means any version in 1.2.X.

The library name / semantic versioning style has been applied to lib_deps where it seemed appropriate, but some of our lib_deps are still fixed to specific versions, maybe due to some paranoia…. For (today's) 2.0.6 release I'm not feeling entirely secure in switching all @1.2.3 over to ~1.2.3. 😨

If some libraries are not published into the registry, now everyone can do this

I can see a few things I would like to add with platformio package publish, but there are caveats.

There is still a small compendium of customized libraries hosted by miscellaneous contributor forks. So for those we are still using .zip (not .tar.gz) links. I don't want to fill the PIO library registry with miscellaneous tweaked forks of "Adafruit Neopixel" if it can be avoided. For example, this is currently required for LPC1768…

https://github.com/p3p/Adafruit_NeoPixel/archive/1.5.0.zip

…but hopefully that will not be the case for very long. It's not known when this will change.

And, this is what we need for LVGL displays…

https://github.com/makerbase-mks/MKS-LittlevGL/archive/master.zip

Should we add it on behalf of makerbase-mks or is it better to let them maintain it?

---

At the moment at least the compilation is working very well, and I think it will be easy to hotfix all the library references as they get published and as we gain confidence in minor updated versions. So, I'm going to proceed to tag what we have with a 2.0.6 version number, and any helpful patches for platformio.ini or the build process that come in thereafter will be published pretty quickly. We usually have to do a few quick patches anyway as the code gets more testing.

[ivankravets]

Maybe, you are right! So, just use the configuration which works for you. In the new registry, you can actually submit packages into your own scope MarlinFirmware/custom-lib. However, we have not released yet PIO Core 4.4.

Please keep the current configuration which works well and let discuss this later ;)