Applications selection at build time #1894

JF002 · 2023-10-23T18:56:53Z

Why ?

InfiniTime is a monolithic firmware : everything (core OS, UI lib, BLE stack, apps) is built into a single firmware image. The current implementation does not allow to easily add new applications and select which apps must be built into the firmware.
This PR tries to improve those points.

Select which apps must be built into the firmware:

You can easily select the applications that will be built into the firmware by adding/removing app from UserAppTypes:

    using UserAppTypes = TypeList<Apps::Alarm,
                                  Apps::HeartRate,
                                  Apps::Paint,
                                  Apps::Metronome,
                                  Apps::Music,
                                  Apps::Navigation,
                                  Apps::Paddle,
                                  Apps::Steps,
                                  Apps::StopWatch,
                                  Apps::Timer,
                                  Apps::Twos
                                  >;
  }

All the apps that are passed as template parameters to UserAppTypes will be built into the firmware. Moreover, the order in this list represents the order of the apps in the application menu.

The menu is automatically sized (number of pages and number of apps per page) to fit all the apps:

Add a new app

As always, start by implementing your app in a class that derives from Pinetime::Applications::Screens::Screen.

Add the .CPP file in the list of source files in the CMake file of the project.

Then, add a new enum value in Pinetime::Application::Apps

Additionally, declare a new AppTraits corresponding to your app. Here is an example from the Metronome app: You simply need to define the name, an icon, and a method Create() that creates the application:

    template <>
    struct AppTraits<Apps::Metronome> {
      static constexpr Apps app = Apps::Metronome;
      static constexpr const char* icon = Screens::Symbols::drum;
      static Screens::Screen* Create(AppControllers& controllers) {
        return new Screens::Metronome(controllers.motorController, *controllers.systemTask);
      };
    };

Finally, add the enum of your application in the template parameters of UserAppTypes

How ?

This is made possible by a bit of template meta programming magic in UserApps.h. Basically, the function CreateAppDescriptions() creates a std::array of AppDescription at build time according to the enum values listed in UserAppTypes. It means that this array is created at build time and then stored in flash memory. It doesn't cost anything at runtime.

Applications are now either "System applications" or "User applications". System applications are apps that are expected to be always built in the firmware (notifications, clock, settings,...). InfiniTime does not work properly if those apps are not present. User applications are applications that are optionally built into the firmware.

App creation

When a new app must be loaded, DisplayApp first checks if the app is a System app, in which case it simply creates it.
If it's not a System app, it looks into the list of User applications and creates it if it's found:

const auto* d = std::find_if(userApps.begin(), userApps.end(), [app](const AppDescription& appDescription) {
        return appDescription.app == app;
      });
      if (d != userApps.end())
        currentScreen.reset(d->create(controllers));

App menu

The App menu, ApplicationList now receives the list of apps as parameters. This list is generated by DisplayApp from the array of AppDescription generated at build time. It builds the menu pages according to this list of Apps.

Results

Developers can now add their application more easily than before, and no data is duplicated.

Developers and fork-maintainers can also select the apps they want to build in their firmware by editing a single line of code. This allows for easier customization of the firmware.

Firmware image sizes:

main branch without any change: 374908B (78.99%)
this branch with all the original apps enabled : 375012B (79.01%)
this branch with none of the User apps enabled : 347476 (73.21%)

What's next ?

We'll be able to merge more PR that add new application in InfiniTime; Not all of them will be built into the official releases of InfiniTime, though (we are still constrained by the available space in flash memory).
We could add a way to easily build applications from external repo (using git submodules, for example). Those applications won't be maintained by the developers/maintainers of InfiniTime but could be easily built into the firmware by developers and fork maintainers.
All watchfaces are currently considered as "System" applications. However, the mecanism introduced in this PR could be extended to watchfaces.
This is one step closer to applications that are dynamically loaded at runtime.

github-actions · 2023-10-23T19:01:11Z

Build size and comparison to main:

Section	Size	Difference
text	377616B	88B
data	940B	0B
bss	63492B	72B

NeroBurner

Just some nitpicks. Not a proper review of the hard bits

src/displayapp/screens/Timer.h

src/drivers/SpiMaster.cpp

src/displayapp/Controllers.h

minacode · 2023-10-30T08:51:02Z

Nice work!
Having tried around in that area a few times in the last year, I came roughly to the same design conclusions, so I obviously like this PR!
This is also way better C++, I guess 😄

One more what's next idea that is in my mind: dependency relations to settings and controllers of apps. In a perfectly modular system, each app would bundle its settings and background tasks/controllers that get registered to the system.

This is one step closer to applications that are dynamically loaded at runtime.

It's true, but I wonder how much of the compile-time magic must be changed to runtime for that. But let's focus on that later.

Now I have to find some time to polish the calculator and test this PR, because if optional apps get merged, it will be a huge step! Thank you for spending the time doing it 🥳

kieranc · 2023-10-30T18:14:44Z

This seems huge, can it be used/extended for watchfaces too?
I would absolutely love to make a build bot (like https://nodemcu-build.com/) which allows the user to select apps/faces and provides them a current build ready to flash.

JF002 · 2023-11-01T19:52:10Z

@minacode

Nice work!
Having tried around in that area a few times in the last year, I came roughly to the same design conclusions, so I obviously like this PR!
This is also way better C++, I guess 😄

Thanks for your feedback! I had this idea in my head for quite some time but... templates are not easy... I honestly can't tell how many hours of CPPCON video I watched before I could write such code :D

One more what's next idea that is in my mind: dependency relations to settings and controllers of apps. In a perfectly modular system, each app would bundle its settings and background tasks/controllers that get registered to the system.

That's probably one of the next steps ! I think we could reduce the coupling between apps and settings/systemtask by having the app "register" to some system events and also by providing them an object that allows them to write their own settings in the file.

It's true, but I wonder how much of the compile-time magic must be changed to runtime for that. But let's focus on that later.

The compile time magic probably won't apply for dynamic apps, but having all the apps behave the same way (like constructing them using the same Create() method) should help also for dynamic apps.

Now I have to find some time to polish the calculator and test this PR, because if optional apps get merged, it will be a huge step! Thank you for spending the time doing it 🥳

Thank, and thank you for your contributions!

@kieranc

This seems huge, can it be used/extended for watchfaces too?
I would absolutely love to make a build bot (like https://nodemcu-build.com/) which allows the user to select apps/faces and provides them a current build ready to flash.

I already have a couple of ideas to used similar implementation for the watchfaces, yes. Watchfaces are probably the apps that need the most flash space!
And a tool that allows to easily build a customized firmware would definitely be awesome!

JF002 · 2023-11-11T15:31:31Z

The CI check that fails is caused by InfiniSim that builds in C++17 while we now need C++20. I created a PR to fix this.

A list of "user applications" is built at compile time. It contains all the info needed to create the application at runtime (ptr to a create() function) and to display the app in the application menu. All applications declare a TypeTrait with these information. When a new app must be loaded, DisplayApp first check if this app is a System app (in which case it creates it like it did before). If it's not a System app, it looks for the app in the list of User applications and creates it if it found it. Those changes allow to more easily add new app and to select which app must be built into the firmware. Switch to C++20 (and fix a few issues in SpiMaster.cpp and Watchdog.cpp.

Fix DisplayAppRecovery so it builds with -std=c++20.

Update documentation about the apps (Apps.md) : fix obsolete information, add doc about user/system apps and update the part about the implementation of a new app.

NeroBurner requested changes Oct 23, 2023

View reviewed changes

src/displayapp/screens/Timer.h Outdated Show resolved Hide resolved

src/drivers/SpiMaster.cpp Show resolved Hide resolved

src/drivers/SpiMaster.cpp Show resolved Hide resolved

src/displayapp/Controllers.h Outdated Show resolved Hide resolved

JF002 mentioned this pull request Nov 11, 2023

Enable C++20 in CMake InfiniTimeOrg/InfiniSim#129

Merged

FintasticMan added the new feature This thread is about a new feature label Nov 16, 2023

JF002 added this to the 1.14.0 milestone Nov 19, 2023

JF002 added 5 commits November 19, 2023 21:06

Application selection at build time

3886ebf

Fix DisplayAppRecovery so it builds with -std=c++20.

Add missing new line at end of file.

27df3ff

Fix code formatting

a868322

Application selection at build time : update documentation

67fa213

Update documentation about the apps (Apps.md) : fix obsolete information, add doc about user/system apps and update the part about the implementation of a new app.

JF002 force-pushed the app-selection-build-time branch from 5641da9 to 67fa213 Compare November 19, 2023 20:09

JF002 merged commit f3d4f04 into main Nov 19, 2023
7 checks passed

This was referenced Dec 3, 2023

Gallery app #1384

Open

New dice-rolling app: InfiniDice! #1326

Merged

This was referenced Dec 10, 2023

App controller #1840

Closed

Hardcoded apps #1571

Closed

Optional apps #1408

Closed

FintasticMan deleted the app-selection-build-time branch December 10, 2023 11:57

JF002 mentioned this pull request Dec 10, 2023

Watch face selection at build time #1926

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Applications selection at build time #1894

Applications selection at build time #1894

JF002 commented Oct 23, 2023

github-actions bot commented Oct 23, 2023 •

edited

Loading

NeroBurner left a comment

minacode commented Oct 30, 2023

kieranc commented Oct 30, 2023

JF002 commented Nov 1, 2023

JF002 commented Nov 11, 2023 •

edited

Loading

Applications selection at build time #1894

Applications selection at build time #1894

Conversation

JF002 commented Oct 23, 2023

Why ?

Select which apps must be built into the firmware:

Add a new app

How ?

App creation

App menu

Results

What's next ?

github-actions bot commented Oct 23, 2023 • edited Loading

NeroBurner left a comment

Choose a reason for hiding this comment

minacode commented Oct 30, 2023

kieranc commented Oct 30, 2023

JF002 commented Nov 1, 2023

JF002 commented Nov 11, 2023 • edited Loading

github-actions bot commented Oct 23, 2023 •

edited

Loading

JF002 commented Nov 11, 2023 •

edited

Loading