Cura 10475 engineplugin

[jellespijker]

plugin system for CuraEngine

gRPC definitions:
CuraEngine_grpc_defintions: https://github.com/Ultimaker/CuraEngine_grpc_defintions
example plugins:
CuraEngine_example_plugin_cpp: https://github.com/Ultimaker/CuraEngine_example_plugin_cpp
CuraEngine_example_plugin_rust https://github.com/Ultimaker/CuraEngine_example_plugin_rust
CuraEngine_example_plugin_python: https://github.com/Ultimaker/CuraEngine_example_plugin_python
PullRequests on existing repo's:
Ultimaker/CuraEngine#1878
#15563
Ultimaker/libArcus#150
Ultimaker/pyArcus#5

The Engine Plugin System uses Protobuf https://protobuf.dev/ and gRPC https://grpc.io/

Suggested Architecture

Cura container

Cura is responsible for starting the engine plugin service before the engine. In the local variant, this is done with a bash command adding the appropriate arguments (such as IP localhost and port number) these arguments are send along to the engine backend plugin which is responsible for starting engine, providing the settings, mesh and engine plugin (IP and port) data.

Engine plugin container

The engine plugin container can be written in any language, provided that it can handle Protobuf messages. The engine plugin service is started and will listen to any message on the specified socket. It should be able to respond to the following message types:

Identity: This should return at the very least the version and maybe in the future possible an authentication response, to ensure that we hook in to a valid plugin.

Process(data, args…): This is a request to process the data, with some additional arguments and return the modified data in the agreed upon format, if an error occurs it should respond with an error message instead.

Shutdown: This should stop listening and exit the program

CuraEngine container

CuraEngine is to be extended with a PluginRegistry, which will take care of the book keeping, coupling a slot, (which is a hook, defined by an certain operation, version range and call signature) with a PluginProxy (which provides an uniform API for the remote plugin service and acts like a regular C++ function object, with a default fallback behaviour if no valid plugin services is running.

Since we now have to take into account that the update cycle for engine plugins might differ from that of Cura and therefor CuraEngine, and that a user might still run an older version of a engine plugin, we also need to validate if a plugin is compatible; This is done with the PluginValidator which is a type that specifies the semver range. This validator can be extended in the future with additional checks, such as an authentication scheme, which would allows use to only run UltiMaker approved plugins (@remco Burema will look into this further)

Because the data transfer between a plugin and the engine is defined as a “generic” Protobuf message the engine also needs a way to convert that Protobuf type to CuraEngine native types. This is done with a Converter which maps the protobuf type to the native types (e.q. Polygons etc.). The converter throws a runtime error when it receives an error response from the plugin.

A slot is defined by a PluginValidator, a specialisation of the Converter<proto::Request, proto::Response> and a SlotID (e.q. POSTPROCESS, SIMPLIFY, etc.).

Proto definitions container

The common types between a plugin and the engine are the Protobuf messages. These messages should live as a single source of truth in their own repository (with a very loose license e.q. MIT).

The Protobuf definitions are:

SlotID (e.q. POSTPROCESS, SIMPLIFY, etc.), these describe the specific functionality of the plugin

Identity_Request request detailed information from a plugin

Identity_Response provides the version of the plugin, can be extended in the future with additional verification methods

Error_Response provides an error message if the plugin encounters an error

Each type of slot should have a corresponding Request and Response message in the definition file, e.q. Simplify_Request , Simplify_Response

Sequence diagram

To use the plugins from the front-end for now use environmental variables as below, while running this Cura branch:

• SIMPLIFY_ENABLE: default disabled, setting this to any value will enable the plugin
• SIMPLIFY_ADDRESS: defaults to localhost
• SIMPLIFY_PORT: defaults to 33700
• POSTPROCESS_ENABLE: default disabled, setting this to any value will enable the plugin
• POSTPROCESS_ADDRESS: defaults to localhost
• POSTPROCESS_PORT: defaults to 33701