Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move node configuration to runtime #688

Closed
tiziano88 opened this issue Mar 9, 2020 · 5 comments
Closed

Move node configuration to runtime #688

tiziano88 opened this issue Mar 9, 2020 · 5 comments

Comments

@tiziano88
Copy link
Collaborator

Currently, an Oak Application specifies a list of node "templates" that the application itself may use to create nodes at runtime. Some of the configuration details are embedded in such templates, e.g.:

node_configs {
name: "app"
wasm_config {
module_bytes: "<bytes>"
}
}
node_configs {
name: "log"
log_config {}
}
grpc_port: 8080
initial_node_config_name: "app"
initial_entrypoint_name: "oak_main"

I think it would be simpler and more flexible to instead reduce the application configuration to just a list of wasm modules (we may use the current format for now, and later on consider a more lightweight option, if desired), and the initial entry point fully qualified name, which would cause the Oak Runtime to instantiate that entry point at application startup.

The main node can then create any subsequent node (including logging and gRPC, if necessary) by invoking the node_create ABI function, which would be changed to accept a serialized node config (in protobuf format), instead of the current configuration name. This way the caller node may configure part or all of the configuration for the newly created node (e.g. by choosing a different gRPC port, or passing dynamically generated configuration details).

This should also allow us to support per-instance application configuration files, in which the same "application" would be parametrized with additional values, similar to how non-oak applications are executed by passing flags and / or environment variables. (cc @ipetr0v who is going to create a separate issue to track that).

@daviddrysdale WDYT?

@tiziano88
Copy link
Collaborator Author

For instance, consider the following code:

pub fn init(level: Level, config: &str) -> Result<(), SetLoggerError> {
// Create a channel and pass the read half to a fresh logging Node.
let (write_handle, read_handle) = crate::channel_create().expect("could not create channel");
crate::node_create(config, "oak_main", read_handle).expect("could not create node");
crate::channel_close(read_handle.handle).expect("could not close channel");
log::set_boxed_logger(Box::new(OakChannelLogger {
channel: crate::io::Sender::new(write_handle),
}))?;
log::set_max_level(level.to_level_filter());
Ok(())
}

which would become something like:

let config = NodeConfiguration {
  config_type: ConfigType::LogConfiguration(...),
};
crate::node_create(&config, "oak_main", read_handle).expect("could not create node"); 

i.e. instead of passing a String, it would pass the actual serialized configuration.

Note that wasm modules would still be referenced by name, but at that point the application configuration becomes much more similar to a plain archive (and maybe we could even consider actually ar for that in the future, but that's another discussion).

@tiziano88
Copy link
Collaborator Author

Also vaguely related to #568, in which we are moving towards a more structured input to the logging pseudo-node, whose schema is defined via protobuf.

@daviddrysdale
Copy link
Contributor

It feels a bit like this might be multiplying entities without necessity; if there's a way of passing a blob of configuration data to a new Node at startup, then there are now two different mechanisms by which information can flow (the config blob and the normal messages-on-channels).

It's also possible (although rather inconvenient) to do Node/Application configuration without any changes to the ABI, e.g. by having a convention that the Application's gRPC service definition include a Configure method that's expected to be the first method invoked after startup.

@daviddrysdale daviddrysdale removed their assignment Apr 25, 2020
@tiziano88 tiziano88 self-assigned this Apr 29, 2020
tiziano88 added a commit to tiziano88/oak that referenced this issue Apr 29, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue Apr 29, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue Apr 30, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 1, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 6, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Ref project-oak#688
@tiziano88 tiziano88 added the P1 label May 12, 2020
tiziano88 added a commit to tiziano88/oak that referenced this issue May 12, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 12, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 14, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 18, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 21, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 21, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 21, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 22, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 22, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 22, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 22, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Also remove parts of the logic around the old ApplicationConfiguration
format from the C++ Oak Runtime, under the assumption that it is going
to be deleted soon.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 22, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Also remove parts of the logic around the old ApplicationConfiguration
format from the C++ Oak Runtime, under the assumption that it is going
to be deleted soon.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 26, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Also remove parts of the logic around the old ApplicationConfiguration
format from the C++ Oak Runtime, under the assumption that it is going
to be deleted soon.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 26, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Also remove parts of the logic around the old ApplicationConfiguration
format from the C++ Oak Runtime, under the assumption that it is going
to be deleted soon.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 26, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Also remove parts of the logic around the old ApplicationConfiguration
format from the C++ Oak Runtime, under the assumption that it is going
to be deleted soon.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 26, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Also remove parts of the logic around the old ApplicationConfiguration
format from the C++ Oak Runtime, under the assumption that it is going
to be deleted soon.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 26, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Also remove parts of the logic around the old ApplicationConfiguration
format from the C++ Oak Runtime, under the assumption that it is going
to be deleted soon.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 26, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Also remove parts of the logic around the old ApplicationConfiguration
format from the C++ Oak Runtime, under the assumption that it is going
to be deleted soon.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 26, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Also remove parts of the logic around the old ApplicationConfiguration
format from the C++ Oak Runtime, under the assumption that it is going
to be deleted soon.

Ref project-oak#688
tiziano88 added a commit to tiziano88/oak that referenced this issue May 26, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Also remove parts of the logic around the old ApplicationConfiguration
format from the C++ Oak Runtime, under the assumption that it is going
to be deleted soon.

Ref project-oak#688
tiziano88 added a commit that referenced this issue May 26, 2020
The ApplicationConfiguration object is now more of a directory of
executable wasm modules, and an initial Node configuration.

Subsequent Nodes are created at runtime by passing the serialized
NodeConfiguration protobuf message to `node_create` directly.

This allows removing an extra indirection from application to the config
of individual nodes, in favour of inlining configs in the Wasm code of
the main node itself.

Also remove parts of the logic around the old ApplicationConfiguration
format from the C++ Oak Runtime, under the assumption that it is going
to be deleted soon.

Ref #688
@daviddrysdale
Copy link
Contributor

This is done now, yesno?

@tiziano88
Copy link
Collaborator Author

Yes, thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants