Refactor parameter specification

[ipetr0v]

After talking to @tiziano88 we decided that it's worth thinking about refactoring a way parameters for Oak applications are specified.

Currently we only have parameters inside an application configuration config.textproto file that is compiled in a config.bin file using Protobuf:

oak/examples/hello_world/config/config.textproto

Lines 1 to 13 in c4865b2

	node_configs {
	name: "app"
	wasm_config {
	module_bytes: "<bytes>"
	}
	}
	node_configs {
	name: "log"
	log_config {}
	}
	grpc_port: 8080
	initial_node_config_name: "app"
	initial_entrypoint_name: "oak_main"

And any change requires a recompilation, so it's not very convenient.

Basically, there are two types of possible parameters for an Oak Application:

• Secure parameters:
  • Parameters that should change the hash value of a Wasm module.
  • An example of a secure parameter could be the SAMPLE_THRESHOLD from the Aggregator example. This SAMPLE_THRESHOLD specifies the amount of samples that should be collected from users before revealing the aggregated value.
• Insecure parameters:
  • Parameters that could be freely changed without worrying about affecting the user trust.
  • An example of an insecure parameter could be an external IP address to connect to (as long as Oak Application checks its TLS certificates).

In the case of insecure parameters it may feel strange to always recompile a config.bin whenever developers need to change something, for example when a backend service changed an IP address.

So I think it makes sense to have secure parameters hardcoded (for now), since all changes will definitely effect the Wasm module hash value. And for insecure parameters the most obvious solution would be to have an additional argument{s} for oak_runner:

./asylo_oak_runner --application=APPLICATION [-- MODULE_ARGUMENTS]

Possible ways of implementing include:

• Pass arguments in the oak_main function, so they could be parsed inside the module, and developers could use whatever configuration they want (JSON, YAML etc.).
• Have an additional config pseudo-node with a corresponding fn get(name: &String) -> String function.

cc @project-oak/core

P.S.
Also I think that grpc_port could be a command-line argument too, so it would be more convenient to deploy Oak applications (as long as we will have only one gRPC server pseudo-node in Oak applications).

[tiziano88]

@ipetr0v let's start with insecure parameters only for now, since we will need them to configure things like gRPC endpoints, logging, authentication, etc.

To take a concrete example, let's imagine we need to provide a username and password to use to connect to a non-Oak server (e.g. for storage).

#696 covers figuring out how to get those secrets provisioned for a specific VM instance, but in this issue let's focus on what to do once we have those secrets available.

I am thinking we could have a single parameter --config=filename added to oak_loader, and the contents of that file would be available to read from the entry point of the "initial" node of an application. This way each application may decide what format to use (e.g. JSON, YAML, TOML, etc.), since it would be parsing that from the node code itself. If it helps, this is conceptually similar to piping the contents of this file as "stdin" to a process.

@ipetr0v @daviddrysdale WDYT?

[ipetr0v]

I was also thinking of doing something similar to Kubernetes, where we will have a YAML file with different nodes and their parameters:

nodes:
    - name: "aggregator"
      parameters:
        - backend_address: 127.0.0.1
    - name: "load_balancer"
      parameters:
        - port: 8080
entrypoint:
    node: "aggregator"
    function: "oak_main"

And each parameter will be passed only to a single node specified in the config (e.g. a dictionary {'port': 8080} will go to the load_balancer's main function).

But since YAML is a textual representation and we want to change it at any point in time without recompiling, we still need to pass our module to the Oak Loader. And there are 3 options:

• Pass a list of Wasm files:

  ./oak_loader --config=config.yaml --modules=./path/module1.wasm,./path/module2.wasm

• Pass a Zip archive with Wasm files:

  ./oak_loader --config=config.yaml --modules=./path/modules.zip

• [In the long run] And similar to Kubernetes we can have a Node Registry, and each module specification will need to specify a URL (just like Docker Registry):

    - name: "aggregator"
      url: "oak.google.com/aggregator"
      parameters:
        - backend_address: 127.0.0.1

[tiziano88]

In your example, how would the "aggregator" node instance (which is a generic Wasm node) get access to its parameters? That's the part I am trying to figure out :) the rest, as you say, can be done in a number of different ways

[ipetr0v]

IIUC each node has its main function:

oak/examples/abitest/module_0/rust/src/lib.rs

Lines 94 to 102 in d22711f

	#[no_mangle]
	pub extern "C" fn frontend_oak_main(in_handle: u64) {
	let _ = std::panic::catch_unwind(|| {
	oak::set_panic_hook();
	let node = FrontendNode::new();
	let dispatcher = Dispatcher::new(node);
	oak::run_event_loop(dispatcher, in_handle);
	});
	}

oak/examples/abitest/module_1/rust/src/lib.rs

Lines 27 to 31 in d22711f

	#[no_mangle]
	pub extern "C" fn backend_oak_main(handle: u64) {
	let _ = std::panic::catch_unwind(|| {
	oak::set_panic_hook();

So parameters for each module can be passed alongside the handle as a:

#[no_mangle]
pub extern "C" fn oak_main(handle: u64, parameters: HashMap<String, String>) {}

or

pub extern "C" fn oak_main(handle: u64, node_sub_config: String) {
  let parameters = serde_yaml::from_str(&node_sub_config)?;
}

[tiziano88]

Conceptually that make sense, but note that neither HashMap nor String are valid Wasm types though, hence my question.

[tiziano88]

@ipetr0v what's the current plan then? An additional channel over which the configuration can be read by the node at startup? What format should it have, and how should it be serialized?

[ipetr0v]

I still would prefer different parameters to main functions in each module, but since currently there is no way to pass them to a Wasm function, I think an additional pseudo-node is a nice solution.
But it may be confusing for developers that will start implementing Oak applications.

[tiziano88]

FYI I just came across this https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/ , in case it helps.

[ipetr0v]

Should this issue be closed?

cc @tiziano88