Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rust App Config Serializer #740

Closed
ipetr0v opened this issue Mar 20, 2020 · 9 comments · Fixed by #1219
Closed

Rust App Config Serializer #740

ipetr0v opened this issue Mar 20, 2020 · 9 comments · Fixed by #1219
Assignees

Comments

@ipetr0v
Copy link
Contributor

ipetr0v commented Mar 20, 2020

We need to add a Rust version of the App Config Serializer for creating binary configuration files for Oak applications.
C++ version of the Serializer was implemented in #459 as a part of #412.

Blocks #723

@ipetr0v ipetr0v self-assigned this Mar 20, 2020
@ipetr0v ipetr0v changed the title Rust app config serializer Rust App Config Serializer Mar 20, 2020
@ipetr0v
Copy link
Contributor Author

ipetr0v commented Mar 20, 2020

Currently I have not found that Prost allows to read textproto files and to serialize protobuf structures into files, compared to the C++ version of ProtoBuf:
https://github.com/protocolbuffers/protobuf/blob/70fc0f0275389a99b0c654778ef937f904921be0/src/google/protobuf/text_format.h#L448-L452
https://github.com/protocolbuffers/protobuf/blob/70fc0f0275389a99b0c654778ef937f904921be0/src/google/protobuf/message_lite.cc#L466-L474

I will continue searching but another possibility is to use non-protobuf (non-textproto) formats for textual config files, use them to create Rust structures and then serialize everything with Serde (Rust serialization).

cc @tiziano88

@daviddrysdale
Copy link
Contributor

Why does this block #723? There's nothing stopping us having a C++ tool that generates a binary serialized protobuf file that the Rust runtime can then read (language interoperability being the point of protobuf, after all).

@tiziano88
Copy link
Collaborator

I agree this does not block anything, the current C++ serializer should work perfectly for both the C++ and Rust runtime. We may want to have different scripts / Bazel rules to make it easier to invoke perhaps, in the short term.

@ipetr0v
Copy link
Contributor Author

ipetr0v commented Mar 23, 2020

Deleted "Blocks"

@tiziano88
Copy link
Collaborator

Probably worth starting work on this, since we know it will necessary for #1183 , @ipetr0v WDYT?

@ipetr0v
Copy link
Contributor Author

ipetr0v commented Jun 30, 2020

We already have TOML files with example info:

name = "abitest"
[modules]
module_0 = { Cargo = { cargo_manifest = "examples/abitest/module_0/rust/Cargo.toml" } }
module_1 = { Cargo = { cargo_manifest = "examples/abitest/module_1/rust/Cargo.toml" } }
[clients]
cpp = { Bazel = { bazel_target = "//examples/abitest/client:client" }, additional_args = [
"--test_exclude=Storage",
"--cert_chain=../../../../../../../../examples/certs/local/local.pem",
"--private_key=../../../../../../../../examples/certs/local/local.key",
] }

Should this be a separate file with a list of Wasm files, or it's better to keep everything together?
I personally prefer a separate file, because in this case it would be possible to just have a file with URLs and no source code.

WDYT?

@tiziano88
Copy link
Collaborator

The example manifests are and should stay separate; they are used to organize how examples are built and run, including clients and other example-specific stuff.

We need an application config manifest that is essentially a replacement for the textproto application definition, and also allow specifying the mapping from names to files (which is currently provided via flags to the app serializer).

The example TOML file then should probably be modified so that it points to the example manifest file, which runner or whatever other mechanism would then pass to the app serializer, after having actually built the local Wasm modules (the app serializer does not build the modules directly)

This should be the first version.

Next, separately, we should add support for specifying a URL as the source for the modules.

@ipetr0v
Copy link
Contributor Author

ipetr0v commented Jul 1, 2020

Should serialized files be put into out (#719) directory, or it's better to create something inside target?

@tiziano88
Copy link
Collaborator

I think we should create an out dir as part of this. target is specific to Rust, I would leave it alone.

ipetr0v added a commit that referenced this issue Jul 3, 2020
This change:
- Adds a Rust version of application configuration serializer
- Updates examples to use Rust serializer

Fixes #740
Ref #719
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants