Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add WCGI to wasmer run #3613

Closed
wants to merge 3 commits into from
Closed

Add WCGI to wasmer run #3613

wants to merge 3 commits into from

Conversation

Michael-F-Bryan
Copy link
Contributor

@Michael-F-Bryan Michael-F-Bryan commented Feb 23, 2023

Do Not Merge: This PR branched off #3599 while we were waiting for #3422 to be merged. Once #3599 is merged, we can rebase onto master and get ready to merge this PR.

This adds the WCGI runner introduced in #3599 to the wasmer CLI.

@Michael-F-Bryan Michael-F-Bryan changed the base branch from master to wasix February 23, 2023 15:01
Base automatically changed from wasix to master February 23, 2023 19:57
/// file's default entrypoint.
///
/// This will infer the [`CgiDialect`] from the WEBC file's metadata
pub fn build_webc(self, webc: impl Into<Bytes>) -> Result<Runner, Error> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think build_webc is a good name. Probably use_webc or from_webc are way better names

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, you may want to provide a command to execute, rather than infer it. And get the webc directly instead of the bytes.

That way there are two steps:

  1. parsing the webc
  2. detecting the command to run and what kind it is
  3. Run the runner

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I deliberately named it that way so we can follow the Runner::builder().foo().bar().build() pattern. Using "use" or "from" would be very confusing.

In this case, we have multiple ways of constructing the final runner (e.g. a WEBC file, a WebAssembly binary, or the filename for something on disk), so there are multiple build_xxx() methods.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the point that the build_webc actually returns a Runner (which I didn't realize before), so I kind of agree on that naming if we want to return a runner.

However, I think the webc shall be parsed beforehand and then chose the command to run.
For example, let's say you have a package with two commands:

  • A (WASI command)
  • B (WCGI command)

You don't know which runner you are going to run, until you have parsed the webc package. And at that point you already have the config and such. So passing a stream of bytes to parse again the webc seems wrong

let rt = wasmer_wasi::runtime::task_manager::tokio::TokioTaskManager::default();

let mut builder = Runner::builder()
.map_dirs(mapped_dirs)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd put here: with_wasi_config(CONFIG). Because that way it can be reused in the future with the WASI runner

Comment on lines +3 to +30
use crate::{
module_loader::{ModuleLoader, ModuleLoaderContext},
Error,
};

use super::LoadedModule;

pub(crate) struct Cached<L> {
loader: L,
invalidated: Box<dyn Fn() -> bool + Send + Sync>,
cached: Mutex<Option<LoadedModule>>,
}

impl<L> Cached<L> {
pub(crate) fn new(loader: L, invalidated: impl Fn() -> bool + Send + Sync + 'static) -> Self {
Self {
loader,
invalidated: Box::new(invalidated),
cached: Mutex::new(None),
}
}
}

#[async_trait::async_trait]
impl<L: ModuleLoader> ModuleLoader for Cached<L> {
async fn load(&self, ctx: ModuleLoaderContext<'_>) -> Result<LoadedModule, Error> {
let mut cached = self.cached.lock().await;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this exactly? We already have the wasmer-cache crate that should handle this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regardless of whether we use a Mutex<Option<LoadedModule>> or a wasmer_cache::FileSystemCache, we still need to have some sort of struct which can be used as a ModuleLoader.

I used a mutex-based solution just because it's trivial to write and was good enough to get things implemented.

Copy link
Member

@syrusakbary syrusakbary Feb 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the wasmer_cache::FileSystemCache is already provided by the CLI, right?
I don't think the caching mechanism should be defined in the runner (but plugged into it).

I think we might be doing an early optimization by having the wcgi-runner standalone instead of being first integrated on the cli first (which I think might lead to simpler implementation)

@Michael-F-Bryan
Copy link
Contributor Author

After a bit of discussion today on Slack, I'm going to close this and move all changes to #3599. We need to redesign how the WCGI runner works because the current design doesn't match how we want wasmer run to work.

@Michael-F-Bryan Michael-F-Bryan deleted the wcgi-in-wasmer-run branch March 6, 2023 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants