Squashed commit of the following:

commit e648284 Author: [email protected] <[email protected]> Date: Tue Dec 15 10:56:36 2020 -0600 better build logging commit a271ffc Author: [email protected] <[email protected]> Date: Mon Dec 14 11:45:41 2020 -0600 add script-format field, remove Option<T> from builder config commit 193ba02 Author: [email protected] <[email protected]> Date: Fri Dec 11 13:53:30 2020 -0600 coalesce bundler TargetType under javascript, using config to distinguish commit db9c356 Author: [email protected] <[email protected]> Date: Fri Dec 11 15:09:39 2020 -0600 rename bundle_config to builder_config, and related names commit e5780a0 Author: [email protected] <[email protected]> Date: Fri Dec 11 13:22:14 2020 -0600 remove check linter commit 1c3b588 Merge: 2a3b5b3 cbbed17 Author: [email protected] <[email protected]> Date: Fri Dec 11 13:02:33 2020 -0600 Merge remote-tracking branch 'caass/master' into malonso/modules-do commit cbbed17 Author: Cassandra Fridkin <[email protected]> Date: Tue Nov 3 00:47:05 2020 -0500 Rewrite gzip buffer-building strategy to maybe be friendlier to rayon commit eeb38cf Author: Cassandra Fridkin <[email protected]> Date: Tue Nov 3 00:20:53 2020 -0500 forgot to run clippy again lol commit 6cba17b Merge: b000958 04584e5 Author: Cassandra Fridkin <[email protected]> Date: Tue Nov 3 00:10:03 2020 -0500 Merge branch 'cloudflare' into master commit b000958 Author: Cassandra Fridkin <[email protected]> Date: Tue Nov 3 00:07:17 2020 -0500 Apply suggestions from code review - Check total bundle size, not individual files - Replace hyphens with underscores in wrangler.toml - Rename MAX_FILE_SIZE to MAX_BUNDLE_SIZE - Duplicate comment about swc_common::SourceMap vs sourcemap::SourceMap - rename entry & entry_file to module & module_path commit 000a79c Author: Cassandra Fridkin <[email protected]> Date: Thu Oct 29 13:42:40 2020 -0400 Fix clippy warnings commit d84f01b Author: Cassandra Fridkin <[email protected]> Date: Thu Oct 29 13:19:39 2020 -0400 Add some documentation commit 90c3489 Merge: 448cf1e 11fe7c1 Author: Cassandra Fridkin <[email protected]> Date: Thu Oct 29 12:45:07 2020 -0400 Merge branch 'cloudflare' into master commit 448cf1e Author: Cassandra Fridkin <[email protected]> Date: Thu Oct 29 12:39:34 2020 -0400 Rewrite our validation strategy one last time Everything compiles now, which is nice. We just need to actually implement Lintable for JavaScript and we should have version one ready to go. commit ea1880e Author: Cassandra Fridkin <[email protected]> Date: Sat Oct 17 22:52:11 2020 -0400 Things might be broken-er in this commit But we're going for the restructure. again. round four. this time, with macros. because i'm sick of defining Lintable for all of these structs commit 031329c Author: Cassandra Fridkin <[email protected]> Date: Sun Oct 4 00:00:10 2020 -0400 Add another monstrosity of a switch statement for expressions I guess...now I just have to...implement Lintable for all these different types of expressions...yippee... commit b3bfde3 Author: Cassandra Fridkin <[email protected]> Date: Sat Oct 3 16:09:59 2020 -0400 Clean up some TODOs because we did them commit 0ace2ac Author: Cassandra Fridkin <[email protected]> Date: Sat Oct 3 11:23:45 2020 -0400 Define Lintable, Parseable, and Validate traits I keep making tons of changes without committing but I think that it seems bigger than it actually is -- mostly what I've done here is define a few traits Lintable: a struct is able to lint itself with a given argument to check against Parseable: a struct is able to create Self from a given input Validate: a struct satisfies both Lintable and Parseable The JS boilerplate is mostly done -- since we actually want to lint expressions, and not statements, the bulk of the code at the moment is just about getting to the expressions in each statement. Essentially a glorifed switch case with a little recursion mixed in. The webassembly is all todo!(), i just wanted to get typechecking working. Basically, this is just yet another commit before we actually parse any AST. But I did write some documentation, which is cool. commit 2ba95fb Author: Cassandra Fridkin <[email protected]> Date: Wed Sep 30 23:13:10 2020 -0400 Add some tests Looks like we will have to wait until the next commit to parse any AST commit 409063e Merge: 9da3a2e d6068cc Author: Cassandra Fridkin <[email protected]> Date: Wed Sep 30 20:53:06 2020 -0400 Merge branch 'cloudflare' into master commit 9da3a2e Author: Cassandra Fridkin <[email protected]> Date: Wed Sep 30 20:47:08 2020 -0400 Load the AST & restructure BundlerOutput for the last time This restructuring is because I think it makes more sense to have each FileType run its own checks. This also cleanly bridges between the functions in `js.rs` and `wasm.rs` and the BundlerOutput struct in `mod.rs`. commit e23f88d Author: Cassandra Fridkin <[email protected]> Date: Tue Sep 29 16:36:23 2020 -0400 Restructure a little bit and mess things up with async I think it makes sense to actually create a `Bundler` struct because like...eventually in my head it makes sense to have one `Bundler` instance that you could like pass around in `wrangler dev` or whatever. I think most of the codebase is more functional-y but i'm sure someone will tell me if this is not the preferred style to do this in. How many more commits can I make before I have to actually parse any javascript? Let's find out, shall we? commit 0fec4d4 Author: Cassandra Fridkin <[email protected]> Date: Mon Sep 28 00:21:24 2020 -0400 Add a few tests I suppose, and comment out constants we're not using yet commit e5b4049 Author: Cassandra Fridkin <[email protected]> Date: Sun Sep 27 23:18:51 2020 -0400 Add "None"s for bundle config so rustc stops complaining commit 7dafd73 Author: Cassandra Fridkin <[email protected]> Date: Sun Sep 27 23:17:06 2020 -0400 Preliminary "file exists" checks Also, use the nice AsRef<Path> idiom that works so well commit 67bfc70 Author: Cassandra Fridkin <[email protected]> Date: Thu Sep 24 20:07:24 2020 -0400 Add bundle as a target I think...at least...that that's what this commit does. Def starting to get into "oh god i hope i don't break anything" territory. commit 5f41321 Author: Cassandra Fridkin <[email protected]> Date: Thu Sep 24 15:36:56 2020 -0400 Add "bundle" type Not sure *exactly* what this does but i basically just copied the parts i thought were relevant from the site code. We have a src dir, which is what needs to be watched, and an output dir, which is what needs to be analyzed, and a build command, which is what needs to be run when we wanna run the build. commit efd3a44 Author: Cassandra Fridkin <[email protected]> Date: Tue Sep 22 22:10:26 2020 -0400 trivial commit so i can PR literally does nothing except break things :blushing:
cloudflare · Feb 25, 2021 · cdba3a4 · cdba3a4
1 parent 98466c7
commit cdba3a4
Show file tree

Hide file tree

Showing 11 changed files with 236 additions and 21 deletions.
diff --git a/src/build/mod.rs b/src/build/mod.rs
@@ -12,10 +12,30 @@ use std::process::Command;
 pub fn build_target(target: &Target) -> Result<String, failure::Error> {
     let target_type = &target.target_type;
     match target_type {
-        TargetType::JavaScript => {
-            let msg = "JavaScript project found. Skipping unnecessary build!".to_string();
-            Ok(msg)
-        }
+        TargetType::JavaScript => match &target.build {
+            None => {
+                let msg = "Basic JavaScript project found. Skipping unnecessary build!".to_string();
+                Ok(msg)
+            }
+            Some(config) => {
+                if let Some((cmd_str, mut cmd)) = config.build_command() {
+                    StdErr::working(format!("Running {}", cmd_str).as_ref());
+                    let build_result = cmd.spawn()?.wait()?;
+                    if build_result.success() {
+                        Ok(String::from("Build completed successfully!"))
+                    } else if let Some(code) = build_result.code() {
+                        Err(failure::err_msg(format!(
+                            "Build failed! Status Code: {}",
+                            code
+                        )))
+                    } else {
+                        Err(failure::err_msg("Build failed."))
+                    }
+                } else {
+                    Ok(String::from("No build command specified, skipping build."))
+                }
+            }
+        },
         TargetType::Rust => {
             let _ = which::which("rustc").map_err(|e| {
                 failure::format_err!(
@@ -31,6 +51,7 @@ pub fn build_target(target: &Target) -> Result<String, failure::Error> {
             let command = command(&args, &binary_path);
             let command_name = format!("{:?}", command);
 
+            StdErr::working("Compiling your project to WebAssembly...");
             commands::run(command, &command_name)?;
             let msg = "Build succeeded".to_string();
             Ok(msg)
@@ -49,8 +70,6 @@ pub fn build_target(target: &Target) -> Result<String, failure::Error> {
 }
 
 pub fn command(args: &[&str], binary_path: &PathBuf) -> Command {
-    StdErr::working("Compiling your project to WebAssembly...");
-
     let mut c = if cfg!(target_os = "windows") {
         let mut c = Command::new("cmd");
         c.arg("/C");

diff --git a/src/commands/kv/mod.rs b/src/commands/kv/mod.rs
@@ -125,6 +125,7 @@ mod tests {
             site: None,
             vars: None,
             text_blobs: None,
+            build: None,
         };
         assert!(kv::get_namespace_id(&target_with_dup_kv_bindings, "").is_err());
     }

diff --git a/src/commands/publish.rs b/src/commands/publish.rs
@@ -72,6 +72,12 @@ pub fn publish(
         }
         Err(e) => Err(e),
     }?;
+
+    // We verify early here, so we don't perform pre-upload tasks if the upload will fail
+    if let Some(build_config) = &target.build {
+        build_config.verify_upload_dir()?;
+    }
+
     if let Some(site_config) = &target.site {
         let path = &site_config.bucket.clone();
         validate_bucket_location(path)?;

diff --git a/src/settings/toml/builder.rs b/src/settings/toml/builder.rs
@@ -0,0 +1,113 @@
+use std::env;
+use std::path::PathBuf;
+use std::process::Command;
+
+use serde::{Deserialize, Serialize};
+
+use super::ScriptFormat;
+
+const UPLOAD_DIR: &str = "dist";
+const WATCH_DIR: &str = "src";
+
+#[derive(Clone, Debug, Deserialize, PartialEq, Serialize)]
+#[serde(deny_unknown_fields)]
+pub struct Builder {
+    command: Option<String>,
+    #[serde(default = "project_root")]
+    pub cwd: PathBuf,
+    #[serde(default = "upload_dir")]
+    pub upload_dir: PathBuf,
+    pub upload_format: ScriptFormat,
+    #[serde(default = "watch_dir")]
+    pub watch_dir: PathBuf,
+}
+
+fn project_root() -> PathBuf {
+    env::current_dir().unwrap()
+}
+
+fn upload_dir() -> PathBuf {
+    project_root().join(UPLOAD_DIR)
+}
+
+fn watch_dir() -> PathBuf {
+    project_root().join(WATCH_DIR)
+}
+
+impl Builder {
+    pub fn verify_watch_dir(&self) -> Result<(), failure::Error> {
+        let watch_canonical = match self.watch_dir.canonicalize() {
+            Ok(path) => path,
+            Err(e) if matches!(e.kind(), std::io::ErrorKind::NotFound) => failure::bail!(
+                "Your provided watch_dir {} does not exist.",
+                self.watch_dir.display()
+            ),
+            Err(e) => failure::bail!(
+                "Error encountered when verifying watch_dir: {}, provided path: {}",
+                e,
+                self.watch_dir.display()
+            ),
+        };
+        let root_canonical = project_root().canonicalize()?;
+        if watch_canonical == root_canonical {
+            failure::bail!("Wrangler doesn't support using the project root as the watch_dir.");
+        }
+        if !self.watch_dir.is_dir() {
+            failure::bail!(format!(
+                "A path was provided for watch_dir that is not a directory: {}",
+                self.watch_dir.display()
+            ));
+        }
+        Ok(())
+    }
+
+    pub fn verify_upload_dir(&self) -> Result<(), failure::Error> {
+        let upload_canonical = match self.upload_dir.canonicalize() {
+            Ok(path) => path,
+            Err(e) if matches!(e.kind(), std::io::ErrorKind::NotFound) => failure::bail!(
+                "Your provided upload_dir {} does not exist.",
+                self.upload_dir.display()
+            ),
+            Err(e) => failure::bail!(
+                "Error encountered when verifying upload_dir: {}, provided path: {}",
+                e,
+                self.upload_dir.display()
+            ),
+        };
+        let root_canonical = project_root().canonicalize()?;
+        if upload_canonical == root_canonical {
+            failure::bail!("Wrangler doesn't support using the project root as the upload_dir.");
+        }
+        if !self.upload_dir.is_dir() {
+            failure::bail!(format!(
+                "A path was provided for upload_dir that is not a directory: {}",
+                self.upload_dir.display()
+            ));
+        }
+        Ok(())
+    }
+
+    pub fn build_command(&self) -> Option<(&str, Command)> {
+        match &self.command {
+            Some(cmd) => {
+                let mut c = if cfg!(target_os = "windows") {
+                    let args: Vec<&str> = cmd.split_whitespace().collect();
+                    let mut c = Command::new("cmd");
+                    c.arg("/C");
+                    c.args(args.as_slice());
+                    c
+                } else {
+                    let mut c = Command::new("sh");
+                    c.arg("-c");
+                    c.arg(cmd);
+                    c
+                };
+
+                c.current_dir(&self.cwd);
+
+                Some((cmd, c))
+            }
+            None => None,
+        }
+    }
+}
diff --git a/src/settings/toml/environment.rs b/src/settings/toml/environment.rs
@@ -4,6 +4,7 @@ use std::path::PathBuf;
 use serde::{Deserialize, Serialize};
 use serde_with::rust::string_empty_as_none;
 
+use crate::settings::toml::builder::Builder;
 use crate::settings::toml::kv_namespace::ConfigKvNamespace;
 use crate::settings::toml::route::RouteConfig;
 use crate::settings::toml::site::Site;
@@ -21,6 +22,7 @@ pub struct Environment {
     #[serde(default, with = "string_empty_as_none")]
     pub zone_id: Option<String>,
     pub webpack_config: Option<String>,
+    pub build: Option<Builder>,
     pub private: Option<bool>,
     pub site: Option<Site>,
     #[serde(alias = "kv-namespaces")]

diff --git a/src/settings/toml/manifest.rs b/src/settings/toml/manifest.rs
@@ -11,6 +11,7 @@ use serde_with::rust::string_empty_as_none;
 
 use crate::commands::{validate_worker_name, DEFAULT_CONFIG_PATH};
 use crate::deploy::{self, DeployTarget, DeploymentSet};
+use crate::settings::toml::builder::Builder;
 use crate::settings::toml::dev::Dev;
 use crate::settings::toml::environment::Environment;
 use crate::settings::toml::kv_namespace::{ConfigKvNamespace, KvNamespace};
@@ -40,6 +41,7 @@ pub struct Manifest {
     #[serde(default, with = "string_empty_as_none")]
     pub zone_id: Option<String>,
     pub webpack_config: Option<String>,
+    pub build: Option<Builder>,
     pub private: Option<bool>,
     // TODO: maybe one day, serde toml support will allow us to serialize sites
     // as a TOML inline table (this would prevent confusion with environments too!)
@@ -290,6 +292,7 @@ impl Manifest {
             target_type,                                 // Top level
             account_id: self.account_id.clone(),         // Inherited
             webpack_config: self.webpack_config.clone(), // Inherited
+            build: self.build.clone(),                   // Inherited
             // importantly, the top level name will be modified
             // to include the name of the environment
             name: self.name.clone(), // Inherited
@@ -309,6 +312,9 @@ impl Manifest {
             if let Some(webpack_config) = &environment.webpack_config {
                 target.webpack_config = Some(webpack_config.clone());
             }
+            if let Some(build) = &environment.build {
+                target.build = Some(build.clone());
+            }
 
             // don't inherit kv namespaces because it is an anti-pattern to use the same namespaces across multiple environments
             target.kv_namespaces = get_namespaces(environment.kv_namespaces.clone(), preview)?;

diff --git a/src/settings/toml/mod.rs b/src/settings/toml/mod.rs
@@ -1,17 +1,21 @@
+mod builder;
 mod dev;
 mod environment;
 mod kv_namespace;
 mod manifest;
 mod route;
+mod script_format;
 mod site;
 mod target;
 mod target_type;
 mod triggers;
 
+pub use builder::Builder;
 pub use environment::Environment;
 pub use kv_namespace::{ConfigKvNamespace, KvNamespace};
 pub use manifest::Manifest;
 pub use route::{Route, RouteConfig};
+pub use script_format::ScriptFormat;
 pub use site::Site;
 pub use target::Target;
 pub use target_type::TargetType;

diff --git a/src/settings/toml/script_format.rs b/src/settings/toml/script_format.rs
@@ -0,0 +1,34 @@
+use std::fmt;
+use std::str::FromStr;
+
+use serde::{Deserialize, Serialize};
+
+#[derive(Clone, Debug, Deserialize, Serialize, PartialEq)]
+pub enum ScriptFormat {
+    #[serde(rename = "service-worker")]
+    ServiceWorker,
+    #[serde(rename = "modules")]
+    Modules,
+}
+
+impl fmt::Display for ScriptFormat {
+    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
+        let printable = match *self {
+            Self::ServiceWorker => "service-worker",
+            Self::Modules => "modules",
+        };
+        write!(f, "{}", printable)
+    }
+}
+
+impl FromStr for ScriptFormat {
+    type Err = failure::Error;
+
+    fn from_str(s: &str) -> Result<Self, Self::Err> {
+        match s {
+            "service-worker" => Ok(Self::ServiceWorker),
+            "modules" => Ok(Self::Modules),
+            _ => failure::bail!("{} is not a valid script format!", s),
+        }
+    }
+}
diff --git a/src/settings/toml/target.rs b/src/settings/toml/target.rs
@@ -1,3 +1,4 @@
+use super::builder::Builder;
 use super::kv_namespace::KvNamespace;
 use super::site::Site;
 use super::target_type::TargetType;
@@ -14,6 +15,7 @@ pub struct Target {
     pub name: String,
     pub target_type: TargetType,
     pub webpack_config: Option<String>,
+    pub build: Option<Builder>,
     pub site: Option<Site>,
     pub vars: Option<HashMap<String, String>>,
     pub text_blobs: Option<HashMap<String, PathBuf>>,

diff --git a/src/sites/mod.rs b/src/sites/mod.rs
@@ -318,6 +318,7 @@ mod tests {
             target_type: TargetType::JavaScript,
             webpack_config: None,
             site: Some(site),
+            build: None,
             vars: None,
             text_blobs: None,
         }

diff --git a/src/watch/mod.rs b/src/watch/mod.rs
@@ -3,10 +3,10 @@ use ignore::overrides::OverrideBuilder;
 use ignore::WalkBuilder;
 pub use watcher::wait_for_changes;
 
-use crate::build::command;
 use crate::settings::toml::{Target, TargetType};
 use crate::terminal::message::{Message, StdOut};
 use crate::wranglerjs;
+use crate::{build::command, build_target};
 use crate::{commands, install};
 
 use notify::{self, RecursiveMode, Watcher};
@@ -28,27 +28,54 @@ pub fn watch_and_build(
     tx: Option<mpsc::Sender<()>>,
 ) -> Result<(), failure::Error> {
     let target_type = &target.target_type;
+    let build = target.build.clone();
     match target_type {
         TargetType::JavaScript => {
-            thread::spawn(move || {
+            let target = target.clone();
+            thread::spawn::<_, Result<(), failure::Error>>(move || {
                 let (watcher_tx, watcher_rx) = mpsc::channel();
-                let mut watcher = notify::watcher(watcher_tx, Duration::from_secs(1)).unwrap();
+                let mut watcher = notify::watcher(watcher_tx, Duration::from_secs(1))?;
 
-                watcher
-                    .watch(JAVASCRIPT_PATH, RecursiveMode::Recursive)
-                    .unwrap();
-                StdOut::info(&format!("watching {:?}", &JAVASCRIPT_PATH));
+                match build {
+                    None => {
+                        watcher.watch(JAVASCRIPT_PATH, RecursiveMode::Recursive)?;
+                        StdOut::info(&format!("watching {:?}", &JAVASCRIPT_PATH));
 
-                loop {
-                    match wait_for_changes(&watcher_rx, COOLDOWN_PERIOD) {
-                        Ok(_path) => {
-                            if let Some(tx) = tx.clone() {
-                                tx.send(()).expect("--watch change message failed to send");
+                        loop {
+                            match wait_for_changes(&watcher_rx, COOLDOWN_PERIOD) {
+                                Ok(_path) => {
+                                    if let Some(tx) = tx.clone() {
+                                        tx.send(()).expect("--watch change message failed to send");
+                                    }
+                                }
+                                Err(e) => {
+                                    log::debug!("{:?}", e);
+                                    StdOut::user_error("Something went wrong while watching.")
+                                }
                             }
                         }
-                        Err(e) => {
-                            log::debug!("{:?}", e);
-                            StdOut::user_error("Something went wrong while watching.")
+                    }
+                    Some(config) => {
+                        config.verify_watch_dir()?;
+                        watcher.watch(config.watch_dir, notify::RecursiveMode::Recursive)?;
+
+                        loop {
+                            match wait_for_changes(&watcher_rx, COOLDOWN_PERIOD) {
+                                Ok(_path) => match build_target(&target) {
+                                    Ok(output) => {
+                                        StdOut::success(&output);
+                                        if let Some(tx) = tx.clone() {
+                                            tx.send(())
+                                                .expect("--watch change message failed to send");
+                                        }
+                                    }
+                                    Err(e) => StdOut::user_error(&e.to_string()),
+                                },
+                                Err(e) => {
+                                    log::debug!("{:?}", e);
+                                    StdOut::user_error("Something went wrong while watching.")
+                                }
+                            }
                         }
                     }
                 }