Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

postprocessors #383

Merged
merged 44 commits into from
Nov 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
8cbc730
init: postprocessors
b1ek Jul 31, 2024
20f4065
init: bshchk postprocessor
b1ek Jul 31, 2024
6a5e7ab
Merge branch 'amber-lang:master' into postprocess
b1ek Aug 4, 2024
f80827a
fix: tests failing
b1ek Aug 4, 2024
9726a0a
docs: mention bshchk in readme
b1ek Aug 4, 2024
9defaff
refactor: change --disable-postprocessors arg name and properly expla…
b1ek Aug 4, 2024
a679d3d
tests: use the proper argument name in cli test
b1ek Aug 4, 2024
b46351d
feat: filter postprocessors with a wildcard
b1ek Aug 19, 2024
cbdb5d9
Merge branch 'master' into postprocess
b1ek Sep 7, 2024
a3ec4b6
tests: postprocessor
b1ek Sep 7, 2024
d914d02
fix: clippy
b1ek Sep 7, 2024
9f2888d
fix: disable postprocessors in tests
b1ek Sep 7, 2024
d33dff1
Merge branch 'master' into postprocess
b1ek Oct 5, 2024
c4417a0
fix: merge issue
b1ek Oct 5, 2024
c7c6e49
refactor: remove input/output enums
b1ek Oct 5, 2024
be855f2
test: disable all postprocessors in most tests
b1ek Oct 15, 2024
0490f4b
tests: disable postprocessors by wildcard in tests::cli
b1ek Oct 15, 2024
d4a3b0b
tests: fix tests::postprocessor
b1ek Oct 15, 2024
5883644
Merge branch 'master' into postprocess
b1ek Oct 30, 2024
7c2d156
remove useless section from readme
b1ek Oct 30, 2024
1e04623
revert variable name
b1ek Oct 30, 2024
5e100c4
rewrite only to have a filter_default method
b1ek Oct 30, 2024
fb61836
pipe stdin,out,err to rust
b1ek Oct 31, 2024
a1136f5
always remove file in all_exist test
b1ek Oct 31, 2024
5002fe0
remove useless function
b1ek Oct 31, 2024
93c21ef
rewrite postprocessor test to be simpler but require all default post…
b1ek Oct 31, 2024
bf2976e
assert that output is the same as well
b1ek Oct 31, 2024
f0c34ff
do not clone postprocessor where it doesnt have to be
b1ek Oct 31, 2024
0048b9b
remove formatter.rs
b1ek Oct 31, 2024
40fe379
fix header
b1ek Oct 31, 2024
cfe7d3d
do not use hashmap in PostProcessor::get_default
b1ek Oct 31, 2024
fb0c54b
change --disable-postprocessor to --no-proc and edit help message to …
b1ek Oct 31, 2024
ad55711
put --no-proc help string in a comment above it
b1ek Oct 31, 2024
3cdaaed
fix type
b1ek Oct 31, 2024
298f313
remove useless inner scopes
b1ek Oct 31, 2024
4770057
fix filtering
b1ek Nov 2, 2024
60626f2
use Rc<RefCell> instead of Arc<Mutex>
hdwalters Nov 2, 2024
9561819
install bshchk in ci
b1ek Nov 2, 2024
7f35407
fix perms
b1ek Nov 2, 2024
20ea412
make clippy happy
b1ek Nov 2, 2024
a541477
make clippy even more happier
b1ek Nov 2, 2024
4552ad7
return false if kill failed
b1ek Nov 9, 2024
c68d6b3
dont pad blocks
b1ek Nov 9, 2024
c1ee41b
Merge branch 'master' into postprocess
b1ek Nov 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,10 @@ jobs:
with:
packages: bc shfmt
version: 1.0
- name: Install bshchk
run: |
sudo curl https://github.com/b1ek/bshchk/releases/download/1.1/bshchk.linux.amd64 -L -o /usr/bin/bshchk
sudo chmod +x /usr/bin/bshchk
Ph0enixKM marked this conversation as resolved.
Show resolved Hide resolved
- uses: dtolnay/rust-toolchain@stable
- name: Cache dependencies installed with cargo
uses: actions/cache@v4
Expand Down
7 changes: 7 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,13 @@ amber-meta = { path = "meta" }
chrono = "0.4.38"
clap = { version = "4.4.18", features = ["derive"] }
colored = "2.0.0"
glob = "0.3"
heraclitus-compiler = "1.8.1"
include_dir = "0.7.4"
itertools = "0.13.0"
similar-string = "1.4.2"
test-generator = "0.3.1"
glob = "0.3"
wildmatch = "2.4.0"

# test dependencies
[dev-dependencies]
Expand Down
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,7 @@

# Amber

Programming language that compiles to Bash. It's a high level programming language that makes it easy to create shell scripts. It's particularly well suited for cloud services.
If [shfmt](https://github.com/mvdan/sh) it is present in the machine it will be used after the compilation to prettify the Bash code generated.
Programming language that compiles to Bash. It's a high level programming language that makes it easy to create shell scripts. It's particularly well suited for cloud services.

> [!Warning]
> This software is not ready for extended usage.
Expand Down
35 changes: 20 additions & 15 deletions src/compiler.rs
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
extern crate chrono;
use crate::docs::module::DocumentationModule;
use crate::modules::block::Block;
use crate::modules::formatter::BashFormatter;
use crate::translate::check_all_blocks;
use crate::translate::module::TranslateModule;
use crate::utils::{ParserMetadata, TranslateMetadata};
use crate::{rules, Cli};
use postprocessor::PostProcessor;
use chrono::prelude::*;
use colored::Colorize;
use heraclitus_compiler::prelude::*;
use wildmatch::WildMatchPattern;
use std::env;
use std::fs;
use std::fs::File;
Expand All @@ -17,6 +18,8 @@ use std::path::PathBuf;
use std::process::{Command, ExitStatus};
use std::time::Instant;

pub mod postprocessor;

const NO_CODE_PROVIDED: &str = "No code has been provided to the compiler";
const AMBER_DEBUG_PARSER: &str = "AMBER_DEBUG_PARSER";
const AMBER_DEBUG_TIME: &str = "AMBER_DEBUG_TIME";
Expand Down Expand Up @@ -164,22 +167,23 @@ impl AmberCompiler {

let mut result = result.join("\n") + "\n";

if !self.cli_opts.disable_format {
if let Some(formatter) = BashFormatter::get_available() {
result = formatter.format(result);
}
let filters = self.cli_opts.no_proc.iter()
.map(|x| WildMatchPattern::new(x)).collect();

let postprocessors = PostProcessor::filter_default(filters);

for postprocessor in postprocessors {
result = postprocessor.execute(result).unwrap_or_else(|_| panic!("Postprocessor {} failed!", postprocessor.name));
}

let header = [
include_str!("header.sh"),
&("# version: ".to_owned() + env!("CARGO_PKG_VERSION").to_string().as_str()),
&("# date: ".to_owned()
+ Local::now()
.format("%Y-%m-%d %H:%M:%S")
.to_string()
.as_str()),
].join("\n");
format!("{}\n{}", header, result)
let header = include_str!("header.sh")
b1ek marked this conversation as resolved.
Show resolved Hide resolved
.replace("{{ version }}", env!("CARGO_PKG_VERSION"))
.replace("{{ date }}", Local::now()
.format("%Y-%m-%d %H:%M:%S")
.to_string()
.as_str()
);
format!("{}{}", header, result)
}

pub fn document(&self, block: Block, meta: ParserMetadata, output: String) {
Expand Down Expand Up @@ -264,6 +268,7 @@ impl AmberCompiler {

#[cfg(test)]
pub fn test_eval(&mut self) -> Result<String, Message> {
self.cli_opts.no_proc = vec!["*".into()];
self.compile().map_or_else(Err, |(_, code)| {
if let Some(mut command) = Self::find_bash() {
let child = command.arg("-c").arg::<&str>(code.as_ref()).output().unwrap();
Expand Down
92 changes: 92 additions & 0 deletions src/compiler/postprocessor.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
use std::cell::RefCell;
use std::io::{BufWriter, Write};
use std::path::PathBuf;
use std::process::{Command, Stdio};
use std::rc::Rc;

use itertools::Itertools;
use wildmatch::WildMatchPattern;

#[derive(Debug, Clone)]
pub struct PostProcessor {
pub name: String,
pub bin: PathBuf,
command: Rc<RefCell<Command>>,
}

impl PostProcessor {
pub fn new<N: Into<String>, B: Into<PathBuf>>(name: N, bin: B) -> Self {
let name: String = name.into();
let bin: PathBuf = bin.into();
let mut command = Command::new(bin.clone());
command.stdin(Stdio::piped());
command.stdout(Stdio::piped());
command.stderr(Stdio::piped());
let command = Rc::new(RefCell::new(command));
Self {
name,
bin,
command,
}
}

pub fn cmd(&self) -> Rc<RefCell<Command>> {
self.command.clone()
}

pub fn is_available(&self) -> bool {
match Command::new(self.bin.clone()).spawn() {
Ok(mut v) => {
v.kill().is_ok()
},
Err(_) => false
}
}

pub fn execute(&self, code: String) -> Result<String, Box<dyn std::error::Error>> {
if !self.is_available() { return Ok(code) }

let mut spawned = self.cmd().borrow_mut().spawn()?;

// send to stdin
if let Some(stdin) = spawned.stdin.as_mut() {
let mut writer = BufWriter::new(stdin);
writer.write_all(code.as_bytes())?;
writer.flush()?;
} else {
return Err(String::new().into())
}

// read from stdout
let res = spawned.wait_with_output()?;
Ok(String::from_utf8(res.stdout)?)
}

pub fn get_default() -> Vec<Self> {
let mut postprocessors = Vec::new();

let shfmt = PostProcessor::new("shfmt", "/usr/bin/shfmt");
shfmt.cmd().borrow_mut().arg("-i").arg("4");
shfmt.cmd().borrow_mut().arg("-ln").arg("bash");
postprocessors.push(shfmt);

let bshchk = PostProcessor::new("bshchk", "/usr/bin/bshchk");
bshchk.cmd().borrow_mut().arg("--ignore-shebang");
postprocessors.push(bshchk);

postprocessors
}

pub fn filter_default(filters: Vec<WildMatchPattern<'*', '?'>>) -> Vec<Self> {
let default = Self::get_default();

default
.iter()
.filter(|x| {
filters.iter()
.all(|xx| !xx.matches(&x.name))
})
.cloned()
.collect_vec()
}
}
4 changes: 3 additions & 1 deletion src/header.sh
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
#!/usr/bin/env bash
# Written in [Amber](https://amber-lang.com/)
# Written in [Amber](https://amber-lang.com/)
# version: {{ version }}
# date: {{ date }}
11 changes: 7 additions & 4 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,13 @@ pub struct Cli {
/// (OUTPUT is dir instead, default: `docs/` if missing it will generate the folder)
#[arg(long)]
docs: bool,

/// Don't format the output file
#[arg(long)]
disable_format: bool,

/// Disable a postprocessor
/// Available postprocessors: shfmt, bshchk
/// To select multiple, pass this argument multiple times with different values.
/// This argument also supports a wilcard match, like "*" or "s*mt"
#[arg(long, verbatim_doc_comment)]
no_proc: Vec<String>,

/// Minify the resulting code
#[arg(long)]
Expand Down
74 changes: 0 additions & 74 deletions src/modules/formatter.rs

This file was deleted.

1 change: 0 additions & 1 deletion src/modules/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ pub mod types;
pub mod imports;
pub mod main;
pub mod builtin;
pub mod formatter;

#[macro_export]
macro_rules! handle_types {
Expand Down
3 changes: 2 additions & 1 deletion src/tests/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ fn bash_error_exit_code() -> Result<(), Box<dyn std::error::Error>> {

// Changes locale to default to prevent locale-specific error messages.
cmd.env("LC_ALL", "C")
.arg("--disable-format")
.arg("--no-proc")
.arg("*")
.arg(file.path());

cmd.assert()
Expand Down
41 changes: 0 additions & 41 deletions src/tests/formatter.rs

This file was deleted.

Loading