Skip to content

Commit

Permalink
feat: build flows rewrite to be independent of powershell and bash (#245
Browse files Browse the repository at this point in the history
)
  • Loading branch information
kakhaUrigashvili authored Oct 27, 2020
1 parent 6b68be0 commit 6b27ef8
Show file tree
Hide file tree
Showing 40 changed files with 1,219 additions and 857 deletions.
14 changes: 11 additions & 3 deletions .github/workflows/integration-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,22 @@ on:
jobs:
build:

runs-on: ubuntu-latest
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
node: [12]

steps:
- uses: actions/checkout@v2
- name: Use Node.js
- name: Use Node.js ${{ matrix.node }}
uses: actions/setup-node@v1
with:
node-version: '12.x'
node-version: ${{ matrix.node }}
- name: Set up Python 3.x
uses: actions/setup-python@v2
with:
python-version: '3.x'
- run: npm install
- run: npm link
- run: npm run integration-test
Expand Down
43 changes: 39 additions & 4 deletions docs/concepts/Deploy-Command.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,15 +38,50 @@ In the deploy process of skillCode, CLI helps skill developers build all of thei
### Code Builder
To support the building of multiple programming languages, CLI's philosophy is to provide a **built-in or customized** build-flows (*i.e. {programmingLanguage}-{builderTool}*) to fullfill normal needs as well as special requests. This is called `CodeBuilder` in CLI.

* Most use cases will be covered by using the **built-in** build-flows. When building skillCode with this flow, CLI will infer the builderTool (based on the type of builder's config file) from the codebase, and further decide which build-flow to execute. Build-flows are represented by cross-OS executable scripts. Current built-in build-flows include:
* nodejs-npm [(scripts)](https://github.com/alexa/ask-cli/tree/master/lib/builtins/build-flows/nodejs-npm)
* python-pip [(scripts)](https://github.com/alexa/ask-cli/tree/master/lib/builtins/build-flows/python-pip)
* java-mvn [(scripts)](https://github.com/alexa/ask-cli/tree/master/lib/builtins/build-flows/java-mvn)
* Most use cases will be covered by using the **built-in** build-flows. When building skillCode with this flow, CLI will infer the builderTool (based on the type of builder's config file) from the codebase, and further decide which build-flow to execute.
* For developers who have further desire to **customize** the build-flow by themselves, CLI also supports the `custom` type of build-flow. If you provide the hook script in the following location, the script will be executed instead of using the built-in build-flows, and it is codebase-agnostic:
* Non-Windows path: `{projectRoot}/hooks/build.sh` file
* Windows path: `{projectRoot}\hooks\build.ps1` file
* Each build flow (either built-in or customized) is executed with two parameters: the path to the build file and if it is verbose and need debug information.

#### Examples of customized build-flows

`{projectRoot}/hooks/build.sh`
```
#!/bin/bash
readonly OUT_FILE=$1
readonly DO_DEBUG=$2
echo $OUT_FILE
echo $DO_DEBUG
# custom logic
```

`{projectRoot}\hooks\build.ps1`
```
#requires -version 3
#----------------[ Parameters ]----------------------------------------------------
param(
[Parameter(Mandatory = $false,
ValueFromPipelineByPropertyName = $true,
HelpMessage = "Name for the AWS Lambda deployable archive")]
[ValidateNotNullOrEmpty()]
[string]
$script:OutFile = "upload.zip",
# Provide additional debug information during script run
[Parameter(Mandatory = $false,
ValueFromPipelineByPropertyName = $true,
HelpMessage = "Enable verbose output")]
[bool]
$script:Verbose = $false
)
# custom logic
```

## Skill Infrastructure
To provision the backend services which are used to execute customized logics for Alexa skills, CLI introduces the `skillInfrastructure` concept to incorporate different deploy mechanisms into one platform. Each deployment flow is presented as a type of deployer, which is the value set in `skillInfrastructure.type`. Another two fields, `skillInfrastructure.userConfig` is designed to configure the deployment, and `skillInfrastructure.deployState` is used to facilitate CD and is not supposed to be modified manually.
Expand Down
93 changes: 93 additions & 0 deletions lib/builtins/build-flows/abstract-build-flow.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
const AdmZip = require('adm-zip');
const childProcess = require('child_process');
const path = require('path');

const Messenger = require('@src/view/messenger');

class AbstractBuildFlow {
/**
* Constructor
* @param {Object} options
* @param {String} options.cwd working directory for build
* @param {String} options.src source directory
* @param {String} options.buildFile full path for zip file to generate
* @param {Boolean} options.doDebug debug flag
*/
constructor({ cwd, src, buildFile, doDebug }) {
this.cwd = cwd;
this.src = src;
this.buildFile = buildFile;
this.stdio = doDebug ? 'inherit' : 'pipe';
this.doDebug = !!doDebug;
this.isWindows = process.platform === 'win32';
this.defaultZipFileDate = new Date(1990, 1, 1);
}

/**
* Creates build zip file
* @param {Object} options
* @param {Object} options.filter filter to apply to exclude files from zip
* @param {Function} callback
*/
createZip(options, callback) {
if (typeof options === 'function') {
callback = options;
options = {};
}
const filter = options.filter || (() => false);

this.debug(`Zipping source files and dependencies to ${this.buildFile}.`);
const zip = new AdmZip();
const zipFileName = path.basename(this.buildFile);

// adding files
zip.addLocalFolder(this.cwd, '', (entry) => entry !== zipFileName && !filter(entry));
// setting create timestamp to the same value to allow consistent hash
zip.getEntries().forEach(e => { e.header.time = this.defaultZipFileDate; });

zip.writeZip(this.buildFile, callback);
}

/**
* Modifies build zip file
* @param {Object} options
* @param {Object} options.entryProcessor function to apply to each zip file entry
* @param {Function} callback
*/
modifyZip(options, callback) {
if (typeof options === 'function') {
callback = options;
options = {};
}
const zip = new AdmZip(this.buildFile);

const entryProcessor = options.entryProcessor || (() => {});

zip.getEntries().forEach(e => {
// setting create timestamp to the same value to allow consistent hash
e.header.time = this.defaultZipFileDate;
entryProcessor(e);
});
zip.writeZip(this.buildFile, callback);
}

/**
* Executes shell command
* @param {String} cmd command
*/
execCommand(cmd) {
childProcess.execSync(cmd, { cwd: this.cwd, stdio: this.stdio });
}

/**
* Outputs debug message
* @param {String} message message
*/
debug(message) {
if (this.doDebug) {
Messenger.getInstance().debug(message);
}
}
}

module.exports = AbstractBuildFlow;
50 changes: 50 additions & 0 deletions lib/builtins/build-flows/custom.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
const fs = require('fs-extra');
const path = require('path');

const AbstractBuildFlow = require('./abstract-build-flow');

class CustomBuildFlow extends AbstractBuildFlow {
/**
* If this file exists than the build flow can handle build
*/
static get manifest() { return process.platform === 'win32' ? 'build.ps1' : 'build.sh'; }

static get _customScriptPath() { return path.join(process.cwd(), 'hooks', CustomBuildFlow.manifest); }

/**
* Returns true if the build flow can handle the build
*/
static canHandle() {
return fs.existsSync(CustomBuildFlow._customScriptPath);
}

/**
* Constructor
* @param {Object} options
* @param {String} options.cwd working directory for build
* @param {String} options.src source directory
* @param {String} options.buildFile full path for zip file to generate
* @param {Boolean} options.doDebug debug flag
*/
constructor({ cwd, src, buildFile, doDebug }) {
super({ cwd, src, buildFile, doDebug });
}

/**
* Executes build
* @param {Function} callback
*/
execute(callback) {
this.debug(`Executing custom hook script ${CustomBuildFlow._customScriptPath}.`);
let command = `${CustomBuildFlow._customScriptPath} "${this.buildFile}" ${this.doDebug}`;
if (this.isWindows) {
const powerShellPrefix = 'PowerShell.exe -Command';
const doDebug = this.doDebug ? '$True' : '$False';
command = `${powerShellPrefix} "& {& '${CustomBuildFlow._customScriptPath}' '${this.buildFile}' ${doDebug} }"`;
}
this.execCommand(command);
callback();
}
}

module.exports = CustomBuildFlow;
57 changes: 57 additions & 0 deletions lib/builtins/build-flows/java-mvn.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
const fs = require('fs-extra');
const path = require('path');

const AbstractBuildFlow = require('./abstract-build-flow');

class JavaMvnBuildFlow extends AbstractBuildFlow {
/**
* If this file exists than the build flow can handle build
*/
static get manifest() { return 'pom.xml'; }

/**
* Returns true if the build flow can handle the build
*/
static canHandle({ src }) {
return fs.existsSync(path.join(src, JavaMvnBuildFlow.manifest));
}

/**
* Constructor
* @param {Object} options
* @param {String} options.cwd working directory for build
* @param {String} options.src source directory
* @param {String} options.buildFile full path for zip file to generate
* @param {Boolean} options.doDebug debug flag
*/
constructor({ cwd, src, buildFile, doDebug }) {
super({ cwd, src, buildFile, doDebug });
}

/**
* Executes build
* @param {Function} callback
*/
execute(callback) {
this.debug(`Building skill artifacts based on the ${JavaMvnBuildFlow.manifest}.`);
this.execCommand('mvn clean org.apache.maven.plugins:maven-assembly-plugin:2.6:assembly '
+ '-DdescriptorId=jar-with-dependencies package');
const targetFolderPath = path.join(this.cwd, 'target');
const jarFileName = fs.readdirSync(targetFolderPath).find(fileName => fileName.endsWith('jar-with-dependencies.jar'));
const jarFilePath = path.join(targetFolderPath, jarFileName);
this.debug(`Renaming the jar file ${jarFilePath} to ${this.buildFile}.`);
fs.moveSync(jarFilePath, this.buildFile, { overwrite: true });

this.modifyZip({ entryProcessor: this._removeCommentsFromPomProperties }, callback);
}

_removeCommentsFromPomProperties(entry) {
// removing comment to allow consistent hashing
if (entry.entryName.includes('pom.properties')) {
const data = entry.getData().toString().replace(/^#.*\n?/mg, '');
entry.setData(data);
}
}
}

module.exports = JavaMvnBuildFlow;
Loading

0 comments on commit 6b27ef8

Please sign in to comment.