Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create patched YML in place #163

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .github/workflows/example-10.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,24 @@ jobs:
mamba-version: ${{ matrix.mamba-version }}
use-mamba: true
python-version: ${{ matrix.python-version }}
clean-patched-environment-file: false

- run: |
mamba info
mamba list
python -VV
printenv | sort
- name: verify unpatched environment.yml is reported and not cleaned
if: contains(matrix.environment-file, '.yml') && !matrix.python-version
run: |
set -eux
ls '${{ steps.setup-miniconda.outputs.environment-file }}'
ls '${{ matrix.environment-file }}'
diff -s '${{ steps.setup-miniconda.outputs.environment-file }}' '${{ matrix.environment-file }}' | grep 'are identical'
- name: verify patched output is reported, correct, and not cleaned
if: contains(matrix.environment-file, '.yml') && matrix.python-version
run: |
set -eux
if [ "$(diff '${{ steps.setup-miniconda.outputs.environment-file }}' '${{ matrix.environment-file }}' | grep -c 'python=${{ matrix.python-version }}')" -ge 1 ] ; then echo ok ; else exit 1 ; fi
python --version | grep "Python ${{ matrix.python-version }}"
awk '/- conda-forge/,/- defaults/' '${{ steps.setup-miniconda.outputs.environment-file }}'
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,26 @@
# CHANGELOG

## [v2.1.1] (unreleased)

### Features

- [#163] leaves the patched `setup-miniconda-patched-{environment.yml}` in-place
if `clean-patched-environment-file: false` is given (otherwise cleans up after
itself)
- [#163] adds action outputs `environment-file`, `environment-file-content` and
`environment-file-was-patched`

### Fixes

- [#161] restores proper ordering of `channels` when `environment-file` is
patched
- [#163] if necessary, writes `setup-miniconda-patched-environment.yml` to the
same location to work with relative paths, e.g. `pip: ["-r requirements.txt"]`

[v2.1.1]: https://github.com/conda-incubator/setup-miniconda/releases/tag/v2.1.0
[#161]: https://github.com/conda-incubator/setup-miniconda/pull/161
[#163]: https://github.com/conda-incubator/setup-miniconda/pull/163

## [v2.1.0] (2021-03-29)

### Features
Expand Down
9 changes: 6 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,15 @@ A `conda-build-version` or `mamba-version` may be provided to install into
The base `condabin/` folder is added to `$PATH` and shell integration is
initialized across all platforms.

By default, this action will then create, and activate an environment by one of:
By default, this action will then create, and _activate_, an environment by one
of:

- creating a mostly-empty `test` environment, containing only the latest
`python-version` and its dependencies
- creating an `test` environment described in a given `environment-file`:
- an `environment.yml`-like file (which can be patched with `python-version`)
- the patched environment will be cleaned up unless
`clean-patched-environment-file: false` is given
- a [lockfile](#example-7-explicit-specification)

This action correctly handles activation of environments and offers the
Expand Down Expand Up @@ -118,9 +121,9 @@ activate the `base` environment. This encourages the practice of not using the
`base` environment to install packages used for the workflow and leave the
`base` environment untouched, with only `conda` (and/or `mamba`) in it.

## Inputs
## Inputs and outputs

For a full list of available inputs for this action see
For a full list of available _inputs_ and _outputs_ for this action see
[action.yml](action.yml).

### Use a different environment name or path
Expand Down
32 changes: 24 additions & 8 deletions action.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,25 @@
name: "Setup Miniconda"
author: Gonzalo Peña-Castellanos (@goanpeca)
description: "Set up Conda package and environment manager with Miniconda."

runs:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just moved these up top so they were more discoverable

using: "node12"
main: "dist/setup/index.js"
post: "dist/delete/index.js"
post-if: "success()"

branding:
icon: "code"
color: "green"

outputs:
environment-file:
description: "The full path to the environment-file used"
environment-file-content:
description: "The content of the environment-file used"
environment-file-was-patched:
description: "Whether a patched environment-file was made due to inputs"

inputs:
installer-url:
description:
Expand Down Expand Up @@ -220,11 +239,8 @@ inputs:
GitHub-hosted runners are "x86" and "x64". Default is "x64".'
required: false
default: "x64"
runs:
using: "node12"
main: "dist/setup/index.js"
post: "dist/delete/index.js"
post-if: "success()"
branding:
icon: "code"
color: "green"
clean-patched-environment-file:
description:
"Whether a patched environment-file (if created) should be cleaned"
required: false
default: "true"
14 changes: 13 additions & 1 deletion dist/delete/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -1161,7 +1161,7 @@ var __importStar = (this && this.__importStar) || function (mod) {
return result;
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.PYTHON_SPEC = exports.WIN_PERMS_FOLDERS = exports.PROFILES = exports.ENV_VAR_CONDA_PKGS = exports.CONDA_CACHE_FOLDER = exports.CONDARC_PATH = exports.BOOTSTRAP_CONDARC = exports.FORCED_ERRORS = exports.IGNORED_WARNINGS = exports.MAMBA_SUBCOMMANDS = exports.KNOWN_EXTENSIONS = exports.BASE_ENV_NAMES = exports.MINIFORGE_DEFAULT_VERSION = exports.MINIFORGE_DEFAULT_VARIANT = exports.MINIFORGE_URL_PREFIX = exports.OS_NAMES = exports.MINIFORGE_ARCHITECTURES = exports.MINICONDA_ARCHITECTURES = exports.MINICONDA_BASE_URL = exports.IS_UNIX = exports.IS_LINUX = exports.IS_MAC = exports.IS_WINDOWS = exports.MINICONDA_DIR_PATH = void 0;
exports.OUTPUT_ENV_FILE_WAS_PATCHED = exports.OUTPUT_ENV_FILE_CONTENT = exports.OUTPUT_ENV_FILE_PATH = exports.PYTHON_SPEC = exports.WIN_PERMS_FOLDERS = exports.PROFILES = exports.ENV_VAR_CONDA_PKGS = exports.CONDA_CACHE_FOLDER = exports.CONDARC_PATH = exports.BOOTSTRAP_CONDARC = exports.FORCED_ERRORS = exports.IGNORED_WARNINGS = exports.MAMBA_SUBCOMMANDS = exports.KNOWN_EXTENSIONS = exports.BASE_ENV_NAMES = exports.MINIFORGE_DEFAULT_VERSION = exports.MINIFORGE_DEFAULT_VARIANT = exports.MINIFORGE_URL_PREFIX = exports.OS_NAMES = exports.MINIFORGE_ARCHITECTURES = exports.MINICONDA_ARCHITECTURES = exports.MINICONDA_BASE_URL = exports.IS_UNIX = exports.IS_LINUX = exports.IS_MAC = exports.IS_WINDOWS = exports.MINICONDA_DIR_PATH = void 0;
const os = __importStar(__webpack_require__(87));
const path = __importStar(__webpack_require__(622));
//-----------------------------------------------------------------------
Expand Down Expand Up @@ -1288,6 +1288,18 @@ exports.WIN_PERMS_FOLDERS = [
* @see https://docs.conda.io/projects/conda-build/en/latest/resources/package-spec.html#package-match-specifications
*/
exports.PYTHON_SPEC = /^(.*::)?python($|\s\=\<\>\!\|)/i;
/**
* Output name for the effective environment-file path used.
*/
exports.OUTPUT_ENV_FILE_PATH = "environment-file";
/**
* Output name for the effective environment-file file content used.
*/
exports.OUTPUT_ENV_FILE_CONTENT = "environment-file-content";
/**
* Output name for whether the effective environment-file file was patched.
*/
exports.OUTPUT_ENV_FILE_WAS_PATCHED = "environment-file-was-patched";


/***/ }),
Expand Down
54 changes: 49 additions & 5 deletions dist/setup/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -9358,7 +9358,7 @@ var __importStar = (this && this.__importStar) || function (mod) {
return result;
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.PYTHON_SPEC = exports.WIN_PERMS_FOLDERS = exports.PROFILES = exports.ENV_VAR_CONDA_PKGS = exports.CONDA_CACHE_FOLDER = exports.CONDARC_PATH = exports.BOOTSTRAP_CONDARC = exports.FORCED_ERRORS = exports.IGNORED_WARNINGS = exports.MAMBA_SUBCOMMANDS = exports.KNOWN_EXTENSIONS = exports.BASE_ENV_NAMES = exports.MINIFORGE_DEFAULT_VERSION = exports.MINIFORGE_DEFAULT_VARIANT = exports.MINIFORGE_URL_PREFIX = exports.OS_NAMES = exports.MINIFORGE_ARCHITECTURES = exports.MINICONDA_ARCHITECTURES = exports.MINICONDA_BASE_URL = exports.IS_UNIX = exports.IS_LINUX = exports.IS_MAC = exports.IS_WINDOWS = exports.MINICONDA_DIR_PATH = void 0;
exports.OUTPUT_ENV_FILE_WAS_PATCHED = exports.OUTPUT_ENV_FILE_CONTENT = exports.OUTPUT_ENV_FILE_PATH = exports.PYTHON_SPEC = exports.WIN_PERMS_FOLDERS = exports.PROFILES = exports.ENV_VAR_CONDA_PKGS = exports.CONDA_CACHE_FOLDER = exports.CONDARC_PATH = exports.BOOTSTRAP_CONDARC = exports.FORCED_ERRORS = exports.IGNORED_WARNINGS = exports.MAMBA_SUBCOMMANDS = exports.KNOWN_EXTENSIONS = exports.BASE_ENV_NAMES = exports.MINIFORGE_DEFAULT_VERSION = exports.MINIFORGE_DEFAULT_VARIANT = exports.MINIFORGE_URL_PREFIX = exports.OS_NAMES = exports.MINIFORGE_ARCHITECTURES = exports.MINICONDA_ARCHITECTURES = exports.MINICONDA_BASE_URL = exports.IS_UNIX = exports.IS_LINUX = exports.IS_MAC = exports.IS_WINDOWS = exports.MINICONDA_DIR_PATH = void 0;
const os = __importStar(__webpack_require__(87));
const path = __importStar(__webpack_require__(622));
//-----------------------------------------------------------------------
Expand Down Expand Up @@ -9485,6 +9485,18 @@ exports.WIN_PERMS_FOLDERS = [
* @see https://docs.conda.io/projects/conda-build/en/latest/resources/package-spec.html#package-match-specifications
*/
exports.PYTHON_SPEC = /^(.*::)?python($|\s\=\<\>\!\|)/i;
/**
* Output name for the effective environment-file path used.
*/
exports.OUTPUT_ENV_FILE_PATH = "environment-file";
/**
* Output name for the effective environment-file file content used.
*/
exports.OUTPUT_ENV_FILE_CONTENT = "environment-file-content";
/**
* Output name for whether the effective environment-file file was patched.
*/
exports.OUTPUT_ENV_FILE_WAS_PATCHED = "environment-file-was-patched";


/***/ }),
Expand Down Expand Up @@ -13633,6 +13645,7 @@ function parseInputs() {
always_yes: "true",
changeps1: "false",
}),
cleanPatchedEnvironmentFile: core.getInput("clean-patched-environment-file"),
});
const errors = RULES.reduce((errors, rule) => {
const msg = rule(inputs, inputs.condaConfig);
Expand Down Expand Up @@ -15871,6 +15884,7 @@ var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, ge
Object.defineProperty(exports, "__esModule", { value: true });
exports.ensureExplicit = void 0;
const conda = __importStar(__webpack_require__(259));
const outputs = __importStar(__webpack_require__(405));
/**
* Install an environment from an explicit file generated `conda list --explicit`
* or `conda-lock`
Expand All @@ -15879,9 +15893,13 @@ exports.ensureExplicit = {
label: "conda create (from explicit)",
provides: (inputs, options) => __awaiter(void 0, void 0, void 0, function* () { var _a, _b; return !!((_b = (_a = options.envSpec) === null || _a === void 0 ? void 0 : _a.explicit) === null || _b === void 0 ? void 0 : _b.length); }),
condaArgs: (inputs, options) => __awaiter(void 0, void 0, void 0, function* () {
var _c;
if (inputs.pythonVersion) {
throw Error(`'python-version: ${inputs.pythonVersion}' is incompatible with an explicit 'environmentFile`);
}
if ((_c = options.envSpec) === null || _c === void 0 ? void 0 : _c.explicit) {
outputs.setEnvironmentFileOutputs(inputs.environmentFile, options.envSpec.explicit);
}
return [
"create",
...conda.envCommandFlag(inputs),
Expand Down Expand Up @@ -18015,7 +18033,7 @@ var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, ge
});
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.setCacheVariable = exports.setPathVariables = void 0;
exports.setEnvironmentFileOutputs = exports.setCacheVariable = exports.setPathVariables = void 0;
/**
* Modify environment variables and action outputs.
*/
Expand Down Expand Up @@ -18051,6 +18069,16 @@ function setCacheVariable(options) {
});
}
exports.setCacheVariable = setCacheVariable;
/**
* Export the effective environment-file path
*/
function setEnvironmentFileOutputs(envFile, envContent, patched = false) {
core.setOutput(constants.OUTPUT_ENV_FILE_PATH, path.resolve(envFile));
core.setOutput(constants.OUTPUT_ENV_FILE_CONTENT, envContent);
core.setOutput(constants.OUTPUT_ENV_FILE_WAS_PATCHED, patched ? "true" : "false");
core.saveState(constants.OUTPUT_ENV_FILE_WAS_PATCHED, patched);
}
exports.setEnvironmentFileOutputs = setEnvironmentFileOutputs;


/***/ }),
Expand Down Expand Up @@ -20558,6 +20586,18 @@ function setupMiniconda(inputs) {
if (inputs.activateEnvironment) {
yield core.group("Ensuring environment...", () => env.ensureEnvironment(inputs, options));
}
if (core.getState(constants.OUTPUT_ENV_FILE_WAS_PATCHED)) {
yield core.group("Maybe cleaning up patched environment-file...", () => __awaiter(this, void 0, void 0, function* () {
const patchedEnv = core.getState(constants.OUTPUT_ENV_FILE_PATH);
if (inputs.cleanPatchedEnvironmentFile === "true") {
fs.unlinkSync(patchedEnv);
core.info(`Cleaned ${patchedEnv}`);
}
else {
core.info(`Leaving ${patchedEnv} in place`);
}
}));
}
core.info("setup-miniconda ran successfully");
});
}
Expand Down Expand Up @@ -20617,13 +20657,13 @@ var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, ge
Object.defineProperty(exports, "__esModule", { value: true });
exports.ensureYaml = void 0;
const fs = __importStar(__webpack_require__(747));
const os = __importStar(__webpack_require__(87));
const path = __importStar(__webpack_require__(622));
const yaml = __importStar(__webpack_require__(414));
const core = __importStar(__webpack_require__(470));
const constants = __importStar(__webpack_require__(211));
const conda = __importStar(__webpack_require__(259));
const utils = __importStar(__webpack_require__(163));
const outputs = __importStar(__webpack_require__(405));
/**
* The current known providers of patches to `environment.yml`
*
Expand Down Expand Up @@ -20690,13 +20730,17 @@ exports.ensureYaml = {
}
if (patchesApplied.length) {
const patchedYaml = yaml.safeDump(Object.assign(Object.assign({}, yamlData), { dependencies }));
envFile = path.join(os.tmpdir(), "environment-patched.yml");
const origPath = path.resolve(inputs.environmentFile);
const origParent = path.dirname(origPath);
envFile = path.join(origParent, `setup-miniconda-patched-${path.basename(origPath)}`);
core.info(`Making patched copy of 'environment-file: ${inputs.environmentFile}'`);
core.info(patchedYaml);
core.info(`Using: ${envFile}\n${patchedYaml}`);
fs.writeFileSync(envFile, patchedYaml, "utf8");
outputs.setEnvironmentFileOutputs(envFile, yaml.safeDump(patchedYaml), true);
}
else {
core.info(`Using 'environment-file: ${inputs.environmentFile}' as-is`);
outputs.setEnvironmentFileOutputs(envFile, fs.readFileSync(inputs.environmentFile, "utf-8"));
}
return [
"env",
Expand Down
3 changes: 3 additions & 0 deletions etc/example-environment-no-name.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
channels:
- conda-forge
- defaults
dependencies:
- black
- anaconda-client
4 changes: 3 additions & 1 deletion etc/example-environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,7 @@ channels:
- conda-forge
- defaults
dependencies:
- black
- anaconda-client
- pip
- pip:
- -r requirements.txt
1 change: 1 addition & 0 deletions etc/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
isort
15 changes: 15 additions & 0 deletions src/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -148,3 +148,18 @@ export const WIN_PERMS_FOLDERS = [
* @see https://docs.conda.io/projects/conda-build/en/latest/resources/package-spec.html#package-match-specifications
*/
export const PYTHON_SPEC = /^(.*::)?python($|\s\=\<\>\!\|)/i;

/**
* Output name for the effective environment-file path used.
*/
export const OUTPUT_ENV_FILE_PATH = "environment-file";

/**
* Output name for the effective environment-file file content used.
*/
export const OUTPUT_ENV_FILE_CONTENT = "environment-file-content";

/**
* Output name for whether the effective environment-file file was patched.
*/
export const OUTPUT_ENV_FILE_WAS_PATCHED = "environment-file-was-patched";
8 changes: 8 additions & 0 deletions src/env/explicit.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import * as types from "../types";
import * as conda from "../conda";
import * as outputs from "../outputs";

/**
* Install an environment from an explicit file generated `conda list --explicit`
Expand All @@ -15,6 +16,13 @@ export const ensureExplicit: types.IEnvProvider = {
);
}

if (options.envSpec?.explicit) {
outputs.setEnvironmentFileOutputs(
inputs.environmentFile,
options.envSpec.explicit
);
}

return [
"create",
...conda.envCommandFlag(inputs),
Expand Down
20 changes: 17 additions & 3 deletions src/env/yaml.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import * as fs from "fs";
import * as os from "os";
import * as path from "path";

import * as yaml from "js-yaml";
Expand All @@ -9,6 +8,7 @@ import * as types from "../types";
import * as constants from "../constants";
import * as conda from "../conda";
import * as utils from "../utils";
import * as outputs from "../outputs";

/**
* Describes whether and how a YAML env should be patched to add a specific package
Expand Down Expand Up @@ -108,14 +108,28 @@ export const ensureYaml: types.IEnvProvider = {

if (patchesApplied.length) {
const patchedYaml = yaml.safeDump({ ...yamlData, dependencies });
envFile = path.join(os.tmpdir(), "environment-patched.yml");
const origPath = path.resolve(inputs.environmentFile);
const origParent = path.dirname(origPath);
envFile = path.join(
origParent,
`setup-miniconda-patched-${path.basename(origPath)}`
);
core.info(
`Making patched copy of 'environment-file: ${inputs.environmentFile}'`
);
core.info(patchedYaml);
core.info(`Using: ${envFile}\n${patchedYaml}`);
fs.writeFileSync(envFile, patchedYaml, "utf8");
outputs.setEnvironmentFileOutputs(
envFile,
yaml.safeDump(patchedYaml),
true
);
} else {
core.info(`Using 'environment-file: ${inputs.environmentFile}' as-is`);
outputs.setEnvironmentFileOutputs(
envFile,
fs.readFileSync(inputs.environmentFile, "utf-8")
);
}

return [
Expand Down
Loading