From 9bbfa37d86c7f79b7ae12d9f3128ba4ada3b6922 Mon Sep 17 00:00:00 2001 From: Saugat Pachhai Date: Thu, 11 Nov 2021 19:09:44 +0545 Subject: [PATCH 1/9] cmd-ref: document exp init --- content/docs/command-reference/exp/index.md | 4 +- content/docs/command-reference/exp/init.md | 122 ++++++++++++++++++++ content/docs/sidebar.json | 4 + 3 files changed, 129 insertions(+), 1 deletion(-) create mode 100644 content/docs/command-reference/exp/init.md diff --git a/content/docs/command-reference/exp/index.md b/content/docs/command-reference/exp/index.md index d9f32671de..b177173bd2 100644 --- a/content/docs/command-reference/exp/index.md +++ b/content/docs/command-reference/exp/index.md @@ -4,6 +4,7 @@ _New in DVC 2.0 (see `dvc version`)_ A set of commands to generate and manage experiments: [run](/doc/command-reference/exp/run), [show](/doc/command-reference/exp/show), +[init](/doc/command-reference/exp/init), [diff](/doc/command-reference/exp/diff), [apply](/doc/command-reference/exp/apply), [branch](/doc/command-reference/exp/branch), @@ -20,7 +21,7 @@ A set of commands to generate and manage experiments: ```usage usage: dvc exp [-h] [-q | -v] - {show,apply,diff,run,gc,branch,list,push,pull,remove} + {show,apply,diff,run,gc,branch,list,push,pull,remove,init} ... positional arguments: @@ -37,6 +38,7 @@ positional arguments: push Push a local experiment to a Git remote. pull Pull an experiment from a Git remote. remove Remove local experiments. + init Initialize experiments. ``` ## Description diff --git a/content/docs/command-reference/exp/init.md b/content/docs/command-reference/exp/init.md new file mode 100644 index 0000000000..1596bf390d --- /dev/null +++ b/content/docs/command-reference/exp/init.md @@ -0,0 +1,122 @@ +# exp init + +Initializes experiments. + +## Synopsis + +```usage +usage: dvc exp init [-h] [-q | -v] [--run] [--interactive] [-f] + [--explicit] [--name NAME] [--code CODE] + [--data DATA] [--models MODELS] [--params PARAMS] + [--metrics METRICS] [--plots PLOTS] [--live LIVE] + [--type {default,live}] + [command] +``` + +## Description + +`dvc exp init` helps you quickly get started with experiments, with the command to +run and tracks data, params, source code, models, metrics and plots by default. + +It reduces boilerplate for initializing [pipeline](/doc/command-reference/dag) +stages in a `dvc.yaml` file by assuming sane defaults about the location of your +data, parameters, source code, models, metrics and plots, which can be customized +through config. + +It also offers guided `--interactive` mode for creating a stage to be `exp run` later. +`dvc exp init` supports creating different types of stages, eg: live if you are using +dvclive to track metrics and plots, etc. + +### The `command` argument + +The `command` argument is optional, if you are using `--interactive` mode. +The `command` sent to `dvc exp init` can be anything your terminal would accept and +run directly, for example a shell built-in, expression, or binary found in +`PATH`. Please remember that any flags sent after the `command` are interpreted +by the command itself, not by `dvc exp init`. + +⚠️ While DVC is platform-agnostic, the commands defined in your +[pipeline](/doc/command-reference/dag) stages may only work on some operating +systems and require certain software packages to be installed. + +Wrap the command with double quotes `"` if there are special characters in it +like `|` (pipe) or `<`, `>` (redirection), otherwise they would apply to +`dvc run` itself. Use single quotes `'` instead if there are environment +variables in it that should be evaluated dynamically. Examples: + +```dvc +$ dvc exp init "./a_script.sh > /dev/null 2>&1" +$ dvc exp init './another_script.sh $MYENVVAR' +``` + + +## Options + +- `-i`, `--interactive` - prompts user for the command to execute and different paths for tracking + outputs and dependencies, unless they are provided through cli arguments explicitly. Interactive mode allows + users to set those location from default values or omit them. + +- `--explicit` - `dvc exp init` assumes default location of your outputs and dependencies (which can be + overriden from the config). By using `--explicit`, it will not use those default values while initializing + experiments. + In `--interactive` mode, prompt won't set default value and all the values for the prompt needs to be explicitly provided, or omitted. + +- `--code` - override the a path to your source file or directory which your experiment depends on. + The default is `src` directory for your code. + +- `--data` - override the path to your data file or directory to track, which your experiment depends on. + The default is `data` directory. + +- `--model` - override the path to your models file or directory to track, which your experiment depends on. + `dvc exp init` assumes `models` directory by default. + +- `--params` - override the path to [parameter dependencies](/doc/command-reference/params) which your experiment depends on. + The default parameters file name is `params.yaml`. Note that `dvc exp init` may fail if the parameters file does not exist + at the time of the invocation. + +- `--metrics` - override the path to metrics file to track, which your experiment produces. Default is `metrics.json` file. + +- `--plots` - override the path to plots file or directory, which your experiment produces. The default is `plots`. + +- `--live` - override the directory `path` for [DVCLive](/doc/dvclive/user-guide/dvclive-with-dvc), which your experiment will write logs to. + The default is `dvclive` directory, which only comes to effect when used with `--type=live`. + +- `--type` - selects the type of the stage to create. Currently it provides two different kinds of stages: `default` and `live`. + If unspecified, `default` stage is created. + + `default` stage creates a stage with `metrics` and `plots` tracked by DVC itself, + and does not track live-created artifacts (unless explicitly specified). + + With `--type=live`, `dvc exp init` will create a stage where `metrics` and `plots` are tracked + by `dvclive` itself (unless `--metrics` and `--plots` are explicitly provided). + +- `-n `, `--name ` - specify a custom name for the stage + generated by this command (e.g. `-n train`). By default, the name of the stage depends on `--type` of the + stage that is being created. If `--type=default, the name of the stage will be `default`, and in case of + `--type=live`, the name of the stage will be `live`. + + Note that the stage name can only contain letters, numbers, dash `-` and underscore `_`. + +- `-f`, `--force` - overwrite an existing stage in `dvc.yaml` file without asking for confirmation. + +- `--run` - runs the experiment after initializing it. + +- `-h`, `--help` - prints the usage/help message, and exit. + +- `-q`, `--quiet` - do not write anything to standard output. Exit with 0 if no + problems arise, otherwise 1. + +- `-v`, `--verbose` - displays detailed tracing information. + +## Setting up custom config/Sharing configs + +## Non-interactive mode + +## Guided/Interactive mode + +## Examples + +#### Default stage + +#### Dvclive stage +diff --git a/content/docs/sidebar.json b/content/docs/sidebar.json diff --git a/content/docs/sidebar.json b/content/docs/sidebar.json index 4621d1e014..4bafa68591 100644 --- a/content/docs/sidebar.json +++ b/content/docs/sidebar.json @@ -260,6 +260,10 @@ "label": "exp show", "slug": "show" }, + { + "label": "exp init", + "slug": "init" + }, { "label": "exp diff", "slug": "diff" From d5aacacdcd40dc5d809ca9009d0e53e4dba6930e Mon Sep 17 00:00:00 2001 From: "restyled-io[bot]" <32688539+restyled-io[bot]@users.noreply.github.com> Date: Thu, 11 Nov 2021 19:11:14 +0545 Subject: [PATCH 2/9] Restyle [WIP] cmd-ref: document exp init (#3016) Co-authored-by: Restyled.io --- content/docs/command-reference/exp/init.md | 103 ++++++++++++--------- 1 file changed, 59 insertions(+), 44 deletions(-) diff --git a/content/docs/command-reference/exp/init.md b/content/docs/command-reference/exp/init.md index 1596bf390d..447267479f 100644 --- a/content/docs/command-reference/exp/init.md +++ b/content/docs/command-reference/exp/init.md @@ -1,7 +1,7 @@ # exp init Initializes experiments. - + ## Synopsis ```usage @@ -15,22 +15,23 @@ usage: dvc exp init [-h] [-q | -v] [--run] [--interactive] [-f] ## Description -`dvc exp init` helps you quickly get started with experiments, with the command to -run and tracks data, params, source code, models, metrics and plots by default. +`dvc exp init` helps you quickly get started with experiments, with the command +to run and tracks data, params, source code, models, metrics and plots by +default. It reduces boilerplate for initializing [pipeline](/doc/command-reference/dag) stages in a `dvc.yaml` file by assuming sane defaults about the location of your -data, parameters, source code, models, metrics and plots, which can be customized -through config. +data, parameters, source code, models, metrics and plots, which can be +customized through config. -It also offers guided `--interactive` mode for creating a stage to be `exp run` later. -`dvc exp init` supports creating different types of stages, eg: live if you are using -dvclive to track metrics and plots, etc. +It also offers guided `--interactive` mode for creating a stage to be `exp run` +later. `dvc exp init` supports creating different types of stages, eg: live if +you are using dvclive to track metrics and plots, etc. ### The `command` argument -The `command` argument is optional, if you are using `--interactive` mode. -The `command` sent to `dvc exp init` can be anything your terminal would accept and +The `command` argument is optional, if you are using `--interactive` mode. The +`command` sent to `dvc exp init` can be anything your terminal would accept and run directly, for example a shell built-in, expression, or binary found in `PATH`. Please remember that any flags sent after the `command` are interpreted by the command itself, not by `dvc exp init`. @@ -49,55 +50,68 @@ $ dvc exp init "./a_script.sh > /dev/null 2>&1" $ dvc exp init './another_script.sh $MYENVVAR' ``` - ## Options -- `-i`, `--interactive` - prompts user for the command to execute and different paths for tracking - outputs and dependencies, unless they are provided through cli arguments explicitly. Interactive mode allows - users to set those location from default values or omit them. +- `-i`, `--interactive` - prompts user for the command to execute and different + paths for tracking outputs and dependencies, unless they are provided through + cli arguments explicitly. Interactive mode allows users to set those location + from default values or omit them. -- `--explicit` - `dvc exp init` assumes default location of your outputs and dependencies (which can be - overriden from the config). By using `--explicit`, it will not use those default values while initializing - experiments. - In `--interactive` mode, prompt won't set default value and all the values for the prompt needs to be explicitly provided, or omitted. +- `--explicit` - `dvc exp init` assumes default location of your outputs and + dependencies (which can be overriden from the config). By using `--explicit`, + it will not use those default values while initializing experiments. In + `--interactive` mode, prompt won't set default value and all the values for + the prompt needs to be explicitly provided, or omitted. -- `--code` - override the a path to your source file or directory which your experiment depends on. - The default is `src` directory for your code. +- `--code` - override the a path to your source file or directory which your + experiment depends on. The default is `src` directory for your code. -- `--data` - override the path to your data file or directory to track, which your experiment depends on. - The default is `data` directory. +- `--data` - override the path to your data file or directory to track, which + your experiment depends on. The default is `data` directory. -- `--model` - override the path to your models file or directory to track, which your experiment depends on. - `dvc exp init` assumes `models` directory by default. +- `--model` - override the path to your models file or directory to track, which + your experiment depends on. `dvc exp init` assumes `models` directory by + default. -- `--params` - override the path to [parameter dependencies](/doc/command-reference/params) which your experiment depends on. - The default parameters file name is `params.yaml`. Note that `dvc exp init` may fail if the parameters file does not exist - at the time of the invocation. +- `--params` - override the path to + [parameter dependencies](/doc/command-reference/params) which your experiment + depends on. The default parameters file name is `params.yaml`. Note that + `dvc exp init` may fail if the parameters file does not exist at the time of + the invocation. -- `--metrics` - override the path to metrics file to track, which your experiment produces. Default is `metrics.json` file. +- `--metrics` - override the path to metrics file to track, which your + experiment produces. Default is `metrics.json` file. -- `--plots` - override the path to plots file or directory, which your experiment produces. The default is `plots`. +- `--plots` - override the path to plots file or directory, which your + experiment produces. The default is `plots`. -- `--live` - override the directory `path` for [DVCLive](/doc/dvclive/user-guide/dvclive-with-dvc), which your experiment will write logs to. - The default is `dvclive` directory, which only comes to effect when used with `--type=live`. +- `--live` - override the directory `path` for + [DVCLive](/doc/dvclive/user-guide/dvclive-with-dvc), which your experiment + will write logs to. The default is `dvclive` directory, which only comes to + effect when used with `--type=live`. -- `--type` - selects the type of the stage to create. Currently it provides two different kinds of stages: `default` and `live`. - If unspecified, `default` stage is created. - - `default` stage creates a stage with `metrics` and `plots` tracked by DVC itself, - and does not track live-created artifacts (unless explicitly specified). +- `--type` - selects the type of the stage to create. Currently it provides two + different kinds of stages: `default` and `live`. If unspecified, `default` + stage is created. - With `--type=live`, `dvc exp init` will create a stage where `metrics` and `plots` are tracked - by `dvclive` itself (unless `--metrics` and `--plots` are explicitly provided). + `default` stage creates a stage with `metrics` and `plots` tracked by DVC + itself, and does not track live-created artifacts (unless explicitly + specified). -- `-n `, `--name ` - specify a custom name for the stage - generated by this command (e.g. `-n train`). By default, the name of the stage depends on `--type` of the - stage that is being created. If `--type=default, the name of the stage will be `default`, and in case of - `--type=live`, the name of the stage will be `live`. + With `--type=live`, `dvc exp init` will create a stage where `metrics` and + `plots` are tracked by `dvclive` itself (unless `--metrics` and `--plots` are + explicitly provided). - Note that the stage name can only contain letters, numbers, dash `-` and underscore `_`. +- `-n `, `--name ` - specify a custom name for the stage generated + by this command (e.g. `-n train`). By default, the name of the stage depends + on `--type` of the stage that is being created. If + `--type=default, the name of the stage will be `default`, and in case of `--type=live`, the name of the stage will be `live`. -- `-f`, `--force` - overwrite an existing stage in `dvc.yaml` file without asking for confirmation. + Note that the stage name can only contain letters, numbers, dash `-` and + underscore `_`. + +- `-f`, `--force` - overwrite an existing stage in `dvc.yaml` file without + asking for confirmation. - `--run` - runs the experiment after initializing it. @@ -119,4 +133,5 @@ $ dvc exp init './another_script.sh $MYENVVAR' #### Default stage #### Dvclive stage + diff --git a/content/docs/sidebar.json b/content/docs/sidebar.json From b35d5a694b3be06e274a34c69876a54c40d1df98 Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Thu, 11 Nov 2021 13:32:52 -0600 Subject: [PATCH 3/9] Apply suggestions from code review --- content/docs/command-reference/exp/init.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/content/docs/command-reference/exp/init.md b/content/docs/command-reference/exp/init.md index 447267479f..b92ae02864 100644 --- a/content/docs/command-reference/exp/init.md +++ b/content/docs/command-reference/exp/init.md @@ -86,7 +86,7 @@ $ dvc exp init './another_script.sh $MYENVVAR' experiment produces. The default is `plots`. - `--live` - override the directory `path` for - [DVCLive](/doc/dvclive/user-guide/dvclive-with-dvc), which your experiment + [DVCLive](/doc/dvclive/dvclive-with-dvc), which your experiment will write logs to. The default is `dvclive` directory, which only comes to effect when used with `--type=live`. @@ -133,5 +133,3 @@ $ dvc exp init './another_script.sh $MYENVVAR' #### Default stage #### Dvclive stage - -diff --git a/content/docs/sidebar.json b/content/docs/sidebar.json From 4d00bb89a8acef4bf0a189be281467c1a2c08bbf Mon Sep 17 00:00:00 2001 From: "restyled-io[bot]" <32688539+restyled-io[bot]@users.noreply.github.com> Date: Fri, 26 Nov 2021 16:30:01 -0800 Subject: [PATCH 4/9] Restyled by prettier (#3018) Co-authored-by: Restyled.io --- content/docs/command-reference/exp/init.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/docs/command-reference/exp/init.md b/content/docs/command-reference/exp/init.md index b92ae02864..b1674d8ee6 100644 --- a/content/docs/command-reference/exp/init.md +++ b/content/docs/command-reference/exp/init.md @@ -86,9 +86,9 @@ $ dvc exp init './another_script.sh $MYENVVAR' experiment produces. The default is `plots`. - `--live` - override the directory `path` for - [DVCLive](/doc/dvclive/dvclive-with-dvc), which your experiment - will write logs to. The default is `dvclive` directory, which only comes to - effect when used with `--type=live`. + [DVCLive](/doc/dvclive/dvclive-with-dvc), which your experiment will write + logs to. The default is `dvclive` directory, which only comes to effect when + used with `--type=live`. - `--type` - selects the type of the stage to create. Currently it provides two different kinds of stages: `default` and `live`. If unspecified, `default` From 7d701badb49b4e3197903b911fefe6eba8d16b11 Mon Sep 17 00:00:00 2001 From: Jorge Orpinel Date: Mon, 29 Nov 2021 16:59:42 -0600 Subject: [PATCH 5/9] Update content/docs/command-reference/exp/index.md --- content/docs/command-reference/exp/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/command-reference/exp/index.md b/content/docs/command-reference/exp/index.md index b177173bd2..94dee1f869 100644 --- a/content/docs/command-reference/exp/index.md +++ b/content/docs/command-reference/exp/index.md @@ -3,8 +3,8 @@ _New in DVC 2.0 (see `dvc version`)_ A set of commands to generate and manage experiments: -[run](/doc/command-reference/exp/run), [show](/doc/command-reference/exp/show), [init](/doc/command-reference/exp/init), +[run](/doc/command-reference/exp/run), [show](/doc/command-reference/exp/show), [diff](/doc/command-reference/exp/diff), [apply](/doc/command-reference/exp/apply), [branch](/doc/command-reference/exp/branch), From e78d569062819f823ce1ee9902ff75395af97976 Mon Sep 17 00:00:00 2001 From: "restyled-io[bot]" <32688539+restyled-io[bot]@users.noreply.github.com> Date: Mon, 29 Nov 2021 17:00:32 -0600 Subject: [PATCH 6/9] Restyled by prettier (#3048) Co-authored-by: Restyled.io --- content/docs/command-reference/exp/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/command-reference/exp/index.md b/content/docs/command-reference/exp/index.md index 94dee1f869..066411ab70 100644 --- a/content/docs/command-reference/exp/index.md +++ b/content/docs/command-reference/exp/index.md @@ -3,8 +3,8 @@ _New in DVC 2.0 (see `dvc version`)_ A set of commands to generate and manage experiments: -[init](/doc/command-reference/exp/init), -[run](/doc/command-reference/exp/run), [show](/doc/command-reference/exp/show), +[init](/doc/command-reference/exp/init), [run](/doc/command-reference/exp/run), +[show](/doc/command-reference/exp/show), [diff](/doc/command-reference/exp/diff), [apply](/doc/command-reference/exp/apply), [branch](/doc/command-reference/exp/branch), From 229baf59f1baf1aaade2f6410ac7c174e64535fe Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Saugat=20Pachhai=20=28=E0=A4=B8=E0=A5=8C=E0=A4=97=E0=A4=BE?= =?UTF-8?q?=E0=A4=A4=29?= Date: Thu, 2 Dec 2021 20:32:05 +0545 Subject: [PATCH 7/9] updates --- content/docs/command-reference/exp/init.md | 44 ++++++++++++---------- 1 file changed, 24 insertions(+), 20 deletions(-) diff --git a/content/docs/command-reference/exp/init.md b/content/docs/command-reference/exp/init.md index b1674d8ee6..0cdae0bd09 100644 --- a/content/docs/command-reference/exp/init.md +++ b/content/docs/command-reference/exp/init.md @@ -15,18 +15,21 @@ usage: dvc exp init [-h] [-q | -v] [--run] [--interactive] [-f] ## Description -`dvc exp init` helps you quickly get started with experiments, with the command -to run and tracks data, params, source code, models, metrics and plots by -default. - -It reduces boilerplate for initializing [pipeline](/doc/command-reference/dag) -stages in a `dvc.yaml` file by assuming sane defaults about the location of your -data, parameters, source code, models, metrics and plots, which can be -customized through config. - -It also offers guided `--interactive` mode for creating a stage to be `exp run` -later. `dvc exp init` supports creating different types of stages, eg: live if -you are using dvclive to track metrics and plots, etc. +`dvc exp init` helps you quickly get started with experiments. It reduces +boilerplate for initializing [pipeline](/doc/command-reference/dag) stages in a +`dvc.yaml` file by assuming sane defaults about the location of your data, +[parameters](/doc/command-reference/params), source code, +[models](/doc/command-reference/), [metrics](/doc/command-reference/metrics) and +[plots](/doc/command-reference/plots), which can be customized through config. + +It also offers guided `--interactive` mode for creating a stage to be +[`exp run`](/doc/command-reference/exp/run) later. `dvc exp init` supports +creating different types of stages, eg: live if you are using +[dvclive](/doc/dvclive) to monitor and checkpoint progress during training of +machine learning models. + +This command is intended to be light-weight and simple and lacks many bells and +whistles that `dvc stage add` provides. ### The `command` argument @@ -77,7 +80,8 @@ $ dvc exp init './another_script.sh $MYENVVAR' [parameter dependencies](/doc/command-reference/params) which your experiment depends on. The default parameters file name is `params.yaml`. Note that `dvc exp init` may fail if the parameters file does not exist at the time of - the invocation. + the invocation, as DVC reads the file to find parameters to track for the + stage. - `--metrics` - override the path to metrics file to track, which your experiment produces. Default is `metrics.json` file. @@ -85,10 +89,9 @@ $ dvc exp init './another_script.sh $MYENVVAR' - `--plots` - override the path to plots file or directory, which your experiment produces. The default is `plots`. -- `--live` - override the directory `path` for - [DVCLive](/doc/dvclive/dvclive-with-dvc), which your experiment will write - logs to. The default is `dvclive` directory, which only comes to effect when - used with `--type=live`. +- `--live` - override the directory `path` for [DVCLive](/doc/dvclive), which + your experiment will write logs to. The default is `dvclive` directory, which + only comes to effect when used with `--type=live`. - `--type` - selects the type of the stage to create. Currently it provides two different kinds of stages: `default` and `live`. If unspecified, `default` @@ -98,9 +101,10 @@ $ dvc exp init './another_script.sh $MYENVVAR' itself, and does not track live-created artifacts (unless explicitly specified). - With `--type=live`, `dvc exp init` will create a stage where `metrics` and - `plots` are tracked by `dvclive` itself (unless `--metrics` and `--plots` are - explicitly provided). + `live` stage is intended for use in deep-learning scenarios, where metrics and + plots are tracked by [dvclive](/doc/dvclive) and supports tracking progress + while training a deep-learning model with + [checkpoints](/doc/command-reference/exp/run#checkpoints). - `-n `, `--name ` - specify a custom name for the stage generated by this command (e.g. `-n train`). By default, the name of the stage depends From 91d06dcb00d9bb7d85f8655bea4e318800d1669b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Saugat=20Pachhai=20=28=E0=A4=B8=E0=A5=8C=E0=A4=97=E0=A4=BE?= =?UTF-8?q?=E0=A4=A4=29?= Date: Thu, 2 Dec 2021 22:38:16 +0545 Subject: [PATCH 8/9] document config --- content/docs/command-reference/config.md | 24 ++++++++++++++++++++++ content/docs/command-reference/exp/init.md | 21 ++++++++++++++++++- 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/content/docs/command-reference/config.md b/content/docs/command-reference/config.md index ebdeb39fc6..58ed471374 100644 --- a/content/docs/command-reference/config.md +++ b/content/docs/command-reference/config.md @@ -221,6 +221,30 @@ connection settings, and configuring a remote is the way that can be done. > hash overlaps: the hash of an external output could collide with > that of a local file with different content. +### exp + +This section overrides default configured workspace paths in `dvc exp init`, +that helps to avoid repeating these paths if all of your projects share a +similar structure. + +The section contains following options, which are only used as a default and can +be overidden explicitly through CLI arguments or through responses in prompts +(in `--interactive` mode). + +- `exp.code` - path to your source file or directory. + +- `exp.data` - path to your data file or directory to track. + +- `exp.models` - path to your models file or directory. + +- `exp.metrics` - path to your metrics file. + +- `exp.params` - path to your parameters file. + +- `exp.plots` - path to your plots file or directory. + +- `exp.live` - path to your dvclive outputs. + ### state > 📖 See diff --git a/content/docs/command-reference/exp/init.md b/content/docs/command-reference/exp/init.md index 0cdae0bd09..e40468f917 100644 --- a/content/docs/command-reference/exp/init.md +++ b/content/docs/command-reference/exp/init.md @@ -126,7 +126,26 @@ $ dvc exp init './another_script.sh $MYENVVAR' - `-v`, `--verbose` - displays detailed tracing information. -## Setting up custom config/Sharing configs +### Setting up custom paths + +`dvc exp init` supports +[setting up custom workspace paths by setting up DVC config](/doc/command-reference/config#exp), +where you can add an `exp` section to the config file to point to the paths for +your code, data, models, parameters, metrics, plots (either images or tabular +data to be plotted), and dvclive outputs. + +```dvc +$ dvc config exp.data datasets/ +$ dvc config exp.params config.yaml +$ dvc config exp.code scripts/train.py +$ dvc config exp.models trained_models/ +$ dvc config exp.metrics reports.json +$ dvc config exp.plots viz/ +``` + +You can leave some configurations to use default values. This can be useful in a +system or global config so that you can avoid repeating these paths if all of +your projects share a similar structure. ## Non-interactive mode From 16f3f4e837aa8c2cb769fb4322056df994f43298 Mon Sep 17 00:00:00 2001 From: dberenbaum Date: Mon, 6 Dec 2021 15:51:27 -0500 Subject: [PATCH 9/9] cmd-ref exp init: updates for initial merge --- content/docs/command-reference/exp/init.md | 73 +++++++--------------- 1 file changed, 21 insertions(+), 52 deletions(-) diff --git a/content/docs/command-reference/exp/init.md b/content/docs/command-reference/exp/init.md index e40468f917..ba7be8fe12 100644 --- a/content/docs/command-reference/exp/init.md +++ b/content/docs/command-reference/exp/init.md @@ -1,6 +1,6 @@ # exp init -Initializes experiments. +Codify project using [DVC metafiles] to run [experiments]. ## Synopsis @@ -9,7 +9,7 @@ usage: dvc exp init [-h] [-q | -v] [--run] [--interactive] [-f] [--explicit] [--name NAME] [--code CODE] [--data DATA] [--models MODELS] [--params PARAMS] [--metrics METRICS] [--plots PLOTS] [--live LIVE] - [--type {default,live}] + [--type {default,dl}] [command] ``` @@ -17,19 +17,19 @@ usage: dvc exp init [-h] [-q | -v] [--run] [--interactive] [-f] `dvc exp init` helps you quickly get started with experiments. It reduces boilerplate for initializing [pipeline](/doc/command-reference/dag) stages in a -`dvc.yaml` file by assuming sane defaults about the location of your data, -[parameters](/doc/command-reference/params), source code, -[models](/doc/command-reference/), [metrics](/doc/command-reference/metrics) and +`dvc.yaml` file by assuming defaults about the location of your data, +[parameters](/doc/command-reference/params), source code, models, +[metrics](/doc/command-reference/metrics) and [plots](/doc/command-reference/plots), which can be customized through config. It also offers guided `--interactive` mode for creating a stage to be [`exp run`](/doc/command-reference/exp/run) later. `dvc exp init` supports -creating different types of stages, eg: live if you are using -[dvclive](/doc/dvclive) to monitor and checkpoint progress during training of -machine learning models. +creating different types of stages, eg: `dl` if you are doing deep learning, +which uses [dvclive](/doc/dvclive) to monitor and checkpoint progress during +training of machine learning models. -This command is intended to be light-weight and simple and lacks many bells and -whistles that `dvc stage add` provides. +This command is intended to be a quick way to start running experiments. To +create more complex stages and pipeliens, use `dvc stage add`. ### The `command` argument @@ -45,7 +45,7 @@ systems and require certain software packages to be installed. Wrap the command with double quotes `"` if there are special characters in it like `|` (pipe) or `<`, `>` (redirection), otherwise they would apply to -`dvc run` itself. Use single quotes `'` instead if there are environment +`dvc exp init` itself. Use single quotes `'` instead if there are environment variables in it that should be evaluated dynamically. Examples: ```dvc @@ -57,7 +57,7 @@ $ dvc exp init './another_script.sh $MYENVVAR' - `-i`, `--interactive` - prompts user for the command to execute and different paths for tracking outputs and dependencies, unless they are provided through - cli arguments explicitly. Interactive mode allows users to set those location + arguments explicitly. Interactive mode allows users to set those locations from default values or omit them. - `--explicit` - `dvc exp init` assumes default location of your outputs and @@ -72,10 +72,6 @@ $ dvc exp init './another_script.sh $MYENVVAR' - `--data` - override the path to your data file or directory to track, which your experiment depends on. The default is `data` directory. -- `--model` - override the path to your models file or directory to track, which - your experiment depends on. `dvc exp init` assumes `models` directory by - default. - - `--params` - override the path to [parameter dependencies](/doc/command-reference/params) which your experiment depends on. The default parameters file name is `params.yaml`. Note that @@ -83,6 +79,10 @@ $ dvc exp init './another_script.sh $MYENVVAR' the invocation, as DVC reads the file to find parameters to track for the stage. +- `--model` - override the path to your models file or directory to track, which + your experiment produces. `dvc exp init` assumes `models` directory by + default. + - `--metrics` - override the path to metrics file to track, which your experiment produces. Default is `metrics.json` file. @@ -91,17 +91,17 @@ $ dvc exp init './another_script.sh $MYENVVAR' - `--live` - override the directory `path` for [DVCLive](/doc/dvclive), which your experiment will write logs to. The default is `dvclive` directory, which - only comes to effect when used with `--type=live`. + only comes to effect when used with `--type=dl`. - `--type` - selects the type of the stage to create. Currently it provides two - different kinds of stages: `default` and `live`. If unspecified, `default` - stage is created. + different kinds of stages: `default` and `dl`. If unspecified, `default` stage + is created. `default` stage creates a stage with `metrics` and `plots` tracked by DVC itself, and does not track live-created artifacts (unless explicitly specified). - `live` stage is intended for use in deep-learning scenarios, where metrics and + `dl` stage is intended for use in deep-learning scenarios, where metrics and plots are tracked by [dvclive](/doc/dvclive) and supports tracking progress while training a deep-learning model with [checkpoints](/doc/command-reference/exp/run#checkpoints). @@ -109,7 +109,7 @@ $ dvc exp init './another_script.sh $MYENVVAR' - `-n `, `--name ` - specify a custom name for the stage generated by this command (e.g. `-n train`). By default, the name of the stage depends on `--type` of the stage that is being created. If - `--type=default, the name of the stage will be `default`, and in case of `--type=live`, the name of the stage will be `live`. + `--type=default, the name of the stage will be `default`, and in case of `--type=dl`, the name of the stage will be `dl`. Note that the stage name can only contain letters, numbers, dash `-` and underscore `_`. @@ -125,34 +125,3 @@ $ dvc exp init './another_script.sh $MYENVVAR' problems arise, otherwise 1. - `-v`, `--verbose` - displays detailed tracing information. - -### Setting up custom paths - -`dvc exp init` supports -[setting up custom workspace paths by setting up DVC config](/doc/command-reference/config#exp), -where you can add an `exp` section to the config file to point to the paths for -your code, data, models, parameters, metrics, plots (either images or tabular -data to be plotted), and dvclive outputs. - -```dvc -$ dvc config exp.data datasets/ -$ dvc config exp.params config.yaml -$ dvc config exp.code scripts/train.py -$ dvc config exp.models trained_models/ -$ dvc config exp.metrics reports.json -$ dvc config exp.plots viz/ -``` - -You can leave some configurations to use default values. This can be useful in a -system or global config so that you can avoid repeating these paths if all of -your projects share a similar structure. - -## Non-interactive mode - -## Guided/Interactive mode - -## Examples - -#### Default stage - -#### Dvclive stage