Show how to parse params into stage cmd #3913
Conversation
@alex000kim showed me how to do this for a project in which I'm using Papermill to run an entire notebook from a single stage. As far as I could find, this approach isn't yet covered by the docs. Thought it might be worth highlighting, even though it's not always the most proper approach.
RCdeWit
added
A: docs
It is documented but quite "hidden": #3670
I think it's a valid and proper approach in many cases. In fact, I consider it a "signature" feature that allows you to track and use parameters in a non-intrusive manner (no need to alter your code to load params as long as you had set up argument parsing) |
Link Check ReportThere were no links to check! |
|
Ah, thanks for pointing that out, @daavoo! In that case, a link to the relevant section would probably suffice here! |
| You can also parse parameters directly from `params.yaml` into the command that | ||
| DVC executes in a stage: | ||
|
|
||
| ```yaml | ||
| stages: | ||
| train: | ||
| cmd: python train_model.py ${seed} model.p | ||
| deps: | ||
| - train_model.py | ||
| - matrix-train.p | ||
| params: | ||
| - seed |
There was a problem hiding this comment.
Wat. I didn't even know this was a thing! It should be in https://dvc.org/doc/user-guide/project-structure/dvcyaml-files TBH.
Not sure it makes sense to have it in a command reference example. Maybe just mentioning templating in general in the stage add ref (if it's not now).
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
➕ @RCdeWit |
Co-authored-by: Restyled.io <[email protected]>
|
How about an |
Co-authored-by: Restyled.io <[email protected]>
shcheklein
requested a deployment
to
dvc-org-param-parsing-c-qo7k9r
jorgeorpinel
added
the
type: enhancement
@alex000kim showed me how to do this for a project in which I'm using Papermill to run an entire notebook from a single stage. As far as I could find, this approach isn't yet covered by the docs. Thought it might be worth highlighting, even though it's not always the most proper approach.