Rules for model output filenames/filepaths, especially with respect to round_id
identification
#13
Replies: 6 comments 2 replies
-
To answer your questions from the US Scenario Modeling Hub:
|
Beta Was this translation helpful? Give feedback.
-
I could see an argument for requiring that the first "part" of a filename be a Mentioning that the conversation |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
I agree with both Nick and Evan |
Beta Was this translation helpful? Give feedback.
-
Thanks all! I've made note of the comments so far and fully agree.
Having said that, One thing I think is important to enforce is adding the |
Beta Was this translation helpful? Give feedback.
-
Transferring some pertinent slack conversations here: evan Anna Krystalli
evan Anna Krystalli Right! That's what I vaguely remembered too which why the use case of avoiding round id information in file names confused me and threw up all these implementation questions! |
Beta Was this translation helpful? Give feedback.
-
Background
An important part of validation consists of validating the submission file name and location within the hub. The file name, it appears, can also be the single source of a
round_id
in the case of hubs whereround_id_from_variable
is false.And given each submission file is targeting a specific round, it is important to be able to accurately determine the
round_id
of a file, not only to validate it but to determine which schema to use to validate the file itself which may well be round dependant. So by extension, having a solid understanding of the expectations of file names and file paths, especially with respect toround_id
identification, is critical.I think it's a good time to also document this topic in more detail as the current coverage is minimal atm. so apart from clarification, I hope this discussion can become the source for additional documentation content.
Current approaches
From the scan of current hub validation methods, what I've been able to decipher is:
round_id
s which are contained as data within the files.origin_date
,forecast_date
,model_projection_date
) and the name of the column is hard coded in the validation code.-
). Underscores (_
) seem to sometimes be used within model/team names.-
.To summarise in
hubverse
terms:round_id_from_variable
set up, where theround_id
is contained, as a single unique value, in the column specified in theround_id
property of thetasks.json
config file.[round_id]-[team_abbr]-[model_abbr]
or[round_id]-[model_id]
(asmodel_id
=[team_abbr]-[model_abbr]
).model_id
._
) are used within parameters liketeam_abbr
etc. Hyphens (-
) are used as separators (and therefore should never be used within parameter names) with the exception of dateround_is
s which DO contain hyphens but follow a very explicit pattern.round_id
has been identified, it can be used to identify the round and therefore any round specific configuration in thetasks.json
config file which will then be used for model output file content (data) validation.What needs to be validated (with respect to the current discussion):
round_id
is valid (i.e. with respect to values specified in the config file).round_id_from_variable
isTRUE
, that:round_id
.round_id
in the file matches theround_id
in the file name.model_id
portion of the file name matches the folder the file is being submitted to and team/model definitions in the corresponding model metadata file.Questions/Actions
model_output
directory structure?) be? Is there a situation where this might deviate? Any sense when and why?round_id_from_variable: true
) #8, all current active hubs and example hubs all follow around_id_from_variable: true
paradigm. So I don't have a realistic use case of a hub whereround_id
is defined as part of the config only (i.e. not contained in the data file) yet. It would be good to develop one, both for demo and testing purposes. If anyone has ideas of what a realistic example of such a hub would look like, please leave your comments in Develop example hub which uses explicit round_ids (notround_id_from_variable: true
) #8.hubDocs
with them.Beta Was this translation helpful? Give feedback.
All reactions