Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix to job.sh model2netcdf line #3075

Merged
merged 14 commits into from
Jan 7, 2023
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ convert data for a single PFT fixed (#1329, #2974, #2981)
the cdo_setup argument in the template job file. In detail, people will need
to specify cdosetup = "module load cdo/2.0.6" in the host section. More details
are in the Create_Multi_settings.R script. (#3052)
- Fixed a bug causing the model2netcdf.ED2() step in jobs.sh to be incorrectly written (#3075)


### Changed
Expand Down
2 changes: 1 addition & 1 deletion models/ed/DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Authors@R: c(person("Mike", "Dietze", role = c("aut", "cre"),
comment = c(ORCID = "0000-0002-7430-7879")),
person("University of Illinois, NCSA", role = c("cph")))
Author: David LeBauer, Mike Dietze, Xiaohui Feng, Dan Wang, Carl
Davidson, Rob Kooper, Shawn Serbin, Alexey Shiklomanov
Davidson, Rob Kooper, Shawn Serbin, Alexey Shiklomanov, Eric R. Scott
Maintainer: Mike Dietze <[email protected]>
Description: The Predictive Ecosystem Carbon Analyzer (PEcAn) is a scientific
workflow management tool that is designed to simplify the management of
Expand Down
1 change: 1 addition & 0 deletions models/ed/NEWS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# PEcAn.ED2 (development version)

* Fixed a bug effecting the generation of job.sh for runs with many PFTs (#3075)
dlebauer marked this conversation as resolved.
Show resolved Hide resolved
* Added optional `process_partial` argument to `model2netcdf.ED2()` to allow it to process existing output from failed runs.

# PEcAn.ED2 1.7.2
Expand Down
2 changes: 1 addition & 1 deletion models/ed/R/write.configs.ed.R
Original file line number Diff line number Diff line change
Expand Up @@ -647,7 +647,7 @@ write.config.jobsh.ED2 <- function(settings, run.id) {
jobsh <- gsub("@BINARY@", settings$model$binary, jobsh)

pft_names <- extract_pfts(settings$pfts)
pft_names <- deparse(dput(pft_names))
pft_names <- deparse1(dput(pft_names))
jobsh <- gsub("@PFT_NAMES@", pft_names, jobsh)

return(jobsh)
Expand Down
2 changes: 2 additions & 0 deletions models/ed/inst/template.job
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
#!/bin/bash -l

# Parsed by PEcAn.ED2::write.config.jobsh.ED2() to generate job.sh files

# create output folder
mkdir -p "@OUTDIR@"
@SCRATCH_MKDIR@
Expand Down
21 changes: 21 additions & 0 deletions models/ed/tests/testthat/test.write.configs.ed.R
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ testdir <- tempfile()
dir.create(testdir)
withr::defer(unlink(testdir, recursive = TRUE))
unzip("data/outdir.zip", exdir = testdir)
# unzip("models/ed/tests/testthat/data/outdir.zip", exdir = testdir)
dlebauer marked this conversation as resolved.
Show resolved Hide resolved
outdir <- file.path(testdir, "outdir")

test_that("write.config.jobsh.ED2() writes correct model2netcdf.ED2() args", {
Expand All @@ -32,6 +33,22 @@ test_that("write.config.jobsh.ED2() writes correct model2netcdf.ED2() args", {
expect_true(any(stringr::str_detect(job.sh, stringr::fixed(expect))))
})

test_that("write.config.jobsh.ED2() works with long list of PFTs", {
settings <-
PEcAn.settings::read.settings(file.path(outdir, "pecan_checked.xml"))
more_pfts <- list(
pft = list(name = "tempconif", ed2_pft_number = 7),
pft = list(name = "temperate.Evergreen_Hardwood", ed2_pft_number = 8),
pft = list(name = "temperate.Early_Hardwood", ed2_pft_number = 9),
pft = list(name = "temperate.North_Mid_Hardwood", ed2_pft_number = 10),
pft = list(name = "temperate.Late_Hardwood", ed2_pft_number = 11)
)
settings$pfts <- append(settings$pfts, more_pfts)
job.sh <- write.config.jobsh.ED2(settings, run.id = "test_run")
expect <- deparse1(dput(extract_pfts(settings$pfts)))
expect_true(any(stringr::str_detect(job.sh, stringr::fixed(expect))))
})

test_that("New ED2IN tags get added at bottom of file", {
#1. read in pecan.xml in data/pecan_checked.xml
settings <- PEcAn.settings::read.settings("data/pecan_checked.xml")
Expand Down Expand Up @@ -89,6 +106,7 @@ test_that("New ED2IN tags get added at bottom of file", {
posteriorid = 9000001416
)
)
old_level <- PEcAn.logger::logger.setLevel("DEBUG")
x <- capture.output(
write.config.ED2(
trait.values = trait.values,
Expand All @@ -99,12 +117,15 @@ test_that("New ED2IN tags get added at bottom of file", {
),
type = "message"
)
PEcAn.logger::logger.setLevel(old_level)


#5. check if new tag exists
ed2in_out <- read_ed2in(file.path(rundir, run.id, "ED2IN"))
expect_equal(ed2in_out$NEW_TAG, 0)

#check that info is printed

expect_true(any(stringr::str_detect(x, "NEW_TAG")))

#check that last non-comment line of ED2IN is "$END"
Expand Down