Combining list of SpatRasters with rast fails with multiple workers

[Aariq]

I'm encountering a bug when trying to combine a list of rasters created by dynamic branching. It seems related to marshaling, since the error message mentions wrap() and I only get the error with multiple crew workers, but not sure.

targets::tar_script({
    targets::tar_option_set(controller = crew::crew_controller_local(workers = 2))
    list(
        geotargets::tar_terra_rast(
            rast_raw,
            terra::rast(system.file("ex/elev.tif", package = "terra"))
        ),
        targets::tar_target(x, 1:3),
        geotargets::tar_terra_rast(
            rast_plus,
            rast_raw + x,
            pattern = map(x),
            iteration = "list"
        ),
        geotargets::tar_terra_rast(
            combined,
            terra::rast(unname(rast_plus))
        )
    )
})
targets::tar_make()
#> ▶ dispatched target x
#> ▶ dispatched target rast_raw
#> ● completed target x [0.028 seconds]
#> ● completed target rast_raw [5.603 seconds]
#> ▶ dispatched branch rast_plus_29239c8a
#> ▶ dispatched branch rast_plus_7cc32924
#> ● completed branch rast_plus_7cc32924 [0.008 seconds]
#> ▶ dispatched branch rast_plus_bd602d50
#> ● completed branch rast_plus_bd602d50 [0.005 seconds]
#> ● completed branch rast_plus_29239c8a [0.006 seconds]
#> ● completed pattern rast_plus
#> ▶ dispatched target combined
#> ▶ ended pipeline [19.278 seconds]
#> Error:
#> ! Error running targets::tar_make()
#> Error messages: targets::tar_meta(fields = error, complete_only = TRUE)
#> Debugging guide: https://books.ropensci.org/targets/debugging.html
#> How to ask for help: https://books.ropensci.org/targets/help.html
#> Last error message:
#>     target combined error: unable to find an inherited method for function ‘wrap’ for signature ‘"NULL"’
#> Last error traceback:
#>     base::tryCatch(base::withCallingHandlers({ NULL base::saveRDS(base::do.c...
#>     tryCatchList(expr, classes, parentenv, handlers)
#>     tryCatchOne(tryCatchList(expr, names[-nh], parentenv, handlers[-nh]), na...
#>     doTryCatch(return(expr), name, parentenv, handler)
#>     tryCatchList(expr, names[-nh], parentenv, handlers[-nh])
#>     tryCatchOne(expr, names, parentenv, handlers[[1L]])
#>     doTryCatch(return(expr), name, parentenv, handler)
#>     base::withCallingHandlers({ NULL base::saveRDS(base::do.call(base::do.ca...
#>     base::saveRDS(base::do.call(base::do.call, base::c(base::readRDS("/var/f...
#>     base::do.call(base::do.call, base::c(base::readRDS("/var/folders/wr/by_l...
#>     (function (what, args, quote = FALSE, envir = parent.frame()) { if (!is....
#>     (function (targets_function, targets_arguments, options, envir = NULL, s...
#>     tryCatch(out <- withCallingHandlers(targets::tar_callr_inner_try(targets...
#>     tryCatchList(expr, classes, parentenv, handlers)
#>     tryCatchOne(expr, names, parentenv, handlers[[1L]])
#>     doTryCatch(return(expr), name, parentenv, handler)
#>     withCallingHandlers(targets::tar_callr_inner_try(targets_function = targ...
#>     targets::tar_callr_inner_try(targets_function = targets_function, target...
#>     do.call(targets_function, targets_arguments)
#>     (function (pipeline, path_store, names_quosure, shortcut, reporter, seco...
#>     crew_init(pipeline = pipeline, meta = meta_init(path_store = path_store)...
#>     self$run_crew()
#>     self$iterate()
#>     self$conclude_worker_task()
#>     tar_assert_all_na(result$error, msg = paste("target", result$name, "erro...
#>     tar_throw_validate(msg %|||% default)
#>     tar_error(message = paste0(...), class = c("tar_condition_validate", "ta...
#>     rlang::abort(message = message, class = class, call = tar_empty_envir)
#>     signal_abort(cnd, .file)

^{Created on 2024-04-09 with reprex v2.1.0}

[Aariq]

I'm stumped on this one. As a workaround, setting deployment = "main" allows this to work.

[Aariq]

Hmmm... getting this same bug with a tar_terra_rast() target that subsets a multi-layer SpatRaster and does some calculations with app(). Works interactively, works with deployment = "main", but errors with multiple workers

[Aariq]

Ok, I suspect the reason for this has something to do with how marshaling works with branching. In this example, rast_plus is a list, which can't be wrap()ed. I think we really need an iteration = "terra_layers" or something that does branching with [[]] and aggregation with terra::c().

[njtierney]

I think you might be right there with some kind of nice iteration handler for geotargets. I can take a look at this later this week/end

[Aariq]

Unable to reproduce this with just crew, although I'm not certain that this is equivalent to how targets and crew interact with marshaling/unmarshaling.

library(crew)
library(terra)
#> terra 1.7.71
library(purrr)
controller <- crew_controller_local(
    name = "example",
    workers = 2,
    seconds_idle = 10
)
controller$start()

marshal <- terra::wrap
unmarshal <- terra::unwrap

rast_list <- list(a = marshal(terra::rast(system.file("ex/logo.tif", package = "terra"))),
                  b = marshal(terra::rast(system.file("ex/logo.tif", package = "terra")) + 1))

#goal is to combine rast list like so:
terra::rast(purrr::map(rast_list, unmarshal))
#> class       : SpatRaster 
#> dimensions  : 77, 101, 6  (nrow, ncol, nlyr)
#> resolution  : 1, 1  (x, y)
#> extent      : 0, 101, 0, 77  (xmin, xmax, ymin, ymax)
#> coord. ref. : Cartesian (Meter) 
#> source(s)   : memory
#> names       : a_1, a_2, a_3, b_1, b_2, b_3 
#> min values  :   0,   0,   0,   1,   1,   1 
#> max values  : 255, 255, 255, 256, 256, 256

controller$push(
    name = "combine",
    command = marshal(terra::rast(purrr::map(rast_list, unmarshal))),
    data = list(rast_list = rast_list, marshal = marshal, unmarshal = unmarshal)
)

controller$wait(mode = "all")
result <- controller$pop()
result$result |> map(unmarshal)
#> [[1]]
#> class       : SpatRaster 
#> dimensions  : 77, 101, 6  (nrow, ncol, nlyr)
#> resolution  : 1, 1  (x, y)
#> extent      : 0, 101, 0, 77  (xmin, xmax, ymin, ymax)
#> coord. ref. : Cartesian (Meter) 
#> source(s)   : memory
#> names       : a_1, a_2, a_3, b_1, b_2, b_3 
#> min values  :   0,   0,   0,   1,   1,   1 
#> max values  : 255, 255, 255, 256, 256, 256

^{Created on 2024-04-12 with reprex v2.1.0}

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.3.2 (2023-10-31)
#>  os       macOS Sonoma 14.4.1
#>  system   x86_64, darwin20
#>  ui       X11
#>  language (EN)
#>  collate  en_US.UTF-8
#>  ctype    en_US.UTF-8
#>  tz       America/Phoenix
#>  date     2024-04-12
#>  pandoc   3.1.13 @ /usr/local/bin/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version date (UTC) lib source
#>  cli           3.6.2   2023-12-11 [1] CRAN (R 4.3.0)
#>  codetools     0.2-20  2024-03-31 [1] CRAN (R 4.3.2)
#>  crew        * 0.9.0   2024-02-08 [1] CRAN (R 4.3.2)
#>  data.table    1.15.4  2024-03-30 [1] CRAN (R 4.3.2)
#>  digest        0.6.35  2024-03-11 [1] CRAN (R 4.3.2)
#>  evaluate      0.23    2023-11-01 [1] CRAN (R 4.3.0)
#>  fansi         1.0.6   2023-12-08 [1] CRAN (R 4.3.0)
#>  fastmap       1.1.1   2023-02-24 [1] CRAN (R 4.3.0)
#>  fs            1.6.3   2023-07-20 [1] CRAN (R 4.3.0)
#>  getip         0.1-4   2023-12-10 [1] CRAN (R 4.3.0)
#>  glue          1.7.0   2024-01-09 [1] CRAN (R 4.3.0)
#>  htmltools     0.5.7   2023-11-03 [1] CRAN (R 4.3.0)
#>  knitr         1.46    2024-04-06 [1] CRAN (R 4.3.2)
#>  later         1.3.2   2023-12-06 [1] CRAN (R 4.3.0)
#>  lifecycle     1.0.4   2023-11-07 [1] CRAN (R 4.3.0)
#>  magrittr      2.0.3   2022-03-30 [1] CRAN (R 4.3.0)
#>  mirai         0.12.1  2024-02-02 [1] CRAN (R 4.3.2)
#>  nanonext      0.13.0  2024-02-07 [1] CRAN (R 4.3.2)
#>  pillar        1.9.0   2023-03-22 [1] CRAN (R 4.3.0)
#>  pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 4.3.0)
#>  processx      3.8.4   2024-03-16 [1] CRAN (R 4.3.2)
#>  promises      1.2.1   2023-08-10 [1] CRAN (R 4.3.0)
#>  ps            1.7.6   2024-01-18 [1] CRAN (R 4.3.0)
#>  purrr       * 1.0.2   2023-08-10 [1] CRAN (R 4.3.0)
#>  R.cache       0.16.0  2022-07-21 [1] CRAN (R 4.3.0)
#>  R.methodsS3   1.8.2   2022-06-13 [1] CRAN (R 4.3.0)
#>  R.oo          1.26.0  2024-01-24 [1] CRAN (R 4.3.2)
#>  R.utils       2.12.3  2023-11-18 [1] CRAN (R 4.3.0)
#>  R6            2.5.1   2021-08-19 [1] CRAN (R 4.3.0)
#>  Rcpp          1.0.12  2024-01-09 [1] CRAN (R 4.3.0)
#>  reprex        2.1.0   2024-01-11 [1] CRAN (R 4.3.0)
#>  rlang         1.1.3   2024-01-10 [1] CRAN (R 4.3.0)
#>  rmarkdown     2.25    2023-09-18 [1] CRAN (R 4.3.0)
#>  rstudioapi    0.15.0  2023-07-07 [1] CRAN (R 4.3.0)
#>  sessioninfo   1.2.2   2021-12-06 [1] CRAN (R 4.3.0)
#>  styler        1.10.2  2023-08-29 [1] CRAN (R 4.3.0)
#>  terra       * 1.7-71  2024-01-31 [1] CRAN (R 4.3.2)
#>  tibble        3.2.1   2023-03-20 [1] CRAN (R 4.3.0)
#>  tidyselect    1.2.1   2024-03-11 [1] CRAN (R 4.3.2)
#>  utf8          1.2.4   2023-10-22 [1] CRAN (R 4.3.0)
#>  vctrs         0.6.5   2023-12-01 [1] CRAN (R 4.3.0)
#>  withr         3.0.0   2024-01-16 [1] CRAN (R 4.3.0)
#>  xfun          0.43    2024-03-25 [1] CRAN (R 4.3.2)
#>  yaml          2.3.8   2023-12-11 [1] CRAN (R 4.3.0)
#> 
#>  [1] /Users/ericscott/Library/R/x86_64/4.3/library
#>  [2] /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

[wlandau]

This was caused by a marshaling bug in targets: ropensci/targets#1266. Fixed in ropensci/targets#1267.

[Aariq]

Confirmed that this works now! Thanks @wlandau!