tar_map() losing names if values is a data frame with a list column

[MarekGierlinski]

I am trying to pass a list of parameters down for processing in tar_map(), iterating over several models with different parameters. The code below shows that though I define parameters as named list, the names are stripped when passing it to a target.

targets::tar_dir({ # tar_dir() runs code from a temporary directory.
  targets::tar_script({
    MODELS <- tibble::tribble(
      ~NAME, ~PARAMS,
      1, list(a = 3, b = 5, c = 25),
      2, list(a = 0, b = 1, c = 42)
    )
    
    list(
      tarchetypes::tar_map(
        values = MODELS,
        names = NAME,
        targets::tar_target(params, PARAMS)
      )
    )
  })
  targets::tar_manifest()
})

# A tibble: 2 × 3
  name     command        pattern
  <chr>    <chr>          <chr>  
1 params_1 list(3, 5, 25) NA     
2 params_2 list(0, 1, 42) NA     

As you can see the lists passed to command are unnamed. Parameter values is set to the tibble:

> MODELS
# A tibble: 2 × 2
   NAME PARAMS          
  <dbl> <list>          
1     1 <named list [3]>
2     2 <named list [3]>

where each element of PARAMS is a named list.

>str(MODELS$PARAMS)
List of 2
 $ :List of 3
  ..$ a: num 3
  ..$ b: num 5
  ..$ c: chr "fist"
 $ :List of 3
  ..$ a: num 0
  ..$ b: num 1
  ..$ c: chr "second"

These names are missing in my target.

[wlandau]

tar_map() substitutes simple objects, symbois, and expressions. Try wrapping your lists in quote() to treat them as expression objects.

targets::tar_script({
  MODELS <- tibble::tribble(
    ~NAME, ~PARAMS,
    1, quote(list(a = 3, b = 5, c = 25)), # quote()
    2, quote(list(a = 0, b = 1, c = 42))  # quote()
  )
  
  list(
    tarchetypes::tar_map(
      values = MODELS,
      names = NAME,
      targets::tar_target(params, PARAMS)
    )
  )
})

targets::tar_manifest()
#> # A tibble: 2 × 2
#>   name     command                   
#>   <chr>    <chr>                     
#> 1 params_1 list(a = 3, b = 5, c = 25)
#> 2 params_2 list(a = 0, b = 1, c = 42)

^{Created on 2022-09-05 with reprex v2.0.2}

[MarekGierlinski]

Thank you, it does work. Looks weird, but works.

[wlandau]

You're welcome. Keep in mind that tar_map() generates code for other targets, so it needs to transform the elements of MODELS into expressions and symbols. Backsolving for the expression that generated an object is highly nontrivial and not always possible. So for objects even a little bit complicated, it is safer to just work with the underlying expressions.

[dsweber2]

We have a related question: we're generating parameters using expand_grid, which means we can't use quote (it will literally quote the transpose and expand grid, rather than partially evaluate); we tried enquote, but it results in the names being dropped problem. (linreg and quantreg are actually objects in our context, so we're using the syms trick to include them)j.

linreg <- 5
quantreg <- 50
targets::tar_script({
  MODELS <- tibble(
  NAME = 1:16,
  PARAMS = map(
    transpose(expand_grid(
      trainer = rlang::syms(c("linreg", "quantreg")),
      ahead = 1:4,
      pop_scaling = c(TRUE, FALSE)
     )), enquote
   )
  )
list(
  tar_map(
    values = MODELS,
    names = NAME,
    tar_target(
      params,
      command = {
        print(PARAMS)
      }
    )
  )
)

which produces

r$> tar_manifest()
/
Attaching package: 'dplyr'

The following objects are masked from 'package:stats':

    filter, lag

The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union

Loading required package: epiprocess
/
Attaching package: 'epiprocess'

The following object is masked from 'package:stats':

    filter

Loading required package: parsnip
# A tibble: 16 × 2
   name      command                                                    
   <chr>     <chr>                                                      
 1 params_10 "{\n     print(base::quote(list(quantreg, 1L, FALSE)))\n }"
 2 params_11 "{\n     print(base::quote(list(quantreg, 2L, TRUE)))\n }" 
 3 params_12 "{\n     print(base::quote(list(quantreg, 2L, FALSE)))\n }"
 4 params_13 "{\n     print(base::quote(list(quantreg, 3L, TRUE)))\n }" 
 5 params_1  "{\n     print(base::quote(list(linreg, 1L, TRUE)))\n }"   
 6 params_14 "{\n     print(base::quote(list(quantreg, 3L, FALSE)))\n }"
 7 params_2  "{\n     print(base::quote(list(linreg, 1L, FALSE)))\n }"  
 8 params_15 "{\n     print(base::quote(list(quantreg, 4L, TRUE)))\n }" 
 9 params_3  "{\n     print(base::quote(list(linreg, 2L, TRUE)))\n }"   
10 params_16 "{\n     print(base::quote(list(quantreg, 4L, FALSE)))\n }"
11 params_4  "{\n     print(base::quote(list(linreg, 2L, FALSE)))\n }"  
12 params_5  "{\n     print(base::quote(list(linreg, 3L, TRUE)))\n }"   
13 params_6  "{\n     print(base::quote(list(linreg, 3L, FALSE)))\n }"  
14 params_7  "{\n     print(base::quote(list(linreg, 4L, TRUE)))\n }"   
15 params_8  "{\n     print(base::quote(list(linreg, 4L, FALSE)))\n }"  
16 params_9  "{\n     print(base::quote(list(quantreg, 1L, TRUE)))\n }" 

r$> MODELS$PARAMS[[1]]
base::quote(list(trainer = linreg, ahead = 1L, pop_scaling = TRUE))

[dshemetov]

@wlandau friendly ping. Not sure if closed discussions provide notifications, but this thread seemed very relevant to the issue.

[wlandau]

You could generate the expressions after expand_grid() on the parameters:

models <- expand_grid(
  trainer = rlang::syms(c("linreg", "quantreg")),
  ahead = 1:4,
  pop_scaling = c(TRUE, FALSE)
) %>%
  pmap(.f = rlang::call2, .fn = "f")
str(models)
#> List of 16
#>  $ : language f(trainer = linreg, ahead = 1L, pop_scaling = TRUE)
#>  $ : language f(trainer = linreg, ahead = 1L, pop_scaling = FALSE)
#>  $ : language f(trainer = linreg, ahead = 2L, pop_scaling = TRUE)
#>  $ : language f(trainer = linreg, ahead = 2L, pop_scaling = FALSE)
#>  $ : language f(trainer = linreg, ahead = 3L, pop_scaling = TRUE)
#>  $ : language f(trainer = linreg, ahead = 3L, pop_scaling = FALSE)
#>  $ : language f(trainer = linreg, ahead = 4L, pop_scaling = TRUE)
#>  $ : language f(trainer = linreg, ahead = 4L, pop_scaling = FALSE)
#>  $ : language f(trainer = quantreg, ahead = 1L, pop_scaling = TRUE)
#>  $ : language f(trainer = quantreg, ahead = 1L, pop_scaling = FALSE)
#>  $ : language f(trainer = quantreg, ahead = 2L, pop_scaling = TRUE)
#>  $ : language f(trainer = quantreg, ahead = 2L, pop_scaling = FALSE)
#>  $ : language f(trainer = quantreg, ahead = 3L, pop_scaling = TRUE)
#>  $ : language f(trainer = quantreg, ahead = 3L, pop_scaling = FALSE)
#>  $ : language f(trainer = quantreg, ahead = 4L, pop_scaling = TRUE)
#>  $ : language f(trainer = quantreg, ahead = 4L, pop_scaling = FALSE)

[yonisidi]

it looks like something isnt reading correctly in the network node names when constructing the visnetwork/mermaid visuals, manifest looks ok

Are the network plots printing the descriptions?

targets::tar_dir({
  targets::tar_script({
    MODELS <- tibble::tribble(
      ~NAME, ~PARAMS,
      1, list(a = 3, b = 5, c = 25),
      2, list(a = 0, b = 1, c = 42)
    ) |>
      dplyr::mutate(
        dplyr::across(-NAME, ~purrr::map(.x, quote))
      )
    
    list(
      tarchetypes::tar_map(
        values = MODELS,
        names = NAME,
        targets::tar_target(params, PARAMS)
      )
    )
  })
  targets::tar_mermaid(targets_only = TRUE)
})
#>  [1] "graph LR"                                                                                                             
#>  [2] "  style Legend fill:#FFFFFF00,stroke:#000000;"                                                                        
#>  [3] "  style Graph fill:#FFFFFF00,stroke:#000000;"                                                                         
#>  [4] "  subgraph Legend"                                                                                                    
#>  [5] "    direction LR"                                                                                                     
#>  [6] "    x2db1ec7a48f65a9b([\"\"Outdated\"\"]):::outdated --- xd03d7c7dd2ddda2b([\"\"Stem\"\"]):::none"                    
#>  [7] "  end"                                                                                                                
#>  [8] "  subgraph Graph"                                                                                                     
#>  [9] "    direction LR"                                                                                                     
#> [10] "    xa28dc0b612a56289([\"params_1<br>1 expr\"]):::outdated --> xa28dc0b612a56289([\"params_1<br>1 expr\"]):::outdated"
#> [11] "    x121c500316efbe56([\"params_2<br>2 expr\"]):::outdated --> x121c500316efbe56([\"params_2<br>2 expr\"]):::outdated"
#> [12] "  end"                                                                                                                
#> [13] "  classDef outdated stroke:#000000,color:#000000,fill:#78B7C5;"                                                       
#> [14] "  classDef none stroke:#000000,color:#000000,fill:#94a4ac;"                                                           
#> [15] "  linkStyle 0 stroke-width:0px;"                                                                                      
#> [16] "  linkStyle 1 stroke-width:0px;"                                                                                      
#> [17] "  linkStyle 2 stroke-width:0px;"

^{Created on 2024-07-09 by the reprex package (v2.0.1)}

# A tibble: 2 × 3
  name     command description
  <chr>    <chr>   <chr>      
1 params_1 expr    1 expr     
2 params_2 expr    2 expr     

[yonisidi]

ah i see

removing the description manually gets back to the previous printout I was expecting

targets::tar_dir({
  targets::tar_script({
    MODELS <- tibble::tribble(
      ~NAME, ~PARAMS,
      1, list(a = 3, b = 5, c = 25),
      2, list(a = 0, b = 1, c = 42)
    ) |>
      dplyr::mutate(
        dplyr::across(-NAME, ~purrr::map(.x, quote))
      )
    
    list(
      tarchetypes::tar_map(
        values = MODELS,
        names = NAME,
        targets::tar_target(params, PARAMS)
      )
    )
  })
  targets::tar_visnetwork(targets_only = TRUE, label = c())
})