How to specify multilayer input? #25

jannes-m · 2020-10-12T20:15:00Z

I could not specify a QGIS argument of type multilayer. I tried a list of raster files, a list of raster files stored on disk, and since I knew that QGIS expects a Python list (see e.g., r-spatial/RQGIS3#4), I also tried to mimic a Python list.

library("raster")
#> Loading required package: sp
library("qgisprocess")
#> Using 'qgis_process' at 'qgis_process'.
#> Run `qgis_configure()` for details.

r = raster(ncol = 100, nrow = 100)
r1 = crop(r, extent(-10, 11, -10, 11))
r2 = crop(r, extent(0, 20, 0, 20))
r1[] = 1:ncell(r1)
r2[] = 1:ncell(r2)

alg = "grass7:r.patch"
# qgis_arguments(alg)

# trying to use raster objects residing in R
qgis_run_algorithm(alg,
    input = list(r1, r2),
    output = file.path(tempdir(), "patch.tif"))
#> Error: Error : Don't know how to convert object of type 'list' to QGIS type 'multilayer'

# trying to use raster files stored on disk
writeRaster(r1, file.path(tempdir(), "r1.tif"))
writeRaster(r2, file.path(tempdir(), "r2.tif"))
qgis_run_algorithm(alg,
    input = c(file.path(tempdir(), "r1.tif"), file.path(tempdir(), "r2.tif")),
    output = file.path(tempdir(), "patch.tif"))
#> Running qgis_process run 'grass7:r.patch' \
#>   '--input=/tmp/RtmpHMHkmq/r1.tif,/tmp/RtmpHMHkmq/r2.tif' \
#>   '--output=/tmp/RtmpHMHkmq/patch.tif'
#> QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-rstudio'
#> 
#> ----------------
#> Inputs
#> ----------------
#> 
#> input:   /tmp/RtmpHMHkmq/r1.tif,/tmp/RtmpHMHkmq/r2.tif
#> output:  /tmp/RtmpHMHkmq/patch.tif
#> ERROR:   An error was encountered while checking parameter values
#>  Incorrect parameter value for input
#> Error in processx::run(path, ...): System command 'qgis_process' failed, exit status: 1, stderr:
#> E> QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-rstudio'
#> E> ERROR:    An error was encountered while checking parameter values
#> E>   Incorrect parameter value for input

# multilayer input should be provided as a python list
qgis_run_algorithm(
  alg,
  input = paste(c("['", file.path(tempdir(), "r1.tif"), "','", file.path(tempdir(), "r2.tif"), "']"), collapse = ""),
  # also tried without success:
  # input = reticulate::r_to_py(list(file.path(tempdir(), "r1.tif"), file.path(tempdir(), "r2.tif"))), 
  output = file.path(tempdir(), "patch.tif"))
#> Running qgis_process run "grass7:r.patch" \
#>   "--input=['/tmp/RtmpHMHkmq/r1.tif','/tmp/RtmpHMHkmq/r2.tif']" \
#>   "--output=/tmp/RtmpHMHkmq/patch.tif"
#> QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-rstudio'
#> 
#> ----------------
#> Inputs
#> ----------------
#> 
#> input:   ['/tmp/RtmpHMHkmq/r1.tif','/tmp/RtmpHMHkmq/r2.tif']
#> output:  /tmp/RtmpHMHkmq/patch.tif
#> ERROR:   An error was encountered while checking parameter values
#>  Incorrect parameter value for input
#> Error in processx::run(path, ...): System command 'qgis_process' failed, exit status: 1, stderr:
#> E> QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-rstudio'
#> E> ERROR:    An error was encountered while checking parameter values
#> E>   Incorrect parameter value for input

^{Created on 2020-10-12 by the reprex package (v0.3.0)}

Session info

devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 4.0.2 (2020-06-22)
#>  os       Ubuntu 20.04 LTS            
#>  system   x86_64, linux-gnu           
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_US.UTF-8                 
#>  ctype    en_US.UTF-8                 
#>  tz       Etc/UTC                     
#>  date     2020-10-12                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version    date       lib
#>  assertthat    0.2.1      2019-03-21 [1]
#>  backports     1.1.10     2020-09-15 [1]
#>  callr         3.4.4      2020-09-07 [1]
#>  cli           2.0.2      2020-02-28 [1]
#>  codetools     0.2-16     2018-12-24 [2]
#>  crayon        1.3.4      2017-09-16 [1]
#>  desc          1.2.0      2018-05-01 [1]
#>  devtools      2.3.2      2020-09-18 [1]
#>  digest        0.6.25     2020-02-23 [1]
#>  ellipsis      0.3.1      2020-05-15 [1]
#>  evaluate      0.14       2019-05-28 [1]
#>  fansi         0.4.1      2020-01-08 [1]
#>  fs            1.5.0      2020-07-31 [1]
#>  glue          1.4.2      2020-08-27 [1]
#>  highr         0.8        2019-03-20 [1]
#>  htmltools     0.5.0      2020-06-16 [1]
#>  knitr         1.30       2020-09-22 [1]
#>  lattice       0.20-41    2020-04-02 [2]
#>  lifecycle     0.2.0      2020-03-06 [1]
#>  magrittr      1.5        2014-11-22 [1]
#>  memoise       1.1.0      2017-04-21 [1]
#>  pillar        1.4.6      2020-07-10 [1]
#>  pkgbuild      1.1.0      2020-07-13 [1]
#>  pkgconfig     2.0.3      2019-09-22 [1]
#>  pkgload       1.1.0      2020-05-29 [1]
#>  prettyunits   1.1.1      2020-01-24 [1]
#>  processx      3.4.4      2020-09-03 [1]
#>  ps            1.4.0      2020-10-07 [1]
#>  qgisprocess * 0.0.0.9000 2020-10-10 [1]
#>  R6            2.4.1      2019-11-12 [1]
#>  raster      * 3.3-13     2020-07-17 [1]
#>  Rcpp          1.0.5      2020-07-06 [1]
#>  remotes       2.2.0      2020-07-21 [1]
#>  rgdal         1.5-16     2020-08-07 [1]
#>  rlang         0.4.8      2020-10-08 [1]
#>  rmarkdown     2.4        2020-09-30 [1]
#>  rprojroot     1.3-2      2018-01-03 [1]
#>  sessioninfo   1.1.1      2018-11-05 [1]
#>  sp          * 1.4-2      2020-05-20 [1]
#>  stringi       1.5.3      2020-09-09 [1]
#>  stringr       1.4.0      2019-02-10 [1]
#>  testthat      2.3.2      2020-03-02 [1]
#>  tibble        3.0.3      2020-07-10 [1]
#>  usethis       1.6.3      2020-09-17 [1]
#>  vctrs         0.3.4      2020-08-29 [1]
#>  withr         2.3.0      2020-09-22 [1]
#>  xfun          0.18       2020-09-29 [1]
#>  yaml          2.2.1      2020-02-01 [1]
#>  source                                  
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.0)                          
#>  CRAN (R 4.0.2)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.1)                          
#>  RSPM (R 4.0.2)                          
#>  CRAN (R 4.0.2)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.2)                          
#>  Github (paleolimbot/qgisprocess@27a6b30)
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.0)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.2)                          
#>  RSPM (R 4.0.0)                          
#> 
#> [1] /usr/local/lib/R/site-library
#> [2] /usr/local/lib/R/library

paleolimbot · 2020-10-12T23:59:20Z

I'll look into tomorrow... multilayer out is serialized as a comma separated (unquoted) file list. But it looks like that didn't work here for input! Something else to try is specifying the argument twice with the same name...

jannes-m · 2020-10-13T19:07:22Z

Update: I have tried to specify the argument twice, however, then only the second input is used.

nyalldawson · 2020-10-13T23:56:47Z

Ok -- this is a QGIS bug. There's no way to specify a list of layers from a string value alone. What do you think would be a safe way to handle this? Would "xxx x xxx.shp","yyyy.shp" work? Or is there a preferred format for the string you'd rather see?

paleolimbot · 2020-10-14T00:42:36Z

For consistency, comma separated filenames? Only because that's what multilayer output is serialized to (maybe with a backslash escape on commas in a filename?). The field parameter takes semicolon separated values and enums are comma separated. JSON IO could also be a fallback for those who need arbitrary filename support (e.g. us).

nyalldawson · 2020-11-26T03:24:49Z

There's a potential fix here: qgis/QGIS#40287

loreabad6 · 2021-05-19T07:05:25Z

Hi! I confirmed there was a bug in the implementation of multiple inputs in the qgis_run_algoritm(), after my comment here. I try to fix this in #54. Hope it helps!

bkavlak · 2021-07-30T10:11:05Z

Hey, sth like this enables to give a list to an argument.

raster_list <- c("example1.tif", "example2.tif", "example3.tif")

# Prepare INPUT List
input_list <- paste0('INPUT="', raster_list, '"')
input_list <- paste0(input_list, collapse = ",")
  
# Prepare function string
func_string <- paste0('qgisprocess::qgis_run_algorithm("gdal:merge",',input_list,',NODATA_INPUT=0,DATA_TYPE=0,.quiet=TRUE)')
  
# Run GDAL MERGE with QGIS
qgis_result <- eval(rlang::parse_expr(func_string))

The rlang::parse_expr(func_string) returns:

qgisprocess::qgis_run_algorithm("gdal:merge", INPUT = "example1.tif", 
    INPUT = "example2.tif", 
    INPUT = "example3.tif", 
    NODATA_INPUT = 0, DATA_TYPE = 0, .quiet = TRUE)

INPUT or LAYER parameters change per algorithm. I couldn't delve into the codes of the package. How should we implement this? Any ideas?

paleolimbot · 2021-07-30T22:55:53Z

I'm not entirely sure I understand, but I think you might be after something like this:

author: DUNNINGTOND
date: 2021-07-30
output: "reprex::reprex_document"
title: full-husky_reprex.R

raster_list <- c("example1.tif", "example2.tif", "example3.tif")
names(raster_list) <- rep("INPUT", length(raster_list))
rlang::list2(!!! raster_list)
#> $INPUT
#> [1] "example1.tif"
#> 
#> $INPUT
#> [1] "example2.tif"
#> 
#> $INPUT
#> [1] "example3.tif"

# should work!
# qgisprocess::qgis_run_algorithm("gdal:merge", !!! raster_list)

The idea is that ... in qgis_run_algorithm() supports tidy dots syntax (through rlang::list2()), so you can use the splicing operator (!!!) to expand a list as if it were multiple arguments.

bkavlak · 2021-07-31T13:23:26Z

This simplifies the task, thanks. I had a hard time passing a list of inputs or layers and found the method above. It'd be perfect if a user can give a list to INPUT or LAYER arguments without thinking about these issues. I just wanted to add feedback.

paleolimbot · 2021-08-31T11:19:13Z

This is fixed in #63 (use qgis_list_input()). Using list() and c() may also work in the future but qgis_list_input() is more explicit and will always work.

library(qgisprocess)
#> Using 'qgis_process' at '/Applications/QGIS-LTR.app/Contents/MacOS/bin/qgis_process'.
#> QGIS version: 3.16.10-Hannover
#> Metadata of 971 algorithms successfully cached.
#> Run `qgis_configure()` for details.

rstr <- system.file("longlake/longlake_depth.tif", package = "qgisprocess")
pts <- system.file("longlake/longlake_depth.gpkg", package = "qgisprocess")

qgis_run_algorithm(
  "saga:addrastervaluestopoints",
  SHAPES = pts,
  GRIDS = qgis_list_input(rstr, rstr),
  RESAMPLING = 0,
  RESULT = qgis_tmp_vector(),
  .quiet = TRUE
)
#> <Result of `qgis_run_algorithm("saga:addrastervaluestopoints", ...)`>
#> List of 1
#>  $ RESULT: 'qgis_outputVector' chr "/var/folders/gt/l87wjg8s7312zs9s7c1fgs900000gn/T//Rtmpwv3nCd/file26603df4be17/file2660eebf7c8.gpkg"

^{Created on 2021-08-31 by the reprex package (v2.0.1)}

paleolimbot mentioned this issue Oct 13, 2020

Complete input argument type parsing #13

Closed

This was referenced Dec 15, 2020

qgis_query_version(): search for qgis version in all output lines #30

Closed

Vignettes #31

Closed

This was referenced May 18, 2021

make multiple input arguments work #33

Merged

Handle multiple inputs as lists with duplicated names in R. #54

Merged

paleolimbot closed this as completed Aug 31, 2021

juliepierson mentioned this issue Nov 14, 2022

Incorrect parameter value for INPUT for gdal:buildvirtualraster rzine-reviews/fiche_rzine_geomorphon#7

Open

florisvdh mentioned this issue Mar 29, 2023

Revising the namespace #133

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to specify multilayer input? #25

How to specify multilayer input? #25

jannes-m commented Oct 12, 2020

paleolimbot commented Oct 12, 2020

jannes-m commented Oct 13, 2020

nyalldawson commented Oct 13, 2020

paleolimbot commented Oct 14, 2020

nyalldawson commented Nov 26, 2020

loreabad6 commented May 19, 2021

bkavlak commented Jul 30, 2021 •

edited

Loading

paleolimbot commented Jul 30, 2021

bkavlak commented Jul 31, 2021

paleolimbot commented Aug 31, 2021

How to specify multilayer input? #25

How to specify multilayer input? #25

Comments

jannes-m commented Oct 12, 2020

paleolimbot commented Oct 12, 2020

jannes-m commented Oct 13, 2020

nyalldawson commented Oct 13, 2020

paleolimbot commented Oct 14, 2020

nyalldawson commented Nov 26, 2020

loreabad6 commented May 19, 2021

bkavlak commented Jul 30, 2021 • edited Loading

paleolimbot commented Jul 30, 2021

author: DUNNINGTOND date: 2021-07-30 output: "reprex::reprex_document" title: full-husky_reprex.R

bkavlak commented Jul 31, 2021

paleolimbot commented Aug 31, 2021

bkavlak commented Jul 30, 2021 •

edited

Loading

author: DUNNINGTOND
date: 2021-07-30
output: "reprex::reprex_document"
title: full-husky_reprex.R