Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing qgis_run_algorithm methods #146

Closed
wants to merge 4 commits into from
Closed

Conversation

jannes-m
Copy link
Collaborator

Hey @florisvdh,
of course, you were right in #31, for method dispatch to work, one needs the same first argument across the methods. Well, this is not entirely true, see example below where method dispatch is always done via the first specified argument, hence, in the first case it is done via the algorithm argument and in the other examples via the .data argument, maybe there is a way to further exploit this... For now, I used a .data argument in all qgis_run_algorithm() methods. It works (see below). However, now that the algorithm argument is no longer the first, one has to explicitly set it, e.g., algorithm = "native:buffer".
This is just a test PR, so that you can easily compare what I have changed. But feel free to close it without further working on it or merging. I have to admit I am also a bit torn if I like the approach or not. On the one hand, it is great that one does not need another function like qgis_run_algorithm_p(), on the other hand the approach introduces a .data argument and still feels a bit rough on the edges.

library("terra")
#> terra 1.7.18
devtools::load_all("~/programming/R/spatial/qgisprocess/")
#> ℹ Loading qgisprocess
#> Attempting to load the cache ... Success!
#> QGIS version: 3.30.0-'s-Hertogenbosch
#> Having access to 1162 algorithms from 6 QGIS processing providers.
#> Run `qgis_configure(use_cached_data = TRUE)` to reload cache and get more details.
dem = system.file("raster/dem.tif", package = "spDataLarge") |>
  rast()
# running qgis_run_algorithm.character which calls qgis_run_algorithm.default
# via qgis_run(); here, method dispatch is done via the algorithm argument
out = qgis_run_algorithm(algorithm = "sagang:sinkremoval", DEM = dem,
                         METHOD = 1)
#> Using `DEM_PREPROC = qgis_tmp_raster()`
#> Argument `THRESHOLD` is unspecified (using QGIS default value).
#> Argument `THRSHEIGHT` is unspecified (using QGIS default value).
# running qgis_run_algorithm.qgis_result
out |>
  qgis_run_algorithm(algorithm = "sagang:sagawetnessindex", 
                     .select = "DEM_PREPROC")
#> Argument `WEIGHT` is unspecified (using QGIS default value).
#> Using `AREA = qgis_tmp_raster()`
#> Using `SLOPE = qgis_tmp_raster()`
#> Using `AREA_MOD = qgis_tmp_raster()`
#> Using `TWI = qgis_tmp_raster()`
#> Argument `SUCTION` is unspecified (using QGIS default value).
#> Using `AREA_TYPE = "[0] total catchment area"`
#> Using `SLOPE_TYPE = "[0] local slope"`
#> Argument `SLOPE_MIN` is unspecified (using QGIS default value).
#> Argument `SLOPE_OFF` is unspecified (using QGIS default value).
#> Argument `SLOPE_WEIGHT` is unspecified (using QGIS default value).
#> <Result of `qgis_run_algorithm("sagang:sagawetnessindex", ...)`>
#> List of 4
#>  $ AREA    : 'qgis_outputRaster' chr "/tmp/RtmpLCh6YR/file215258611718/file2152ec2457a.tif"
#>  $ AREA_MOD: 'qgis_outputRaster' chr "/tmp/RtmpLCh6YR/file215258611718/file2152797ae096.tif"
#>  $ SLOPE   : 'qgis_outputRaster' chr "/tmp/RtmpLCh6YR/file215258611718/file21526fd14a2c.tif"
#>  $ TWI     : 'qgis_outputRaster' chr "/tmp/RtmpLCh6YR/file215258611718/file21527f766cc3.tif"

# running qgis_run_algorithm.character
system.file("longlake/longlake_depth.gpkg", package = "qgisprocess") |>
  qgis_run_algorithm(
    algorithm = "native:buffer",
    DISTANCE = 100,
    DISSOLVE = TRUE,
    MITER_LIMIT = 2,
    OUTPUT = qgis_tmp_vector(),
    END_CAP_STYLE = 0,
    JOIN_STYLE = 0
  )
#> Argument `SEGMENTS` is unspecified (using QGIS default value).
#> <Result of `qgis_run_algorithm("native:buffer", ...)`>
#> List of 1
#>  $ OUTPUT: 'qgis_outputVector' chr "/tmp/RtmpLCh6YR/file215258611718/file21527073ae2.gpkg"

Created on 2023-03-30 with reprex v2.0.2

Session info
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.3 (2023-03-15)
#>  os       Arch Linux
#>  system   x86_64, linux-gnu
#>  ui       X11
#>  language (EN)
#>  collate  en_US.UTF-8
#>  ctype    en_US.UTF-8
#>  tz       Europe/Berlin
#>  date     2023-03-30
#>  pandoc   2.19.2 @ /usr/lib/rstudio/resources/app/bin/quarto/bin/tools/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  ! package     * version    date (UTC) lib source
#>    assertthat    0.2.1      2019-03-21 [1] CRAN (R 4.2.1)
#>    brio          1.1.3      2021-11-30 [1] CRAN (R 4.2.1)
#>    cachem        1.0.7      2023-02-24 [1] CRAN (R 4.2.3)
#>    callr         3.7.3      2022-11-02 [1] CRAN (R 4.2.3)
#>    cli           3.6.1      2023-03-23 [1] CRAN (R 4.2.3)
#>    codetools     0.2-19     2023-02-01 [2] CRAN (R 4.2.3)
#>    crayon        1.5.2      2022-09-29 [1] CRAN (R 4.2.3)
#>    desc          1.4.2      2022-09-08 [1] CRAN (R 4.2.3)
#>    devtools      2.4.5      2022-10-11 [1] CRAN (R 4.2.3)
#>    digest        0.6.31     2022-12-11 [1] CRAN (R 4.2.3)
#>    ellipsis      0.3.2      2021-04-29 [1] CRAN (R 4.2.1)
#>    evaluate      0.20       2023-01-17 [1] CRAN (R 4.2.3)
#>    fansi         1.0.4      2023-01-22 [1] CRAN (R 4.2.3)
#>    fastmap       1.1.1      2023-02-24 [1] CRAN (R 4.2.3)
#>    fs            1.6.1      2023-02-06 [1] CRAN (R 4.2.3)
#>    glue          1.6.2      2022-02-24 [1] CRAN (R 4.2.1)
#>    htmltools     0.5.5      2023-03-23 [1] CRAN (R 4.2.3)
#>    htmlwidgets   1.6.2      2023-03-17 [1] CRAN (R 4.2.3)
#>    httpuv        1.6.9      2023-02-14 [1] CRAN (R 4.2.3)
#>    jsonlite      1.8.4      2022-12-06 [1] CRAN (R 4.2.3)
#>    knitr         1.42       2023-01-25 [1] CRAN (R 4.2.3)
#>    later         1.3.0      2021-08-18 [1] CRAN (R 4.2.1)
#>    lifecycle     1.0.3      2022-10-07 [1] CRAN (R 4.2.3)
#>    magrittr      2.0.3      2022-03-30 [1] CRAN (R 4.2.1)
#>    memoise       2.0.1      2021-11-26 [1] CRAN (R 4.2.1)
#>    mime          0.12       2021-09-28 [1] CRAN (R 4.2.1)
#>    miniUI        0.1.1.1    2018-05-18 [1] CRAN (R 4.2.1)
#>    pillar        1.9.0      2023-03-22 [1] CRAN (R 4.2.3)
#>    pkgbuild      1.4.0      2022-11-27 [1] CRAN (R 4.2.3)
#>    pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 4.2.1)
#>    pkgload       1.3.2      2022-11-16 [1] CRAN (R 4.2.3)
#>    prettyunits   1.1.1      2020-01-24 [1] CRAN (R 4.2.1)
#>    processx      3.8.0      2022-10-26 [1] CRAN (R 4.2.3)
#>    profvis       0.3.7      2020-11-02 [1] CRAN (R 4.2.1)
#>    promises      1.2.0.1    2021-02-11 [1] CRAN (R 4.2.1)
#>    ps            1.7.3      2023-03-21 [1] CRAN (R 4.2.3)
#>    purrr         1.0.1      2023-01-10 [1] CRAN (R 4.2.3)
#>  P qgisprocess * 0.0.0.9000 2022-08-17 [?] Github (paleolimbot/qgisprocess@81d0d66)
#>    R.cache       0.16.0     2022-07-21 [1] CRAN (R 4.2.1)
#>    R.methodsS3   1.8.2      2022-06-13 [1] CRAN (R 4.2.1)
#>    R.oo          1.25.0     2022-06-12 [1] CRAN (R 4.2.1)
#>    R.utils       2.12.2     2022-11-11 [1] CRAN (R 4.2.3)
#>    R6            2.5.1      2021-08-19 [1] CRAN (R 4.2.1)
#>    rappdirs      0.3.3      2021-01-31 [1] CRAN (R 4.2.1)
#>    Rcpp          1.0.10     2023-01-22 [1] CRAN (R 4.2.3)
#>    remotes       2.4.2      2021-11-30 [1] CRAN (R 4.2.1)
#>    reprex        2.0.2      2022-08-17 [1] CRAN (R 4.2.1)
#>    rlang         1.1.0      2023-03-14 [1] CRAN (R 4.2.3)
#>    rmarkdown     2.21       2023-03-26 [1] CRAN (R 4.2.3)
#>    rprojroot     2.0.3      2022-04-02 [1] CRAN (R 4.2.1)
#>    rstudioapi    0.14       2022-08-22 [1] CRAN (R 4.2.3)
#>    sessioninfo   1.2.2      2021-12-06 [1] CRAN (R 4.2.1)
#>    shiny         1.7.4      2022-12-15 [1] CRAN (R 4.2.3)
#>    stringi       1.7.12     2023-01-11 [1] CRAN (R 4.2.3)
#>    stringr       1.5.0      2022-12-02 [1] CRAN (R 4.2.3)
#>    styler        1.9.1      2023-03-04 [1] CRAN (R 4.2.3)
#>    terra       * 1.7-18     2023-03-06 [1] CRAN (R 4.2.3)
#>    testthat    * 3.1.7      2023-03-12 [1] CRAN (R 4.2.3)
#>    tibble        3.2.1      2023-03-20 [1] CRAN (R 4.2.3)
#>    urlchecker    1.0.1      2021-11-30 [1] CRAN (R 4.2.1)
#>    usethis       2.1.6      2022-05-25 [1] CRAN (R 4.2.1)
#>    utf8          1.2.3      2023-01-31 [1] CRAN (R 4.2.3)
#>    vctrs         0.6.1      2023-03-22 [1] CRAN (R 4.2.3)
#>    withr         2.5.0      2022-03-03 [1] CRAN (R 4.2.1)
#>    xfun          0.38       2023-03-24 [1] CRAN (R 4.2.3)
#>    xtable        1.8-4      2019-04-21 [1] CRAN (R 4.2.1)
#>    yaml          2.3.7      2023-01-23 [1] CRAN (R 4.2.3)
#> 
#>  [1] /home/jannes/R/x86_64-pc-linux-gnu-library/4.2
#>  [2] /usr/lib/R/library
#> 
#>  P ── Loaded and on-disk path mismatch.
#> 
#> ──────────────────────────────────────────────────────────────────────────────

@jannes-m
Copy link
Collaborator Author

Just fyi, using UseMethod("qgis_run_algorithm", .data), we could ensure that method dispatch always depends on .data. For consistency reasons and an easier debugging experience I like this better, so I will update the PR accordingly 😄

Copy link
Member

@florisvdh florisvdh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Congratulations @jannes-m, as far as I can imagine so far, this should work quite well! I simply didn't want to go as far as you did with adding .data in qgis_run_algorithm(). That's a choice to be made carefully though.

The separate review comments are about further improvement (not questioning the change as such), as is the following one:

  • for .data, several other types are to be potentially captured before going to the default method, which essentially ignores .data: at least list, qgis_list_input, qgis_dict_input, and the different possible qgis_output* classes, logical and numeric. These were all covered in the default method of qgis_pipe().

E.g. following line from qgis_pipe.default() is not yet mimicked:

if (stringr::str_detect(class(.data), "^qgis_output")) {
    .data <- unclass(.data)
  }

About the choice to make such change: it is still questionable to me whether making qgis_run_algorithm() pipe-friendly is worth the trouble if that is at the cost of putting a rather strange argument at the front (seen from an average user's perspective), which makes the function less easy to understand. Also it breaks existing scripts (to avoid that, we would need to choose a new name and still support/deprecate previous one). And it takes more care in supporting all possible .data classes (see above), since you can't use the default method for that (contrary to qgis_pipe()). You'd also need to rework many tests in the package... (And maybe, maybe not, there are still some overlooked consequences, I didn't yet overthink and check all rabbit holes).

I'm really impressed of the possibility created by this nice piece of work! Still I currently incline to being more conservative by providing an as-simple-as-possible function outside the piping context. To me, being able to pipe easily is very welcome and wanted, but splitting that over two more straightforward functions (i.e. with a more limited list of different arguments) is also a way to user-friendliness (similar trade-off as in #140 (comment) BTW, but there the argument ordering feels natural in both cases).

Anyway the experiment is very valuable and you are certainly free to perfection it further. But the question is if it will be the best way to cater for the user (hence to take a decision to actually adopt this).

Further comments by others are most welcome of course. What's your thought @paleolimbot ?

!!!qgis_algorithm_args,
.data = NULL,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not quite sure, but probably .data = NULL should also be added to the list in line 93 above:

default_run_alg_args <- list(PROJECT_PATH = NULL, ELLIPSOID = NULL, .quiet = TRUE)

Comment on lines +5 to +7
qgis_run_algorithm <- function(.data = NULL,
algorithm, ..., PROJECT_PATH = NULL, ELLIPSOID = NULL,
.raw_json_input = NULL, .quiet = FALSE) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here you'll need to add .select = "OUTPUT" and .clean = TRUE as well. Not to make those arguments work, but for the documentation (only the generic will be shown in the documentation, the methods are hidden from documentation and pkg index).

#' Run QGIS algorithms.
#' See the [QGIS docs](https://docs.qgis.org/testing/en/docs/user_manual/processing_algs/qgis/index.html)
#' for a detailed description of the algorithms provided
#' 'out of the box' on QGIS (versions >= 3.14).
#'
#' @param .data some data needed for piping
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#' @param .data some data needed for piping
#' @param .data Optional.
#' Passed to the first input of `algorithm`.
#' If `.data` is a `qgis_result` (the result of a previous processing
#' step), `.data[[.select]]` is passed instead.
#' It can also be one of the qgis_output* classes
#' (e.g. `qgis_outputVector`, `qgis_outputRaster`),
#' a character, a numeric, a logical, a list or any other
#' possible algorithm input argument type.

The 'pipe-friendly' character of the function can be described in general. It is not necessary for this argument to be used in a piping context.

.clean = TRUE,
.quiet = TRUE
) {
browser()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to be dropped?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reviewing and your comments, @florisvdh. And no worries, I completely agree with you, the .data argument feels somehow wrong. And yes, qgisprocess does not lend itself naturally to piping, so in my opinion, if you have to pipe, use the native pipe and its placeholder to make crystal clear what you intend to do. So probably, the qgis_pipe() is a good compromise, so let's go with that and close this PR.

@jannes-m jannes-m closed this Mar 30, 2023
@florisvdh
Copy link
Member

Thanks for reviewing and your comments, @florisvdh. And no worries, I completely agree with you, the .data argument feels somehow wrong. And yes, qgisprocess does not lend itself naturally to piping, so in my opinion, if you have to pipe, use the native pipe and its placeholder to make crystal clear what you intend to do. So probably, the qgis_pipe() is a good compromise, so let's go with that and close this PR.

Thanks for sharing these considerations @jannes-m, and for your enthusiasm and open mindset!

@jannes-m jannes-m deleted the qgis-run-alg-class branch April 5, 2023 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants