Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix int_pctl.tune_results() with PSOCK #887

Merged
merged 1 commit into from
Apr 17, 2024
Merged

fix int_pctl.tune_results() with PSOCK #887

merged 1 commit into from
Apr 17, 2024

Conversation

simonpcouch
Copy link
Contributor

Closes #885 using a trick that @topepo showed me some years back to work around this PSOCK/foreach oddity. With this PR:

library(tidymodels)

set.seed(991)
delivery_split <- initial_validation_split(deliveries, prop = c(0.6, 0.2), 
                                           strata = time_to_delivery)
delivery_rs <- validation_set(delivery_split)

lm_res <- 
  linear_reg() %>% 
  fit_resamples(
    time_to_delivery ~ .,
    resamples = delivery_rs,
    control = control_resamples(save_pred = TRUE)
  )

res_seq <- int_pctl(lm_res)

library(doMC)
#> Loading required package: foreach
#> 
#> Attaching package: 'foreach'
#> The following objects are masked from 'package:purrr':
#> 
#>     accumulate, when
#> Loading required package: iterators
#> Loading required package: parallel

registerDoMC(cores = 5)

res_mc <- int_pctl(lm_res)
#> Warning: ! tune detected a parallel backend registered with foreach but no backend
#>   registered with future.
#> ℹ Support for parallel processing with foreach was soft-deprecated in tune
#>   1.2.1.
#> ℹ See ?parallelism (`?tune::parallelism()`) to learn more.

library(doParallel)
cl <- makePSOCKcluster(parallel::detectCores(logical = FALSE))
registerDoParallel(cl)

res_psoc <- int_pctl(lm_res)
#> Warning: ! tune detected a parallel backend registered with foreach but no backend
#>   registered with future.
#> ℹ Support for parallel processing with foreach was soft-deprecated in tune
#>   1.2.1.
#> ℹ See ?parallelism (`?tune::parallelism()`) to learn more.

res_psoc
#> # A tibble: 2 × 6
#>   .metric .estimator .lower .estimate .upper .config             
#>   <chr>   <chr>       <dbl>     <dbl>  <dbl> <chr>               
#> 1 rmse    bootstrap   3.59      3.76   3.96  Preprocessor1_Model1
#> 2 rsq     bootstrap   0.689     0.713  0.735 Preprocessor1_Model1

Created on 2024-04-17 with reprex v2.1.0

Since we've discussed writing out the internal comp_metrics() function before—and since it's named so similarly to compute_metrics()—this seems preferable to me to newly exporting it.

@simonpcouch
Copy link
Contributor Author

Wanted to make sure the usual suspects with future do okay as well:

library(tidymodels)
library(future)
library(future.mirai)

set.seed(991)
delivery_split <- initial_validation_split(deliveries, prop = c(0.6, 0.2), 
                                           strata = time_to_delivery)
delivery_rs <- validation_set(delivery_split)

lm_res <- 
  linear_reg() %>% 
  fit_resamples(
    time_to_delivery ~ .,
    resamples = delivery_rs,
    control = control_resamples(save_pred = TRUE)
  )

plan(multisession, workers = 4)

int_pctl(lm_res)
#> # A tibble: 2 × 6
#>   .metric .estimator .lower .estimate .upper .config             
#>   <chr>   <chr>       <dbl>     <dbl>  <dbl> <chr>               
#> 1 rmse    bootstrap   3.59      3.76   3.96  Preprocessor1_Model1
#> 2 rsq     bootstrap   0.689     0.713  0.734 Preprocessor1_Model1

plan(cluster, workers = 4)

int_pctl(lm_res)
#> # A tibble: 2 × 6
#>   .metric .estimator .lower .estimate .upper .config             
#>   <chr>   <chr>       <dbl>     <dbl>  <dbl> <chr>               
#> 1 rmse    bootstrap   3.59      3.76   3.96  Preprocessor1_Model1
#> 2 rsq     bootstrap   0.689     0.713  0.734 Preprocessor1_Model1

plan(mirai_multisession, workers = 4)

int_pctl(lm_res)
#> # A tibble: 2 × 6
#>   .metric .estimator .lower .estimate .upper .config             
#>   <chr>   <chr>       <dbl>     <dbl>  <dbl> <chr>               
#> 1 rmse    bootstrap   3.59      3.76   3.96  Preprocessor1_Model1
#> 2 rsq     bootstrap   0.689     0.713  0.734 Preprocessor1_Model1

Created on 2024-04-17 with reprex v2.1.0

@simonpcouch simonpcouch requested a review from topepo April 17, 2024 12:10
@topepo topepo merged commit 668eec2 into main Apr 17, 2024
9 checks passed
@topepo topepo deleted the fix-885 branch April 17, 2024 15:43
topepo added a commit that referenced this pull request Apr 18, 2024
Copy link

github-actions bot commented May 2, 2024

This pull request has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.

@github-actions github-actions bot locked and limited conversation to collaborators May 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

int_pctl.tune_results fails with PSOCK
2 participants