-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error using tune_race_anova() with spatialsample objects and parallel computing #74
Labels
bug
an unexpected problem or unintended behavior
Comments
Hi @simonpcouch Thanks very much for looking into this. Unfortunately I'm still getting an error message when using parallel computing. I've included a reprex in case it's useful. library(tidymodels)
library(spatialsample)
library(finetune)
library(sf)
#> Linking to GEOS 3.11.2, GDAL 3.7.2, PROJ 9.3.0; sf_use_s2() is TRUE
# Data
data("ames", package = "modeldata")
# Convert to sf object for spatial resampling
ames_sf <- sf::st_as_sf(
x = ames[1:200, ],
coords = c("Longitude", "Latitude"),
crs = 4326
)
# Spatial resampling using the spatialsample package
set.seed(123)
spatial_block_folds <- spatial_block_cv(ames_sf, v = 5)
# Workflow
bart_spec <-
parsnip::bart(trees = tune()) |>
set_mode("regression") |>
set_engine("dbarts")
bart_rec <-
recipe(Sale_Price ~ Year_Built + Bldg_Type + Gr_Liv_Area,
data = ames
)
bart_wflow <-
workflow() |>
add_model(bart_spec) |>
add_recipe(bart_rec)
# Tuning using tune_race_anova()
cores <- parallel::detectCores(logical = FALSE)
cl <- parallel::makePSOCKcluster(cores)
doParallel::registerDoParallel(cl)
tune_race_with_control <-
bart_wflow |>
tune_race_anova(
resamples = spatial_block_folds,
control = control_race(pkgs = "sf")
)
#> Warning: All models failed. Run `show_notes(.Last.tune.result)` for more
#> information.
#> Error in `test_parameters_gls()`:
#> ! There were no valid metrics for the ANOVA model.
#> Backtrace:
#> ▆
#> 1. ├─finetune::tune_race_anova(...)
#> 2. └─finetune:::tune_race_anova.workflow(...)
#> 3. └─finetune:::tune_race_anova_workflow(...)
#> 4. └─finetune:::test_parameters_gls(res, control$alpha, opt_metric_time)
#> 5. └─cli::cli_abort("There were no valid metrics for the ANOVA model.")
#> 6. └─rlang::abort(...)
# Show more information
show_notes(.Last.tune.result)
#> unique notes:
#> ─────────────────────────────────────────────────────
#> Error in `vec_size()`:
#> ! `x` must be a vector, not a <sfc_POINT/sfc> object. Created on 2024-01-22 with reprex v2.0.2 Session infosessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 4.3.2 (2023-10-31 ucrt)
#> os Windows 11 x64 (build 22621)
#> system x86_64, mingw32
#> ui RTerm
#> language (EN)
#> collate English_Australia.utf8
#> ctype English_Australia.utf8
#> tz Australia/Perth
#> date 2024-01-22
#> pandoc 3.1.1 @ C:/Program Files/RStudio/resources/app/bin/quarto/bin/tools/ (via rmarkdown)
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> package * version date (UTC) lib source
#> backports 1.4.1 2021-12-13 [1] CRAN (R 4.3.0)
#> boot 1.3-28.1 2022-11-22 [2] CRAN (R 4.3.2)
#> broom * 1.0.5 2023-06-09 [1] CRAN (R 4.3.1)
#> class 7.3-22 2023-05-03 [2] CRAN (R 4.3.2)
#> classInt 0.4-10 2023-09-05 [1] CRAN (R 4.3.1)
#> cli 3.6.1 2023-03-23 [1] CRAN (R 4.3.1)
#> codetools 0.2-19 2023-02-01 [2] CRAN (R 4.3.2)
#> colorspace 2.1-0 2023-01-23 [1] CRAN (R 4.3.1)
#> data.table 1.14.8 2023-02-17 [1] CRAN (R 4.3.1)
#> dbarts 0.9-23 2023-01-23 [1] CRAN (R 4.3.1)
#> DBI 1.1.3 2022-06-18 [1] CRAN (R 4.3.1)
#> dials * 1.2.0 2023-04-03 [1] CRAN (R 4.3.1)
#> DiceDesign 1.9 2021-02-13 [1] CRAN (R 4.3.1)
#> digest 0.6.33 2023-07-07 [1] CRAN (R 4.3.1)
#> doParallel 1.0.17 2022-02-07 [1] CRAN (R 4.3.1)
#> dplyr * 1.1.4 2023-11-17 [1] CRAN (R 4.3.2)
#> e1071 1.7-13 2023-02-01 [1] CRAN (R 4.3.1)
#> evaluate 0.23 2023-11-01 [1] CRAN (R 4.3.2)
#> fansi 1.0.5 2023-10-08 [1] CRAN (R 4.3.1)
#> fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.3.1)
#> finetune * 1.1.0.9005 2024-01-21 [1] Github (tidymodels/finetune@dfcd6d7)
#> foreach 1.5.2 2022-02-02 [1] CRAN (R 4.3.1)
#> fs 1.6.3 2023-07-20 [1] CRAN (R 4.3.1)
#> furrr 0.3.1 2022-08-15 [1] CRAN (R 4.3.1)
#> future 1.33.1 2023-12-22 [1] CRAN (R 4.3.2)
#> future.apply 1.11.1 2023-12-21 [1] CRAN (R 4.3.2)
#> generics 0.1.3 2022-07-05 [1] CRAN (R 4.3.1)
#> ggplot2 * 3.4.4 2023-10-12 [1] CRAN (R 4.3.1)
#> globals 0.16.2 2022-11-21 [1] CRAN (R 4.3.0)
#> glue 1.6.2 2022-02-24 [1] CRAN (R 4.3.1)
#> gower 1.0.1 2022-12-22 [1] CRAN (R 4.3.0)
#> GPfit 1.0-8 2019-02-08 [1] CRAN (R 4.3.1)
#> gtable 0.3.4 2023-08-21 [1] CRAN (R 4.3.1)
#> hardhat 1.3.0 2023-03-30 [1] CRAN (R 4.3.1)
#> htmltools 0.5.7 2023-11-03 [1] CRAN (R 4.3.2)
#> infer * 1.0.5 2023-09-06 [1] CRAN (R 4.3.1)
#> ipred 0.9-14 2023-03-09 [1] CRAN (R 4.3.1)
#> iterators 1.0.14 2022-02-05 [1] CRAN (R 4.3.1)
#> KernSmooth 2.23-22 2023-07-10 [2] CRAN (R 4.3.2)
#> knitr 1.45 2023-10-30 [1] CRAN (R 4.3.2)
#> lattice 0.21-9 2023-10-01 [2] CRAN (R 4.3.2)
#> lava 1.7.3 2023-11-04 [1] CRAN (R 4.3.2)
#> lhs 1.1.6 2022-12-17 [1] CRAN (R 4.3.1)
#> lifecycle 1.0.4 2023-11-07 [1] CRAN (R 4.3.2)
#> listenv 0.9.0 2022-12-16 [1] CRAN (R 4.3.1)
#> lme4 1.1-34 2023-07-04 [1] CRAN (R 4.3.1)
#> lubridate 1.9.3 2023-09-27 [1] CRAN (R 4.3.1)
#> magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.3.1)
#> MASS 7.3-60 2023-05-04 [2] CRAN (R 4.3.2)
#> Matrix 1.6-1.1 2023-09-18 [1] CRAN (R 4.3.1)
#> minqa 1.2.6 2023-09-11 [1] CRAN (R 4.3.1)
#> modeldata * 1.2.0 2023-08-09 [1] CRAN (R 4.3.1)
#> munsell 0.5.0 2018-06-12 [1] CRAN (R 4.3.1)
#> nlme 3.1-163 2023-08-09 [2] CRAN (R 4.3.2)
#> nloptr 2.0.3 2022-05-26 [1] CRAN (R 4.3.1)
#> nnet 7.3-19 2023-05-03 [2] CRAN (R 4.3.2)
#> parallelly 1.36.0 2023-05-26 [1] CRAN (R 4.3.0)
#> parsnip * 1.1.1.9007 2024-01-21 [1] Github (tidymodels/parsnip@07961a0)
#> pillar 1.9.0 2023-03-22 [1] CRAN (R 4.3.1)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.3.1)
#> prodlim 2023.08.28 2023-08-28 [1] CRAN (R 4.3.1)
#> proxy 0.4-27 2022-06-09 [1] CRAN (R 4.3.1)
#> purrr * 1.0.2 2023-08-10 [1] CRAN (R 4.3.1)
#> R.cache 0.16.0 2022-07-21 [1] CRAN (R 4.3.1)
#> R.methodsS3 1.8.2 2022-06-13 [1] CRAN (R 4.3.0)
#> R.oo 1.25.0 2022-06-12 [1] CRAN (R 4.3.0)
#> R.utils 2.12.2 2022-11-11 [1] CRAN (R 4.3.1)
#> R6 2.5.1 2021-08-19 [1] CRAN (R 4.3.1)
#> Rcpp 1.0.11 2023-07-06 [1] CRAN (R 4.3.1)
#> recipes * 1.0.9 2023-12-13 [1] CRAN (R 4.3.2)
#> reprex 2.0.2 2022-08-17 [1] CRAN (R 4.3.1)
#> rlang 1.1.1 2023-04-28 [1] CRAN (R 4.3.1)
#> rmarkdown 2.25 2023-09-18 [1] CRAN (R 4.3.1)
#> rpart 4.1.21 2023-10-09 [2] CRAN (R 4.3.2)
#> rsample * 1.2.0 2023-08-23 [1] CRAN (R 4.3.1)
#> rstudioapi 0.15.0 2023-07-07 [1] CRAN (R 4.3.1)
#> s2 1.1.4 2023-05-17 [1] CRAN (R 4.3.1)
#> scales * 1.3.0 2023-11-28 [1] CRAN (R 4.3.2)
#> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.3.1)
#> sf * 1.0-15 2023-12-18 [1] CRAN (R 4.3.2)
#> spatialsample * 0.4.0 2023-05-17 [1] CRAN (R 4.3.1)
#> styler 1.10.2 2023-08-29 [1] CRAN (R 4.3.1)
#> survival 3.5-7 2023-08-14 [2] CRAN (R 4.3.2)
#> tibble * 3.2.1 2023-03-20 [1] CRAN (R 4.3.1)
#> tidymodels * 1.1.1 2023-08-24 [1] CRAN (R 4.3.2)
#> tidyr * 1.3.0 2023-01-24 [1] CRAN (R 4.3.1)
#> tidyselect 1.2.0 2022-10-10 [1] CRAN (R 4.3.1)
#> timechange 0.2.0 2023-01-11 [1] CRAN (R 4.3.1)
#> timeDate 4032.109 2023-12-14 [1] CRAN (R 4.3.2)
#> tune * 1.1.2.9011 2024-01-21 [1] Github (tidymodels/tune@ff29ea0)
#> units 0.8-4 2023-09-13 [1] CRAN (R 4.3.1)
#> utf8 1.2.4 2023-10-22 [1] CRAN (R 4.3.2)
#> vctrs 0.6.4 2023-10-12 [1] CRAN (R 4.3.1)
#> withr 3.0.0 2024-01-16 [1] CRAN (R 4.3.2)
#> wk 0.8.0 2023-08-25 [1] CRAN (R 4.3.1)
#> workflows * 1.1.3 2023-02-22 [1] CRAN (R 4.3.1)
#> workflowsets * 1.0.1 2023-04-06 [1] CRAN (R 4.3.1)
#> xfun 0.41 2023-11-01 [1] CRAN (R 4.3.2)
#> yaml 2.3.7 2023-01-23 [1] CRAN (R 4.3.0)
#> yardstick * 1.3.0.9000 2024-01-21 [1] Github (tidymodels/yardstick@66dce0f)
#>
#> [1] C:/Users/jacob/AppData/Local/R/win-library/4.3
#> [2] C:/Program Files/R/R-4.3.2/library
#>
#> ────────────────────────────────────────────────────────────────────────────── |
Ah, I see. #74 should do the trick. :) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The problem
I'm having trouble using the tune_race_anova() function with a spatial_block_cv object from the spatialsample package whilst using parallel computing. I receive the error “There were no valid metrics for the ANOVA model.” and the show_notes(.Last.tune.result) gives Error in
FUN()
: !x
must be a vector, not a <sfc_POINT/sfc> object. The error does not occur if I don't use parallel computing.I get a similar error when using the tune_grid() function from the tune package. However, if I specify control = control_grid(pkgs = "sf"), the tune_grid() function will work with the spatial_block_cv object and parallel computing.
I think the issue is that specifying control = control_race(pkgs = "sf") in the tune_race_anova() function is not being passed to control$pkgs (line 232 in the function code). I can get the tune_race_anova() function to work with parallel computing and a spatial_block_cv object if I modify the function by including “sf” in the list of packages passed to control$pkgs.
The reprex shows tuning using the tune_grid() function, with and without the control = control_grid(pkgs = "sf") argument to show the error and how it is addressed; as well as tuning using the tune_race_anova() function with and without the control = control_race(pkgs = "sf") to show that the error remains.
I did have some success using the workaround suggested for #39 which I have included at the end of the reprex.
Thanks for your help!
Reproducible example
Created on 2023-05-30 with reprex v2.0.2
Session info
The text was updated successfully, but these errors were encountered: