Directly accept lists of target objects

[wlandau]

Prework

• Read and agree to the code of conduct and contributing guidelines.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.
• New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first.
• Format your code according to the tidyverse style guide.

Issue

Users currently need to go through tar_pipeline(), which is mysterious for new users and annoying for experienced ones.

# _targets.R
library(targets)
tar_pipeline(
  tar_target(data, get_data()),
  tar_target(analysis, analyze_data(data))
)

Proposal

Directly accept lists of target objects instead. It's easier and more convenient, and it alludes to how you can construct pipelines piecemeal.

# _targets.R
library(targets)
list(
  tar_target(data, get_data()),
  tar_target(analysis, analyze_data(data))
)

Let's deprecate tar_pipeline() and tar_bind() to indicate the transition.

FYI @limnoliver and @tjmahr

[liutiming]

Like this idea! One caveat I thought of is this: I am planning to put groups of targets into scripts and source them - current _target.R is 250 lines long and is getting longer. In this case will it still work? I think it probably can by combining different targets lists with c() at the end of _targets.R but just to let you know about this use case.

[liutiming]

Edit: I realise you must have been aware of this from comment here #252 (comment)

[wlandau]

Yeah, that technique just got a lot easier. You can just combine those lists of targets with c() or list() (you can have lists within lists). No need to go through tar_pipeline() or tar_bind() anymore.

[arnold-c]

@wlandau I just downloaded the updated version to try and implement the new changes, and unfortunately I'm getting the error Error : invalid pipeline. Error: callr subprocess failed: invalid pipeline.. A simplified command used is below (have tested that this also doesn't work.

library(targets)
library(tarchetypes)

# Set target-specific options such as packages.
tar_option_set(packages = c( "here", "readr", "tidyverse"))

list(
    tar_target(raw_data_file, here::here("data", "path/to/data.csv"), format = "file"),
    tar_target(raw_data, read_csv(raw_data_file, col_types = cols()))
)

The error trace is:

Stack trace:

 Process 64737:
 1. targets:::tar_make()
 2. targets:::callr_outer(targets_function = tar_make_inner, targets_arguments = targets_arguments,  ...
 3. targets:::trn(is.null(callr_function), callr_inner(target_script_path(),  ...
 4. base:::do.call(callr_function, prepare_callr_arguments(callr_function,  ...
 5. (function (func, args = list(), libpath = .libPaths(), repos = default_repos(),  ...
 6. callr:::get_result(output = out, options)
 7. throw(newerr, parent = remerr[[2]])

 x callr subprocess failed: invalid pipeline. 

 Process 87556:
 19. (function (targets_script, targets_function, targets_arguments)  ...
 20. base:::do.call(targets_function, targets_arguments)
 21. (function (pipeline, names_quosure, reporter)  ...
 22. targets:::pipeline_validate_lite(pipeline)
 23. targets:::pipeline_validate_lite.default(pipeline)
 24. targets:::throw_validate("invalid pipeline.")
 25. base:::stop(condition_validate(...))
 26. (function (e)  ...

 x invalid pipeline. 

However, using tar_pipeline() does work (in both more complex and simpler versions), just showing a deprecation message.

library(targets)
library(tarchetypes)

# Set target-specific options such as packages.
tar_option_set(packages = c( "here", "readr", "tidyverse"))

tar_pipeline(
    tar_target(raw_data_file, here::here("data", "path/to/data.csv"), format = "file"),
    tar_target(raw_data, read_csv(raw_data_file, col_types = cols()))
)

[wlandau]

I cannot reproduce that error. Is it possible that you installed an earlier commit (mid-PR for #254)? Here is what I get using d112af5:

library(readr)
library(targets)
write_csv(mtcars, "data.csv")
tar_script({
  library(targets)
  tar_option_set(packages = c( "here", "readr", "tidyverse"))
  list(
    tar_target(raw_data_file, "data.csv", format = "file"),
    tar_target(raw_data, read_csv(raw_data_file, col_types = cols()))
  )
})

tar_make()
#> ● run target raw_data_file
#> ● run target raw_data

^{Created on 2021-01-03 by the reprex package (v0.3.0)}

Session info

devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 4.0.2 (2020-06-22)
#>  os       Ubuntu 18.04.5 LTS          
#>  system   x86_64, linux-gnu           
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_US.UTF-8                 
#>  ctype    en_US.UTF-8                 
#>  tz       America/New_York            
#>  date     2021-01-03                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version    date       lib source        
#>  assertthat    0.2.1      2019-03-21 [1] CRAN (R 4.0.2)
#>  callr         3.5.1      2020-10-13 [1] CRAN (R 4.0.2)
#>  cli           2.2.0      2020-11-20 [1] CRAN (R 4.0.2)
#>  codetools     0.2-18     2020-11-04 [1] CRAN (R 4.0.2)
#>  crayon        1.3.4      2017-09-16 [1] CRAN (R 4.0.2)
#>  data.table    1.13.4     2020-12-08 [1] CRAN (R 4.0.2)
#>  desc          1.2.0      2018-05-01 [1] CRAN (R 4.0.2)
#>  devtools      2.3.2      2020-09-18 [1] CRAN (R 4.0.2)
#>  digest        0.6.27     2020-10-24 [1] CRAN (R 4.0.2)
#>  ellipsis      0.3.1      2020-05-15 [1] CRAN (R 4.0.2)
#>  evaluate      0.14       2019-05-28 [1] CRAN (R 4.0.2)
#>  fansi         0.4.1      2020-01-08 [1] CRAN (R 4.0.2)
#>  fs            1.5.0      2020-07-31 [1] CRAN (R 4.0.2)
#>  glue          1.4.2      2020-08-27 [1] CRAN (R 4.0.2)
#>  highr         0.8        2019-03-20 [1] CRAN (R 4.0.2)
#>  hms           0.5.3      2020-01-08 [1] CRAN (R 4.0.2)
#>  htmltools     0.5.0      2020-06-16 [1] CRAN (R 4.0.2)
#>  igraph        1.2.6      2020-10-06 [1] CRAN (R 4.0.2)
#>  knitr         1.30       2020-09-22 [1] CRAN (R 4.0.2)
#>  lifecycle     0.2.0      2020-03-06 [1] CRAN (R 4.0.2)
#>  magrittr      2.0.1      2020-11-17 [1] CRAN (R 4.0.2)
#>  memoise       1.1.0      2017-04-21 [1] CRAN (R 4.0.2)
#>  pillar        1.4.7      2020-11-20 [1] CRAN (R 4.0.2)
#>  pkgbuild      1.1.0      2020-07-13 [1] CRAN (R 4.0.2)
#>  pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 4.0.2)
#>  pkgload       1.1.0      2020-05-29 [1] CRAN (R 4.0.2)
#>  prettyunits   1.1.1      2020-01-24 [1] CRAN (R 4.0.2)
#>  processx      3.4.5      2020-11-30 [1] CRAN (R 4.0.2)
#>  ps            1.5.0      2020-12-05 [1] CRAN (R 4.0.2)
#>  purrr         0.3.4      2020-04-17 [1] CRAN (R 4.0.2)
#>  R6            2.5.0      2020-10-28 [1] CRAN (R 4.0.2)
#>  readr       * 1.4.0      2020-10-05 [1] CRAN (R 4.0.2)
#>  remotes       2.2.0      2020-07-21 [1] CRAN (R 4.0.2)
#>  rlang         0.4.9      2020-11-26 [1] CRAN (R 4.0.2)
#>  rmarkdown     2.5        2020-10-21 [1] CRAN (R 4.0.2)
#>  rprojroot     2.0.2      2020-11-15 [1] CRAN (R 4.0.2)
#>  sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 4.0.2)
#>  stringi       1.5.3      2020-09-09 [1] CRAN (R 4.0.2)
#>  stringr       1.4.0      2019-02-10 [1] CRAN (R 4.0.2)
#>  targets     * 0.0.0.9004 2021-01-03 [1] local         
#>  testthat      3.0.0      2020-10-31 [1] CRAN (R 4.0.2)
#>  tibble        3.0.4      2020-10-12 [1] CRAN (R 4.0.2)
#>  tidyselect    1.1.0      2020-05-11 [1] CRAN (R 4.0.2)
#>  usethis       2.0.0      2020-12-10 [1] CRAN (R 4.0.2)
#>  vctrs         0.3.6      2020-12-17 [1] CRAN (R 4.0.2)
#>  withr         2.3.0      2020-09-22 [1] CRAN (R 4.0.2)
#>  xfun          0.19       2020-10-30 [1] CRAN (R 4.0.2)
#>  yaml          2.2.1      2020-02-01 [1] CRAN (R 4.0.2)
#> 
#> [1] /home/landau/R/R-4.0.2/library

[arnold-c]

Thanks for the reply. I'm sorry about this, but it turns out that I just needed to restart R - sorry I forgot to check that first.