Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prohibit branching over file target stems. #136

Closed
3 tasks done
wlandau opened this issue Aug 7, 2020 · 1 comment
Closed
3 tasks done

Prohibit branching over file target stems. #136

wlandau opened this issue Aug 7, 2020 · 1 comment

Comments

@wlandau
Copy link
Member

wlandau commented Aug 7, 2020

Prework

  • I understand and agree to the code of conduct.
  • I understand and agree to the contributing guidelines.
  • Be considerate of the maintainer's time and make it as easy as possible to troubleshoot any problems you identify. Read here and here to learn about minimal reproducible examples. Format your code according to the tidyverse style guide to make it easier for others to read.

Description

Right now, if you branch over a file target, that file target should not be a stem because downstream branches do not invalidate if the files change.

Reproducible example

It is possible to incorrectly map over files. Need to lock that down.

options(crayon.enabled = FALSE, tidyverse.quiet = TRUE)
library(targets)
library(tidyverse)
write_csv(tibble(x1 = 1, x2 = 1), "a.csv")
write_csv(tibble(x1 = 1, x2 = 1), "b.csv")
tar_script({
  options(crayon.enabled = FALSE, tidyverse.quiet = TRUE)
  library(readr)
  tar_pipeline(
    tar_target(paths, c("a.csv", "b.csv"), format = "file"),
    tar_target(data, read_csv(paths, col_types = "dd"), pattern = map(paths))
  )
})
tar_make()
#> ● run target paths
#> ● run branch data_5085a9cd
#> ● run branch data_b7dcad9d
write_csv(tibble(x1 = 2, x2 = 2), "b.csv")
tar_make()
#> ● run target paths
#> ✓ skip branch data_5085a9cd
#> ✓ skip branch data_b7dcad9d

Created on 2020-08-07 by the reprex package (v0.3.0)

Desired result

We have two options.

  1. Require paths to already be a pattern.
  2. Invalidate all of data.
@wlandau
Copy link
Member Author

wlandau commented Aug 7, 2020

Let's go with (1). What users actually want here is to invalidate some sub-targets but not others, and this is impossible in the reprex because of the composite hash.

@wlandau wlandau changed the title Prohibit or repair branching over file target stems. Prohibit branching over file target stems. Aug 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant