Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve type stability of dynamic branching #1106

Merged
merged 4 commits into from
Dec 10, 2019
Merged

Improve type stability of dynamic branching #1106

merged 4 commits into from
Dec 10, 2019

Conversation

wlandau
Copy link
Member

@wlandau wlandau commented Dec 10, 2019

Summary

This PR improves the type stability of dynamic branching. It changes the return values from group() dynamic targets and from readd(), so this is technically a breaking change. However, dynamic branching is still new and this change is important enough that a little discomfort is worth it in the end. Type stability is too important, as I discuss in #1087 (comment) and #1105 (comment).

Previously, readd() always returned a list for dynamic targets, as I demonstrated in #1105 (comment). Now, it returns the result of vec_c() which respects the types of the sub-targets.

library(drake)
plan <- drake_plan(
  x = seq_len(2),
  y = mtcars[x, ],
  z = target(y, dynamic = group(y, .by = x))
)
make(plan)
#> target x
#> target y
#> dynamic z
#> subtarget z_25909f33
#> subtarget z_7fb9021c
#> aggregate z

readd(z)
#>   mpg cyl disp  hp drat    wt  qsec vs am gear carb
#> 1  21   6  160 110  3.9 2.620 16.46  0  1    4    4
#> 2  21   6  160 110  3.9 2.875 17.02  0  1    4    4

Created on 2019-12-10 by the reprex package (v0.3.0)

Similarly, group()'ed dynamic dependencies were always lists before. Now, they respect the types of the sub-targets.

library(drake)
plan <- drake_plan(
  x = c(1, 1, 2, 2),
  y = mtcars[seq_len(4), ],
  z = target(y, dynamic = group(y, .by = x))
)
make(plan)
#> In drake, consider r_make() instead of make(). r_make() runs make() in a fresh R session for enhanced robustness and reproducibility.
#> target x
#> target y
#> dynamic z
#> subtarget z_6f19200e
#> subtarget z_e9abea21
#> aggregate z

readd(subtargets(z)[1], character_only = TRUE)
#>               mpg cyl disp  hp drat    wt  qsec vs am gear carb
#> Mazda RX4      21   6  160 110  3.9 2.620 16.46  0  1    4    4
#> Mazda RX4 Wag  21   6  160 110  3.9 2.875 17.02  0  1    4    4

readd(z)
#>    mpg cyl disp  hp drat    wt  qsec vs am gear carb
#> 1 21.0   6  160 110 3.90 2.620 16.46  0  1    4    4
#> 2 21.0   6  160 110 3.90 2.875 17.02  0  1    4    4
#> 3 22.8   4  108  93 3.85 2.320 18.61  1  1    4    1
#> 4 21.4   6  258 110 3.08 3.215 19.44  1  0    3    1

Created on 2019-12-10 by the reprex package (v0.3.0)

In some ways, we need to be more careful about how we aggregate targets. drake will no longer automatically compartmentalize sub-targets in lists for us. When we aggregate sub-targets, we may get a value we do not expect (i.e. an unexpectedly long vector) but I think this PR is still the right way to go.

library(drake)
plan <- drake_plan(
  w = c(1, 1, 2, 2),
  x = letters[seq_len(4)],
  y = LETTERS[seq_len(4)],
  z = target(c(x, y), dynamic = group(x, y, .by = w))
)
make(plan)
#> target w
#> target x
#> target y
#> dynamic z
#> subtarget z_bf3d8724
#> subtarget z_b9172820
#> aggregate z

readd(z)
#> [1] "a" "b" "A" "B" "c" "d" "C" "D"

readd(subtargets(z)[1], character_only = TRUE)
#> [1] "a" "b" "A" "B"

Created on 2019-12-10 by the reprex package (v0.3.0)

Related GitHub issues and pull requests

Checklist

@codecov-io
Copy link

codecov-io commented Dec 10, 2019

Codecov Report

Merging #1106 into master will not change coverage.
The diff coverage is 100%.

Impacted file tree graph

@@          Coverage Diff           @@
##           master   #1106   +/-   ##
======================================
  Coverage     100%    100%           
======================================
  Files          47      47           
  Lines        7424    7413   -11     
======================================
- Hits         7424    7413   -11
Impacted Files Coverage Δ
R/drake_plan.R 100% <ø> (ø) ⬆️
R/drake_plan_helpers.R 100% <ø> (ø) ⬆️
R/make.R 100% <ø> (ø) ⬆️
R/dynamic.R 100% <100%> (ø) ⬆️
R/manage_memory.R 100% <100%> (ø) ⬆️
R/cache.R 100% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1518467...1966ac3. Read the comment docs.

@wlandau wlandau merged commit efcaa46 into master Dec 10, 2019
@wlandau wlandau deleted the 1105 branch December 10, 2019 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants