Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Change funnel values to "open" and "closed" instead of a logical value #526

Merged
merged 1 commit into from
Jan 29, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 13 additions & 13 deletions R/ducktbl.R
Original file line number Diff line number Diff line change
Expand Up @@ -57,14 +57,14 @@
#' In dtplyr and dbplyr, there are no unfunneled frames: collection always needs to be
#' explicit.
#'
#' A funneled duckplyr frame can be converted to an unfunneled one with `as_duckdb_tibble(funnel = FALSE)`.
#' A funneled duckplyr frame can be converted to an unfunneled one with `as_duckdb_tibble(funnel = "open")`.
#' The [collect.duckplyr_df()] method triggers computation and converts to a plain tibble.
#' Other useful methods include [compute_file()] for storing results in a file,
#' and [compute.duckplyr_df()] for storing results in temporary storage on disk.
#'
#' Beyond safety regarding memory usage, funneled frames also allow you
#' to check that all operations are supported by DuckDB:
#' for a funneled frame with `funnel = FALSE`, fallbacks to dplyr are not possible.
#' for a funneled frame with `funnel = "closed"`, fallbacks to dplyr are not possible.
#' As a reminder, computing via DuckDB is currently not always possible,
#' see `vignette("limits")` for the supported operations.
#' In such cases, the original dplyr implementation is used, see [fallback] for details.
Expand Down Expand Up @@ -108,12 +108,12 @@
#'
#' x$a
#'
#' y <- duckdb_tibble(a = 1, .funnel = TRUE)
#' y <- duckdb_tibble(a = 1, .funnel = "closed")
#' y
#' try(length(y$a))
#' length(collect(y)$a)
#' @export
duckdb_tibble <- function(..., .funnel = FALSE) {
duckdb_tibble <- function(..., .funnel = "open") {
out <- tibble::tibble(...)
as_duckdb_tibble(out, funnel = .funnel)
}
Expand All @@ -126,7 +126,7 @@ duckdb_tibble <- function(..., .funnel = FALSE) {
#' @param x The object to convert or to test.
#' @rdname duckdb_tibble
#' @export
as_duckdb_tibble <- function(x, ..., funnel = FALSE) {
as_duckdb_tibble <- function(x, ..., funnel = "open") {
# Handle the funnel arg in the generic, only the other args will be dispatched
as_duckdb_tibble <- function(x, ...) {
UseMethod("as_duckdb_tibble")
Expand All @@ -135,7 +135,7 @@ as_duckdb_tibble <- function(x, ..., funnel = FALSE) {
funnel_parsed <- funnel_parse(funnel)
out <- as_duckdb_tibble(x, ...)

if (funnel_parsed$funnel) {
if (funnel_parsed$funnel == "closed") {
as_funneled_duckplyr_df(
out,
funnel_parsed$allow_materialization,
Expand Down Expand Up @@ -173,11 +173,11 @@ funnel_parse <- function(funnel, call = caller_env()) {
}
}
allow_materialization <- is.finite(n_rows) || is.finite(n_cells)
funnel <- TRUE
} else if (!is.logical(funnel)) {
cli::cli_abort("{.arg funnel} must be a logical scalar or a named vector", call = call)
funnel <- "closed"
} else if (!is.character(funnel)) {
cli::cli_abort("{.arg funnel} must be an unnamed character vector or a named numeric vector", call = call)
} else {
allow_materialization <- !isTRUE(funnel)
allow_materialization <- !identical(funnel, "closed")
}

list(
Expand All @@ -195,7 +195,7 @@ as_duckdb_tibble.tbl_duckdb_connection <- function(x, ...) {
con <- dbplyr::remote_con(x)
sql <- dbplyr::remote_query(x)

read_sql_duckdb(sql, funnel = TRUE, con = con)
read_sql_duckdb(sql, funnel = "closed", con = con)
}

#' @export
Expand Down Expand Up @@ -272,7 +272,7 @@ is_duckdb_tibble <- function(x) {

#' @param funnel Only adds the class, does not recreate the relation object!
#' @noRd
new_duckdb_tibble <- function(x, class = NULL, funnel = FALSE, error_call = caller_env()) {
new_duckdb_tibble <- function(x, class = NULL, funnel = "open", error_call = caller_env()) {
if (is.null(class)) {
class <- c("tbl_df", "tbl", "data.frame")
}
Expand All @@ -284,7 +284,7 @@ new_duckdb_tibble <- function(x, class = NULL, funnel = FALSE, error_call = call
}

class(x) <- unique(c(
if (funnel) "funneled_duckplyr_df",
if (!identical(funnel, "open")) "funneled_duckplyr_df",
"duckplyr_df",
class
))
Expand Down
4 changes: 2 additions & 2 deletions R/funnel.R
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,12 @@ is_funneled_duckplyr_df <- function(x) {

get_funnel_duckplyr_df <- function(x) {
if (!is_funneled_duckplyr_df(x)) {
return(FALSE)
return("open")
}

funnel <- attr(x, "funnel")
if (is.null(funnel)) {
return(TRUE)
return("closed")
}

funnel
Expand Down
8 changes: 4 additions & 4 deletions R/io2.R
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ read_parquet_duckdb <- function(path, ..., funnel = c(cells = 1e6), options = li
#' # Materialize explicitly
#' collect(df)$a
#'
#' # Automatic materialization with funnel = FALSE
#' df <- read_csv_duckdb(path, funnel = FALSE)
#' # Automatic materialization with funnel = "open"
#' df <- read_csv_duckdb(path, funnel = "open")
#' df$a
#'
#' # Specify column types
Expand Down Expand Up @@ -150,9 +150,9 @@ duckfun <- function(table_function, args, ..., funnel) {

# Start with funnel, to avoid unwanted materialization
df <- duckdb$rel_to_altrep(rel, allow_materialization = FALSE)
out <- new_duckdb_tibble(df, funnel = TRUE)
out <- new_duckdb_tibble(df, funnel = "closed")

if (!isTRUE(funnel)) {
if (!identical(funnel, "closed")) {
out <- as_duckdb_tibble(out, funnel = funnel)
}

Expand Down
2 changes: 1 addition & 1 deletion R/relational.R
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ check_funneled <- function(x, duckplyr_error, call = caller_env()) {
cli::cli_abort(parent = duckplyr_error_parent, call = call, c(
"This operation cannot be carried out by DuckDB, and the input is a funneled duckplyr frame.",
"*" = duckplyr_error_msg,
"i" = "Use {.code compute(funnel = FALSE)} to materialize to temporary storage and continue with {.pkg duckplyr}.",
"i" = 'Use {.code compute(funnel = "open")} to materialize to temporary storage and continue with {.pkg duckplyr}.',
"i" = 'See {.run vignette("funnel")} for other options.'
))
}
Expand Down
4 changes: 2 additions & 2 deletions R/sql.R
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,9 @@ read_sql_duckdb <- function(sql, ..., funnel = c(cells = 1e6), con = NULL) {
meta_rel_register(rel, expr(duckdb$rel_from_sql(con, !!sql)))

df <- duckdb$rel_to_altrep(rel, allow_materialization = FALSE)
out <- new_duckdb_tibble(df, funnel = TRUE)
out <- new_duckdb_tibble(df, funnel = "closed")

if (!isTRUE(funnel)) {
if (!identical(funnel, "closed")) {
out <- as_duckdb_tibble(out, funnel = funnel)
}

Expand Down
4 changes: 2 additions & 2 deletions man/compute.duckplyr_df.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions man/compute_file.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 5 additions & 5 deletions man/duckdb_tibble.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions man/read_file_duckdb.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion tests/testthat/test-compute_file.R
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ test_that("compute_csv()", {
test_that("compute_csv() funnel", {
df <- data.frame(x = c(1, 2))
withr::defer(unlink("test.csv"))
out <- compute_csv(df, path = "test.csv", funnel = TRUE)
out <- compute_csv(df, path = "test.csv", funnel = "closed")

expect_true(is_funneled_duckplyr_df(out))
expect_identical(collect(out), as_tibble(df))
Expand Down
16 changes: 8 additions & 8 deletions tests/testthat/test-ducktbl.R
Original file line number Diff line number Diff line change
Expand Up @@ -7,23 +7,23 @@ test_that("Can construct", {
expect_identical(duckdb_tibble(a = 1)$a, 1)
})

test_that(".funnel = TRUE forbids materialization", {
tbl <- duckdb_tibble(a = 1, .funnel = TRUE)
test_that('.funnel = "closed" forbids materialization', {
tbl <- duckdb_tibble(a = 1, .funnel = "closed")
expect_error(length(tbl$a))
})

test_that(".funnel = c(rows = ) forbids materialization", {
test_that('.funnel = c(rows = ) forbids materialization', {
tbl <- duckdb_tibble(a = 1:10, .funnel = c(rows = 5))
expect_error(length(tbl$a))
})

test_that(".funnel = c(cells = ) forbids materialization", {
test_that('.funnel = c(cells = ) forbids materialization', {
tbl <- duckdb_tibble(a = 1:10, b = 1, .funnel = c(cells = 10))
expect_error(length(tbl$a))
})

test_that(".funnel = TRUE forbids materialization for as_duckdb_tibble()", {
tbl <- as_duckdb_tibble(data.frame(a = 1), funnel = TRUE)
test_that('.funnel = "closed" forbids materialization for as_duckdb_tibble', {
tbl <- as_duckdb_tibble(data.frame(a = 1), funnel = "closed")
expect_error(length(tbl$a))
})

Expand Down Expand Up @@ -64,14 +64,14 @@ test_that("as_duckdb_tibble() and dbplyr tables", {
dplyr::copy_to(dest = con)

duck <- db_tbl %>%
as_duckdb_tibble(funnel = TRUE) %>%
as_duckdb_tibble(funnel = "closed") %>%
mutate(b = 2)

expect_error(length(duck$b))

db <- db_tbl %>%
mutate(b = 2) %>%
as_duckdb_tibble(funnel = TRUE)
as_duckdb_tibble(funnel = "closed")

expect_error(length(db$b))

Expand Down
6 changes: 3 additions & 3 deletions tests/testthat/test-funnel.R
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
test_that("funneled duckplyr frames will collect", {
tbl <- duckdb_tibble(a = 1, .funnel = TRUE)
tbl <- duckdb_tibble(a = 1, .funnel = "closed")
expect_identical(
collect(tbl),
tibble(a = 1)
Expand All @@ -15,7 +15,7 @@ test_that("unfunneled duckplyr frames are converted to data frames", {
})

test_that("funneled duckplyr frames are converted to data frames", {
tbl <- duckdb_tibble(a = 1, .funnel = TRUE)
tbl <- duckdb_tibble(a = 1, .funnel = "closed")
expect_identical(
as.data.frame(tbl),
data.frame(a = 1)
Expand All @@ -31,7 +31,7 @@ test_that("unfunneled duckplyr frames are converted to tibbles", {
})

test_that("funneled duckplyr frames are converted to tibbles", {
tbl <- duckdb_tibble(a = 1, .funnel = TRUE)
tbl <- duckdb_tibble(a = 1, .funnel = "closed")
expect_identical(
as_tibble(tbl),
tibble(a = 1)
Expand Down
2 changes: 1 addition & 1 deletion tests/testthat/test-sql.R
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ test_that("read_sql_duckdb() works", {
con <- withr::local_db_connection(DBI::dbConnect(duckdb::duckdb()))

expect_identical(
read_sql_duckdb("SELECT 1 AS a", con = con, funnel = FALSE),
read_sql_duckdb("SELECT 1 AS a", con = con, funnel = "open"),
duckdb_tibble(a = 1L)
)
})
2 changes: 1 addition & 1 deletion vignettes/developers.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ Learn more about usethis at <https://usethis.r-lib.org/>.

The default mode for `as_duckdb_tibble()` and `duckdb_tibble()` is unfunneled.
This means that the dplyr operations are carried out by DuckDB when possible, and also available as data frames upon first request.
Use `as_duckdb_tibble(funnel = TRUE)` or `duckdb_tibble(.funnel = TRUE)` to avoid materializing intermediate data and to ensure that all operations are carried out by DuckDB or fail.
Use `as_duckdb_tibble(funnel = "closed")` or `duckdb_tibble(.funnel = "closed")` to avoid materializing intermediate data and to ensure that all operations are carried out by DuckDB or fail.
Funneling can also limit the number of rows or cells that are materialized:

```{r}
Expand Down
8 changes: 4 additions & 4 deletions vignettes/funnel.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ The example below demonstrates the use of funneled duckplyr frames.
```{r}
flights_funneled <-
flights |>
duckplyr::as_duckdb_tibble(funnel = TRUE)
duckplyr::as_duckdb_tibble(funnel = "closed")
```

In this example, `flights_funneled` is a funneled duckplyr frame.
Expand Down Expand Up @@ -195,7 +195,7 @@ The same pipeline with an unfunneled frame works, but the computation is carried

```{r}
flights_funneled |>
duckplyr::as_duckdb_tibble(funnel = FALSE) |>
duckplyr::as_duckdb_tibble(funnel = "open") |>
group_by(origin) |>
summarize(n = n()) |>
ungroup()
Expand All @@ -207,13 +207,13 @@ See `?fallback` for details on fallbacks, and `vignette("limits")` for the opera

### Unfunneling

A funneled duckplyr frame can be converted to an unfunneled one with `as_duckdb_tibble(funnel = FALSE)`.
A funneled duckplyr frame can be converted to an unfunneled one with `as_duckdb_tibble(funnel = "open")`.
The `collect.duckplyr_df()` method triggers computation and converts to a plain tibble.
The difference between the two is the class of the returned object:

```{r}
flights_funneled |>
duckplyr::as_duckdb_tibble(funnel = FALSE) |>
duckplyr::as_duckdb_tibble(funnel = "open") |>
class()

flights_funneled |>
Expand Down
Loading