full_join generates <NA> entries when joining character vectors with different encodings

[jarauh]

This is another facet of the wellknown encodings problem (e.g. #1885). Sorry if the example is more complicated than necessary.

library(dplyr)
x <- "fa\xE7ile"
xx <- iconv(x, "latin1", "UTF-8")

x == xx  # TRUE

left <- matrix(c(x, "facile", "1", "2"), ncol = 2)
colnames(left) <- c("c1", "c2")
left <- data.frame(left, stringsAsFactors = FALSE)
right <- matrix(c(xx, "facile", "A", "B"), ncol = 2)
colnames(right) <- c("c1", "c3")
right <- data.frame(right, stringsAsFactors = FALSE)

full_join(left, right, by = "c1")

Output:

      c1   c2   c3
1 façile    1 <NA>
2 facile    2    B
3   <NA> <NA>    A

Note the last row that contains an entry in the column that is used for joining.

[krlmlr]

Thanks. I think you should be using UTF-8 only for column data, dplyr will be more careful about that in the future.

[jarauh]

@krlmlr I agree. I can work around this, but I wanted to report it, since even knowing that comparing strings with different encodings is problematic, having new entries appear in the by-column when doing a full_join seems like a separate bug.