Fread should exclude UTF-8 BOM from column names

[nigmastar]

Currently, fread seems to include the utf-8 BOM inside the file.

library(data.table)

# Create file

f <- "test_utf-8.csv"
cat(intToUtf8(c(239, 187, 191)), 'a,b,c\n1,2,3\n', file = f, sep = '')

# Import file

dt <- fread(f)
names(dt)
## [1] "a" "b"    "c"

[jangorecki]

is there an easy workaround for that?

[httassadar]

Not sure if the issue title makes sense, why file names?

But the post does describe the issue I really hope it's fixed.

[arunsrinivasan]

The file you provide is incorrectly encoded.

> readBin("test_utf-8.csv", raw(), file.info("test_utf-8.csv")$size)
#  [1] c3 af c2 bb c2 bf 61 2c 62 2c 63 0a 31 2c 32 2c 33 0a

whereas it should be:

#  [1] ef bb bf 61 2c 62 2c 63 0a 31 2c 32 2c 33 0a

And performing fread("test_utf-8.csv") seems to work fine, although the BOM is still included:

ans = fread("test_utf-8.csv")
#    a b c
# 1: 1 2 3
charToRaw(names(ans)[1L])
# [1] ef bb bf 61

[arunsrinivasan]

With this fix, now I get:

charToRaw(names(ans)[1L])
# [1] 61