Any plans for out-of-memory data formats?

[hope-data-science]

In many times, the memory of the computer might not be large enough, and for each computation we don't really have to read all the data into RAM to carry out the computation. Could we possibly parse a csv instead of reading it into a RAM. I've seen this design in fst, with a new class named fst_table, and modify it to be used in tidyfst. I've tested the performance as well in another work (https://hope-data-science.github.io/tidyft/articles/Introduction.html).

Any plans to make it work for csv in data.table? I think this might be even more memory efficient and time saving with the powerful fread and fwrite.

Thanks.

[MichaelChirico]

I think this is covered in #1336 #1721 and #3104 ; please re-open if there's something novel not mentioned in those