[r/python] Python str, R character, and Python bytes are cast to large variants

[mojaveazure]

When writing a meta data column consisting of a str or bytes in Python, or character in R, types are cast to large variants when reading back (Arrow Large UTF8 for str and character, Arrow Large Binary for bytes); I don't know if this is intentional, but if it is this behaivor should be documented somewhere

R example:

> df <- data.frame(soma_joinid = bit64::seq.integer64(0L, 99L), str = character(100L))
> tbl <- arrow::as_arrow_table(df)
> sdf <- SOMADataFrameCreate("df-raw", schema = tbl$schema, domain = list(soma_joinid = c(0L, 100L)))
> sdf$write(tbl)
> sdf$close()
> sdf <- SOMADataFrameOpen(uri)
> sdf$read()$concat()
[1] Table
100 rows x 3 columns
$soma_joinid <int64 not null>
$str <large_string>

[johnkerl]

@mojaveazure this is 100% intentional and is documented at least in various comments around the codebase (which I can collect for you at some point). Agreed the documentation needs to be clearer / more centralized.

More info later, but for now:

• Arrow has string / binary which have 32-bit offsets and large_string / large_binary which have 64-bit offsets
• TileDB Embedded only has 64-bit offsets
• We decided collectively and as a team that on outgest we would only produce the large variants

[mojaveazure]

this is 100% intentional

Great, as long as it's intentional and not a bug then I'm 100% on-board. But yeah, more documentation, especially user-facing so that users aren't confused as to why it's one type going in and one type coming out, would be great