Omit GeoJSON properties with null values #206
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In geoarrow/geoarrow-rs#588 I got a bug report about a GeoJSON file that failed to read. The issue here is a column that's numeric for the first N rows but then is
null
for some rowN + 1
. Currently the GeoJSON reader coerces anull
value to the string"null"
:geozero/geozero/src/geojson/geojson_reader.rs
Lines 220 to 221 in 3378dda
This is hard to work with because in my processor I'd have to check every input string value (from any geozero reader, for any column) against the string "null" and manage type conversions. E.g. if the first few rows of a column are the string
"null"
but then I see an f64 value, I need to be able to coerce that column to a Float64Array (which in Arrow has its own null bitmask).I'd argue that the simplest approach in this specific case is to do a no-op for GeoJSON null values. That
{"a": 1, "b": null}
is equivalent to{"a": 1}
. That seems like a much smaller change than, say, adding aColumnValue::Null
variant. But interested to hear others' thoughts.