Handle all location types: geopoint, geotrace, geoshape

[florianm]

Spatial musings

ODK Collect can capture points (geopoints), lines (geotrace), and polygons (geoshape).
ODK Central's 0.7 OData submissions export geopoints either as GeoJSON (ODK Central and ruODK default) or WKT.
ODK Central's 0.7 OData submissions export geotrace and geoshape always as GeoJSON.
Since v0.8, GeoJSON and WKT export seems to work across all geo field types.

What does a data analyst want to do with each location type?
How much parsing should ruODK provide?
As a data analyst, I need to:

• extract simple lon/lat columns from each geo type - easy for points, harder for trace/shape: centroid lon/lat, or first point lon/lat
• export data as spreadsheets to share with minimally tech savvy users - requires lon/lat in separate columns for each geo type
• create a map (leaflet) and put geopoints, geotraces, geoshapes on it - requires a format leaflet can map plus working examples in a new vignette "spatial is not special"

Which formats should ruODK output:

• sf::sfc_POLYGON via sf::st_as_sfc() (note sf github actions is broken, introducing sf to ruODK will shaft GHA)
• st::SpatialPolygons via rgeos::readWKT()
• data model considerations: ODK forms can have multiple location columns, R spatial objects have exactly and up to one geometry column
• performance consideration when creating spatial types (by attribute or by feature?)

Implementation

Current behaviour

• GeoPoint if WKT is split into _{longitude, latitude, altitude}
• GeoPoint if GeoJSON is split during rectangling into anonymous fields _{11,12,13} (or other numbers)
• Geotrace and geoshape are not handled and remain GeoJSON

Suggested new behaviour (discuss)

• Retain the original column to allow parsing into spatially enabled formats (st_point, sfg and friends)
• Annotate fields with _{longitude, latitude, altitude, accuracy}
• For geotraces and geoshapes, which point should be extracted? Options: first point, centroid. I'd go for first point, as that's actually part of the shape.
• An obvious scope boundary is introducing new dependencies. If we just extract point coordinates we don't need sf, st, rgeos, rgdal which can be a pain to install. We could provide examples for going full spatial as a starting point.

Probably out of scope

• Do more with the geopoint - turn into sfg (simple feature geometry) in addition to splitting out coordinates (great to have for leaflet maps)
• Parse geotrace and geoshape into sfg (keep GeoJSON? always parse?) First cut here.

Challenges

odata_submission_rectangle needs to exempt geo{point, trace, shape} from unnesting

odata_submission_rectangle will blindly (as it has no form introspection ...yet) unnest any list column. This works OK-ish for GeoJSON points as these have a set length. Unnesting geotraces and geoshapes will inevitably end in tears over their variable number of points.

This means that odata_submission_get(parse = TRUE, wkt = FALSE) will fail with geotraces or geoshapes.

Therefore odata_submission_rectangle needs a parameter form_schema = NULL to pass an optional form schema, extract field names of geo{point, trace, shape}s, and exempt those from rectangling. This would preserve GeoJSON and parse GeoJSON into list columns, which then could be further parsed into spatial classes.

Getting too funky with spatial classes will add dependencies

Downside: sf, st, rgdal, rgeos dependencies could be a pain for users to install.

Upside: Adding some nice helpers and working examples (mapping, spatial operations, coordinate extraction) would be immensely useful to less spatially versed R users.

Testing

Test form: https://sandbox.central.getodk.org/#/projects/14/forms/build_Locations_1589344221/submissions

Added to CONTRIBUTING.md / Test:

ODKC_TEST_FID_WKT="build_Locations_1589344221"

As DBCA still runs a production server on ODK Central v0.6, and the spatial output has changed since then, I will address this issue after migrating our server. ETA mid 2020.

Checklist

Once spatial parsing has changed, make sure to update all the docs and examples.

• Add example data containing all geofield types for each option: GeoJSON / WKT, parsed / raw, plus form_schema. Use in tests and examples. Keep example data up to date with package functions.
• Add source for geofield example form to inst/extdata
• Function examples (odata_submission_get, _rectangle, handle_ru_geo*)
• Update vignettes
• Update skeleton.Rmd

[florianm]

Current status:

• GHA test suite passes
• odata_submission_rectangle excludes spatial fields from unnesting if given a form_schema
• ruODK handles geopoint, geotrace, geoshape
• original values are parsed but retained (GeoJSON > nested list, WKT > text), additionally, point coords (using first point for trace and shape) are extracted into lon, lat, alt, and acc (where given)
• location test data included
• vignettes are updated (WIP - good enough for now but added comments with ideas for improvement)

This issue could make it into ropensci/software-review#335