Support latitude2/longitude2 encoding for image

[pixelspark]

The maps functionality of Vega would be a lot more useful if it would also support tiles. This would allow for e.g. satellite or road map backgrounds, or even visualisation of tile data from WMTS services.

I fully understand that Vega is not supposed to be a GIS visualisation toolkit. However I also believe that with a few tweaks it would be possible to at least enable some basic tiling functionality. In vega/vega-lite#6580, support was added for placing images on a map. For "rect" marks, it is already possible to set both "latitude/longitude" and "latitude2/longitude2", which will make the rect span the specified area on the map. For tiling, all that would be needed is to combine the two, e.g. support "latitude2/longitude2" for "image" marks to specify width and height of the image on the map. Then all that is needed is some data set or trick that provides the right tile URLs.

E.g. this currently works:

 {
      "mark": {"type": "rect"},
      "data": {
        "values":  [ 
          {"lat": 52.0, "lng": 5.1, "latto": 53.0, "lngto": 6.1}, ...
        ]
      },
      "encoding": {
        "latitude": {"field": "lat"},
        "longitude": {"field": "lng"},
        "latitude2": {"field": "latto"},
        "longitude2": {"field": "lngto"}
      }

With this change, it should also work for "mark": {"type": "image"}. An example specification that can be pasted in the editor:

{
  "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
  "layer": [
    {
      "mark": {"type": "image", "width":10, "height":10},
      "data": {
        "values":  [ 
         {"lat": 52.0, "lng": 5.1, "latto": 53.0, "lngto": 6.1},
          {"lat": 53.0, "lng": 5.1, "latto": 54.0, "lngto": 6.1},
          {"lat": 53.0, "lng": 3.1, "latto": 54.0, "lngto": 4.1}
        ]
      },
      "encoding": {
        "latitude": {"field": "lat", "type": "quantitative"},
        "longitude": {"field": "lng", "type": "quantitative"},
        "url": {"datum": "https://a.tile.openstreetmap.org/12/2106/1351.png", "type": "nominal"},
        "latitude2": {"field": "latto"},
        "longitude2": {"field": "lngto"}
      }
    },

     {
      "mark": {"type": "rect"},
      "data": {
        "values":  [ 
          {"lat": 52.0, "lng": 5.1, "latto": 53.0, "lngto": 6.1},
          {"lat": 53.0, "lng": 5.1, "latto": 54.0, "lngto": 6.1},
          {"lat": 53.0, "lng": 3.1, "latto": 54.0, "lngto": 4.1}
        ]
      },
      "encoding": {
        "latitude": {"field": "lat", "type": "quantitative"},
        "longitude": {"field": "lng", "type": "quantitative"},
        "latitude2": {"field": "latto"},
        "longitude2": {"field": "lngto"}
      }
    }
  ],
  "projection": {"type": "mercator"},
  "width": 900,
  "height": 500,
  "config": {"view": {"stroke": null}}
}

Given that this is already supported for area I suspect this is easy to implement, though I am not sure if the various projections do things that can be applied to vectors but cannot be applied to raster images. I may be able to implement this change myself if someone can point me in the right directions.

[domoritz]

We should support this. I think we would be pretty close to supporting this hack then: #1212 (comment).

If you look at the diff in vega/vega-lite#7296, you can see which files I had to change. I think the fix for this feature will be very similar.

[mattijn]

Support for basemaps is also raised in this vega lite issue: vega/vega-lite#5758

[jwoLondon]

I would dearly love to see georeferenced images in Vega-Lite. It would open up a lot of interesting cartographic possibilities.

However, I think it would be wrong to follow the rect approach. Georeferencing a rectangle (or image) by only its corner points in projected space will lead to all sorts of poor geo referencing anomalies. Here's an illustrative example:

{
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  
  "layer": [
    {
      "data": { "graticule": { "step": [15,15]} },
      "mark": {
        "type": "geoshape",
        "filled": false,
        "stroke": "black",
        "strokeWidth": 0.1
      }
    },
    {
      "data": {
        "url": "https://gicentre.github.io/data/geoTutorials/world-110m.json",
        "format": {"type": "topojson","feature": "countries1"}
      },
      "mark": { "type": "geoshape","fill": "black","fillOpacity": 0.1} },
    {
      "data": {
        "values": {
          "type": "Feature",
          "geometry": {
            "type": "Polygon",
            "coordinates": [[[-20,70],[40,70],[40,0],[-20,0],[-20,70]]]
          }
        }
      },
      "projection": {
        "type": "orthographic",
        "rotate": [30,-25,0]
      },
      "mark": {
        "type": "geoshape",
        "stroke": "#00a2f3",
        "fill": "#00a2f3",
        "fillOpacity": 0.3
      }
    },
    {
      "data": {
        "values": [{"lng1": -20, "lat1": 70, "lng2": 40, "lat2": 0}]
      },
      "projection": {
        "type": "orthographic",
        "rotate": [30,-25,0]
      },
      "encoding": {
        "longitude": {"field": "lng1"},
        "latitude": {"field": "lat1"},
        "longitude2": {"field": "lng2"},
        "latitude2": {"field": "lat2"}
      },
      "mark": {
        "type": "rect",
        "stroke": "#f3a200",
        "fill": "#f3a200",
        "fillOpacity": 0.3
      }
    }
  ]
}

The blue area is a rectangle correctly georeferenced to the projected space (because it is defined as geoJSON). The orange rectangle is by corner points only. Clearly the orange rectangle is not correct at this scale and this projection.

You may argue that for larger scale maps over a smaller area, the difference between the geo and Cartesian space would mean the difference between the two is typically much less. However, in many ways this is even more problematic because a projected image might look correct, when in fact it is not. In the Vega example above, smallish differences might be obvious as both the image and vectors are representing the same boundaries. But for a point overlay, the errors will not be as apparent, but still very much present.

Mixing differently projected features in the same space is a really bad idea. I think any georeferenced image (and rectangle) should be projected across its area and not just at corners. For rectangles, this isn't too hard - one could internally convert the vertices to a geoJSON feature (much as I have done above) and then treat as any other geoJSON. For images, this is more complicated. Typically one samples at regular intervals in the projected (screen) space and calculates the inverse map projection transform to sample the image pixel location at the relevant point.

[kanitw]

@domoritz Should we move this to Vega since it doesn't seem like it's feasible in Vega yet?