G3W Suite to publish QGIS Cholera Project as a WebGisService

[Adrian49]

Would it be possible to use this open source solution to create an updated cholera map?
What would it take to move the idea forward?

[lindbrook]

Let's make sure we're on the same page (and address your earlier comments).

Dodson and Tobler

Their digitization, which essentially projects a nominal set of 2D Cartesian xy coordinates onto Snow's map, is the basis with some modification of the data and the plots used in and computed by the 'cholera' package.

While the data no longer seems to be hosted on NCGIA, they are preserved in the 'HistData' R package and probably elsewhere as well. Wherever you get the data, you should be able to easily import them into QGIS.

There are three key datasets/files: one that records the xy location of pumps, one that records the xy location of each "bar" or "dot" and one that records the xy endpoints of the straight line segments used to approximate the roads. For what it's worth, you can export these data as CSVs or TXTs from R.

Maps

I think I don't completely understand what you mean by a map. In my mind, the various plots, which you can see in the README or can replicate using 'cholera' and R, are maps.

Do you have something in mind that looks like Snow's original map or something that relies on Google Maps or Open Street Maps? Or do you mean something that is in pure GIS format (e.g., a QGIS project)?

QGIS

I'm not a GIS/QGIS person so take my ignorance into account.

Some of the functionality of 'cholera' seems readily available (e.g., Voronoi polygons).

Some seem to require a lot of work. Here I'm thinking of computing and constructing walking distances and neighborhoods. For the former, you need a way to construct/import the graph network of roads (as a shapefile?), then you can compute the shortest walking path. For the latter, I'm not sure. To construct the walking neighborhood polygons in R, I use brute force simulation. I place regularly spaced points across the map to find the extent of neighborhoods (a polygon). I don't know how easy it would be to do/script this in QGIS.

Is there a way to construct a user-friendly interface in QGIS? In R, I can use functions. For example, to compute and plot the walking path between case 1 and pump 10, use the following:

plot(walkingPathB(origin = 1, destination = 10, type = "case-pump"))

[Adrian49]

I've been thinking about some of the issues you raise, lindbrook. Waldo Tobler is a giant in the quantitative geography world. His work is much respected by many people in the GIS world such as Michael Goodchild, Paul Longley, David Maguire, and many more. So when you mentioned his name my ears pricked up. Additionally, John Snow's cholera map is a classic in the GIS world and hailed by many as predating modern GIS analysis and visualisation techniques. So, I come to to your work from a GIS perspective. I know about R but I struggle knowing how to consume it.
A map in my mind is a cartographic visualisation of geospatial data. However, I do understand when you say 'the various plots, which you can see in the README or can replicate using 'cholera' and R, are maps.' So, when I say map I have something in mind ' something in mind that looks like Snow's original map or something that relies on Google Maps or Open Street Maps.' Moreover, I ' mean something that is in pure GIS format e.g., a QGIS or ArcGIS Pro project.'
What excites me is the prospect of combining both perspectives to give users an interactive mapping experience via a WebGIS Service using G3W or something similar. Unfortunately, there is no user friendly QGIS GUI unless you build your own. That's just the way it is in the GIS world. I'm not a GIS expert, but I do know about georeferencing, shapefiles and cartography. However, when it comes to understanding 'plot(walkingPathB(origin = 1, destination = 10, type = "case-pump"))' my eyes glaze over. I don't use the Python window
in QGIS or ArcGIS Pro but I wished I could and it's not because of the lack of trying ... follow my drift.
Sincerely
Adrian

[lindbrook]

My "maps" can a bit too abstract. Integrating Snow's map or a real-world map has always been in the back of my mind. It's a nice-to-have feature that I may try to tackle in the future. One big constraint is the fact that I'm dealing with a historical map. The streets of Soho have changed a bit since the 1850s and I get the feeling that historical maps are not readily available in GIS.

I've been working to georeference the map. I'll eventually post the ground control points (GCPs) and document the specifics of what I did. Right now, those data are more proof-of-concept and I've been concentrating on translating functionality to use lat-long coordinates.

If you can, point me to an example (it need not be something you've done) of a really good interactive GIS project.

[Adrian49]

I work with historical maps too. I’m sending you a few screen shots and photos of a GIS project I’m doing for Mapping Brisbane History here in Brisbane, Australia. This will give you a bit of an idea of how I create layouts (maps).
The first is a screen shot of my QGIS project window. On the left you will notice the layers window with spatial layers I have created e.g. mountains, towns, roads, creeks as .shp files and two georeferenced historical maps and a base map, Open Street Map (OSM).
The layout (map) is actually a combination of all these spatial layers using various transparency levels to visualize the data. I put a lot of effort into the cartographic side of things having to do with design and purpose sometime to the extreme.
The problem is how are users able to interact with a GIS Project? A project hosted locally on a home computer is not a solution. However, if you imagine a GIS projects hosted on the www as a Web GIS Service using G3W or a web map using ArcGIS Pro AIP for Java then that’s different.
What the user sees is a version of the project window with links to the data and various other functionalities that can be added to enhance the user experience. For instance, the ability to add user data and export maps in various formats to name a few.
A good example of what I mean is TopoSvaldbard published by the Norwegian Polar Institute. This is a web map that uses the ESRI API. You can find it at https://toposvalbard.npolar.no/ There are other examples using open source which I can send if you are interested. This is where I’m heading with the Mapping Brisbane History Project!
One final observation if I may. What you’re doing isn’t all that different to what I’m doing. We both deal with uncertainty.
Cheers

[Adrian49]

I guess this wasn't helpful.

[lindbrook]

One reason that I'm adding lat-long is to make the project more GIS-friendly .

I'm not sure how fully GIS or Spatial Data Analysis I should or can take the project. I need to understand both better. But it's not a near term objective (perhaps when I get to v. 1.0). That said, feel free to make the case why GIS is the better way to analyze, visualize and understand these data.

[Adrian49]

I georeferenced a high resolution of Snow's Cholera Map ( 18MB) using 10 control points.
I ran the georeferencer to orient Snow's Map on the Soho District
I applied 40% global opacity to "see through" the maps
I took a screen shot_1 at a scale of 1:13,152
I took a screen shot_2 at a scale of 1:3,288
Time about 10 minutes
Next, I plan to digitize all the pumps in Snow's Map as a layer
See results ...

[Adrian49]

I have enclosed an essay I wrote some years back.
A GIS Project is just another of understanding, analyzing and visualizing the data.
Dr John Snow_6.pdf

[Adrian49]

I have mapped deaths in Broad Street In QGIS displayed as single symbols and labeled as counts for each location.
I can then display deaths as graduated symbols and perform a hot spot analysis.
QGIS can integrate with R using the Processing R Provider Plugin, so analysis of the data is possible using R scripts.
All this can be done within QGIS which combines the best of both worlds.

[Adrian49]

Deaths on Broad Street displayed as graduated symbols.
Building a database for the entire map may take some time.
But I think you get the drift of the final product.

[lindbrook]

I'd like to better understand your position (assuming you're representative of GIS users). It seems to me that you don't have objections per se but I get the sense that something doesn't feel right to you about how I'm presenting the data. If you could put that into words, I'd appreciate it.

To me, putting aside debates about tools (e.g., GIS, computer language), it almost seems to boil down to aesthetics: a tradeoff between maximalist context - using existing maps a foundation - versus information-forward minimalism - more abstract graphs.

For example, while it's probably non-trivial, I could probably add the analog Soho map as a background image to my graphs but in many cases the result would get too "busy" (to me at least). Consider what the graph below would look like overlayed on the Soho map.

For what it's worth my current working solution in QGIS:
Transformation type -- Thin Plate Spline
Resampling method -- Nearest Neighbour
Control points -- appox. 80

[Adrian49]

"Consider what the graph below would look like overlayed on the Soho map".
You mean this? Not too bust at all!

[Adrian49]

Is this an appropriate maximalist - minimalist trade off?
Looks perfectly to me. So, what does it mean to you, Peter?

[lindbrook]

The choice of map may be a good example. Maximalist would be a satellite image or a 19th century Ordnance Survey map with all the buildings and landmarks. Snow's map already abstracts away from that - not all the buildings are present. Minimalist might be just the streets of Soho illustrated as a network of lines.

The choice depends on your purpose and application.

For me, the emphasis is on "interactive" exploratory analysis and on a "good" estimate of point-to-point and walking distance. So while the image you create may be more than adequate for some purposes (I've also created similar plots in QGIS), the fact that not all the corresponding points "neatly" align may mean that there's room for improvement when estimating distance.

Anyway, like mentioned above, I may try to add the map as a background image in the future. Here I have two separate concerns (maybe you have more insight about this). First, many repositories have a hard limit on the size of software packages: this makes including a high resolution image a potential bottleneck. Second, I use different scales/magnification, which may affect the background image. I guess this is not an issue when you're making a single final map in QGIS (or you have some sort of map tile server - another potential distribution bottleneck). Two examples:

# walking path from John Snow's home to Golden Square
plot(walkingPathB("john snow", "golden square", latlong = TRUE))

# walking path from John Snow's home to the Broad Street pump
plot(walkingPathB("john snow", "broad street", latlong = TRUE))
addLandmarks(latlong = TRUE)