Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spatiotemporal visualization of active subglacial lake surfaces over ICESat-2 cycles #180

Merged
merged 6 commits into from
Oct 26, 2020

Conversation

weiji14
Copy link
Owner

@weiji14 weiji14 commented Oct 14, 2020

Going from point clouds to data cubes! Going to make some cool 3D perspective views of changing ice surface over time.

dsm_whillans_ix_cycles_3-8

Note: below is actually Lower Subglacial Lake Conway (the little one), not Subglacial Lake Conway.

dsm_lower_subglacial_lake_conway_cycle_7

TODO:

  • Spatiotemporal data cube of ice elevation surfaces per ICESat-2 cycle (69e5b63)
  • Refactor spatiotemporal_cube function (ec1eac1, 665995a)
  • Update atlxi_lake.ipynb with new subglacial lake analysis (a63fad2)
  • Make 3D plots/animations of changing subglacial lake surfaces over different ICESat-2 cycles

In a separate PR, automate the spatiotemporal analysis across several lakes in a more scripted for-loop manner.

It's easier to look at raster grid surfaces than point clouds, and visualize them in 3D too! Using the good ol' `blockmedian` and `surface` modules from PyGMT to interpolate ICESat-2 points onto a grid. This is done on a (3 month) cycle by cycle basis, and the resulting NetCDF files are stacked into a single NetCDF data cube using xarray. Will need to work on using a proper datetime64 time axis instead of just cycle number, and perhaps allow for kwargs into the `surface` instead of the current hardcoded parameters.
@weiji14 weiji14 added the feature 🚀 Brand new feature label Oct 14, 2020
@weiji14 weiji14 self-assigned this Oct 14, 2020
Newer gmt with xarray shading support, and a newer intake-xarray too!
Refactor to move the NetCDF files into the folder correctly, overwriting the filepath(s) if the file is already present. Also added a 'projection' parameter so that the projection information is stored in the NetCDF grid and viewable directly in QGIS without needing to set a projection first. Using the proj4 string of Antarctic Polar Stereographic EPSG:3031 from https://spatialreference.org/ref/epsg/3031/proj4/. Note to self, the grid_mapping coordinate should actually be an attribute, either handle that using xarray.open_rasterio, use rioxarray, or wait until the pangeo/xarray community standardizes on what to do with this.
Better handle elevation outliers that had resulted in sharp peaks and troughs in the output NetCDF grid. Doing so by telling `pygmt.surface` to limit the output solution to a lower and upper bound, determined as ±3 median absolute deviations from the median of the entire dataframe table. Originally I was clipping the grid in a post-processing step, but decided it's much better to impose the limits in the surface algorithm itself! The elevation statistics of the test case in Greenland also looks more sensible now than before.
Main changes are an elev_filter setting in `deepicedrain.vizplots.plot_crossovers`, and the addition of annotated track numbers in the crossover_area map plot using GMT's quoted line functionality. Lots of other little bits too, including a refresh of the subglacial_lake ID (key) and name (value) pairs, a quick improvement to the inefficient brute-force crossover for-loop, and organizing figures inside 'figures/{lakename}' instead of just the 'figures' directory. Still a big chunk of local code yet to be committed, need to refactor and streamline the processing to be able to loop through many subglacial lakes in a more hands off manner!
@weiji14 weiji14 marked this pull request as ready for review October 26, 2020 09:54
@ghost
Copy link

ghost commented Oct 26, 2020

Congratulations 🎉. DeepCode analyzed your code in 1.793 seconds and we found no issues. Enjoy a moment of no bugs ☀️.

👉 View analysis in DeepCode’s Dashboard | Configure the bot

Visualizing ice surface changes in 3D, because it's more intuitive to the eye! There's a bit of an art to 3D perspective plots, and PyGMT's `grdview` does a fine job at it, though there's many lines of code involved! The figure consists of two plots. On the left is a shaded ice surface elevation grid overlaid with the subglacial lake outline in yellow. On the right is an elevation difference/anomaly grid view (relative to ICESat-2 Cycle 3).

Still need to wait for `subplot` and `plot3d` to be wrapped in PyGMT to simplify the code and make things look more polished. Animated GIF is made using imagemagick's `convert` in bash as it was higher quality (and less lines of code) than Pillow. Also have a code cell showing an alternative HvPlot-based 2D map of the same time-series grid, complete with an interactive slider to cycle through each ICESat-2 cycle.
@weiji14 weiji14 force-pushed the spatiotemporal_cube branch from d2a932c to 38237d1 Compare October 26, 2020 09:58
@weiji14 weiji14 closed this in 3d1534e Oct 26, 2020
@weiji14 weiji14 merged commit 3d1534e into master Oct 26, 2020
@weiji14 weiji14 deleted the spatiotemporal_cube branch October 26, 2020 10:04
@sourcery-ai
Copy link

sourcery-ai bot commented Oct 26, 2020

Sourcery Code Quality Report

✅  Merging this PR will increase code quality in the affected files by 0.04%.

Quality metrics Before After Change
Complexity 2.70 ⭐ 2.75 ⭐ 0.05 👎
Method Length 172.85 😞 186.21 😞 13.36 👎
Working memory 11.55 😞 11.71 😞 0.16 👎
Quality 67.77% 🙂 67.81% 🙂 0.04% 👍
Other metrics Before After Change
Lines 1186 1540 354
Changed files Quality Before Quality After Quality Change
atlxi_lake.py 48.64% 😞 47.08% 😞 -1.56% 👎
deepicedrain/init.py 81.52% ⭐ 79.01% ⭐ -2.51% 👎
deepicedrain/spatiotemporal.py 75.18% ⭐ 71.17% 🙂 -4.01% 👎
deepicedrain/vizplots.py 64.49% 🙂 64.32% 🙂 -0.17% 👎
deepicedrain/tests/test_spatiotemporal_cube.py 81.71% ⭐

Here are some functions in these files that still need a tune-up:

File Function Complexity Length Working Memory Quality Recommendation
deepicedrain/spatiotemporal.py spatiotemporal_cube 6 ⭐ 265 ⛔ 21 ⛔ 35.77% 😞 Try splitting into smaller methods. Extract out complex expressions
deepicedrain/vizplots.py plot_crossovers 3 ⭐ 239 ⛔ 20 ⛔ 40.59% 😞 Try splitting into smaller methods. Extract out complex expressions
deepicedrain/vizplots.py plot_alongtrack 1 ⭐ 199 😞 26 ⛔ 42.72% 😞 Try splitting into smaller methods. Extract out complex expressions
deepicedrain/spatiotemporal.py point_in_polygon_gpu 4 ⭐ 171 😞 17 ⛔ 46.87% 😞 Try splitting into smaller methods. Extract out complex expressions
deepicedrain/vizplots.py IceSat2Explorer.view 2 ⭐ 224 ⛔ 13 😞 49.55% 😞 Try splitting into smaller methods. Extract out complex expressions

Legend and Explanation

The emojis denote the absolute quality of the code:

  • ⭐ excellent
  • 🙂 good
  • 😞 poor
  • ⛔ very poor

The 👍 and 👎 indicate whether the quality has improved or gotten worse with this pull request.


Please see our documentation here for details on how these metrics are calculated.

We are actively working on this report - lots more documentation and extra metrics to come!

Let us know what you think of it by mentioning @sourcery-ai in a comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature 🚀 Brand new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant