🗃️ Chunk ATL11 dataset along cycle_number when loading

Reading in all our ATL11 Zarr files in chunks of 6 along "cycle_number". Should be a bit more optimal (fingers crossed) when calculating along the time dimension, compared to having chunks with 2 "cycle_number"s before. Needed for what is coming next - calculating dhdt over Antarctica! But first, we'll need to package up some of the functions properly.
weiji14 · May 27, 2020 · d34455d · d34455d
1 parent 4c91dfe
commit d34455d
Show file tree

Hide file tree

Showing 2 changed files with 6 additions and 0 deletions.
diff --git a/atl11_play.ipynb b/atl11_play.ipynb
@@ -123,8 +123,11 @@
    },
    "outputs": [],
    "source": [
+    "# Load dataset from all Zarr stores\n",
+    "# Aligning chunks spatially along cycle_number (i.e. time)\n",
     "ds = xr.open_mfdataset(\n",
     "    paths=stores,\n",
+    "    chunks={\"cycle_number\": 6},\n",
     "    group=\"pt123\",\n",
     "    engine=\"zarr\",\n",
     "    combine=\"nested\",\n",

diff --git a/atl11_play.py b/atl11_play.py
@@ -63,8 +63,11 @@
 print(f"{len(stores)} reference ground track Zarr stores")
 
 # %%
+# Load dataset from all Zarr stores
+# Aligning chunks spatially along cycle_number (i.e. time)
 ds = xr.open_mfdataset(
     paths=stores,
+    chunks={"cycle_number": 6},
     group="pt123",
     engine="zarr",
     combine="nested",