Notebook 4a - Documentation Improvements (#177)

* nb4a - Draft 1 * nb4a - Draft 2 * updated Rosetta Matrix Directions * moved variables to the top * added pre and post rosetta images * moved testing presets into one cell * cropped images Co-authored-by: Noah F. Greenwald <noahfgreenwald@gmail.com>
angelolab · Aug 10, 2022 · 97b90d1 · 97b90d1
1 parent ef2df33
commit 97b90d1
Show file tree

Hide file tree

Showing 4 changed files with 75 additions and 42 deletions.
diff --git a/templates/.DS_Store b/templates/.DS_Store
diff --git a/templates/4a_compensate_image_data.ipynb b/templates/4a_compensate_image_data.ipynb
@@ -5,9 +5,25 @@
    "id": "ca8b60ef",
    "metadata": {},
    "source": [
+    "# Image Compensation\n",
     "## This notebook is an example: create a copy before running it or you will get merge conflicts!"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "237267d0",
+   "metadata": {},
+   "source": [
+    "Rosetta is the normalization process for your images produced by the MIBI. By normalizing the images you can reduce forms contamination that may show up.\n",
+    "\n",
+    "For example, we illustrate Pre and Post Rosetta processing on the CD11c channel.\n",
+    "\n",
+    "<table><tr>\n",
+    "    <td> <img src=\"./img/CD11c_pre_rosetta_cropped.png\" style=\"width:100%\"/> </td>\n",
+    "    <td> <img src=\"./img/CD11c_post_rosetta_cropped.png\" style=\"width:100%\"/> </td>\n",
+    "</tr></table>\n"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -25,73 +41,79 @@
     "import pandas as pd\n",
     "from mibi_bin_tools import bin_files\n",
     "from toffy import rosetta\n",
-    "\n",
     "from ark.utils.io_utils import list_folders, list_files"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "d1db8364",
-   "metadata": {},
-   "source": [
-    "### First, make a folder for evaluating rosetta normalization, and put the full path below"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "7955e002",
+   "id": "95eb4672",
    "metadata": {},
-   "outputs": [],
    "source": [
-    "base_dir = 'path/to/base/dir'"
+    "## 1. Setup"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "64aa3139",
+   "id": "d1db8364",
    "metadata": {},
    "source": [
-    "### Next, copy over the .bin files for the ~10 FOVs will you use for testing. In addition to the .bin files, make sure to copy over the .JSON files with the same name into this folder. Place them in a folder named *example_bins*.\n",
+    "First, make a folder for evaluating rosetta normalization, and put the full path below\n",
+    "\n",
+    "Next, copy the Rosetta Matrix `commercial_rosetta_matrix_v1.csv` from the `/files` directory of toffy into `base_dir`. \n",
+    "\n",
+    "**UPDATE LATER TO BE COMPATIBLE WITH THE NEW AUTOMATED PANEL ADJUSTMENT SCRIPT**\n",
     "\n",
-    "### For example, fov-1-scan-1.bin, fov-1-scan-1.json, fov-23-scan-1.bin, fov-23-scan-1.json, etc"
+    "Select a name for the `output_folder`.\n",
+    "\n",
+    "Select the commercial Rosetta Matrix you wish to use."
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "952e3323",
+   "id": "7955e002",
    "metadata": {},
    "outputs": [],
    "source": [
+    "base_dir = 'path/to/base/dir'\n",
+    "\n",
     "# this folder should contain the bins and JSONs for the ~10 fovs\n",
-    "test_bin_dir = os.path.join(base_dir, 'bin_files')"
+    "test_bin_dir = os.path.join(base_dir, 'example_bins')\n",
+    "\n",
+    "panel_file_name = 'example_panel_file.csv'\n",
+    "\n",
+    "# pick an informative name\n",
+    "output_folder = 'rosetta_output'\n",
+    "\n",
+    "rosetta_matrix = 'commercial_rosetta_matrix_v1.csv'"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "e0647556",
-   "metadata": {},
-   "source": [
-    "### Next, copy the *commercial_rosetta_matrix_v1.csv* and the *example_panel_file.csv* files from the *files* directory of toffy into *base_dir*. Make sure to update the Target column of *example_panel_file.csv* with the details of your panel. For a complete description of panel formating, see the [panel description](https://github.com/angelolab/toffy#panel-format). Once you've updated the panel file, put the new name below. "
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "267afe31-8076-48d0-ac93-15de8009751b",
+   "id": "64aa3139",
    "metadata": {},
-   "outputs": [],
    "source": [
-    "panel_file_name = 'example_panel_file.csv'"
+    "Next, copy over the `.bin` files for the ~10 FOVs will you use for testing. In addition to the `.bin` files, make sure to copy over the `.json` files with the same name into this folder. Place them in a folder named `example_bins`.\n",
+    "\n",
+    "For example:\n",
+    "\n",
+    "```sh\n",
+    "path\\\\to\\\\base\\\\dir\n",
+    "└── example_bins\n",
+    "    ├── fov-1-scan-1.bin\n",
+    "    ├── fov-1-scan-1.json\n",
+    "    ├── fov-23-scan-1.bin\n",
+    "    ├── fov-23-scan-1.json\n",
+    "    └── ...\n",
+    "```"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "d0ab49c5",
    "metadata": {},
    "source": [
-    "### We'll then use this panel file to extract the images from the bin files\n"
+    "We'll then use this panel file to extract the images from the bin files.\n"
    ]
   },
   {
@@ -130,7 +152,8 @@
    "id": "7239e333",
    "metadata": {},
    "source": [
-    "### We'll now process the images with rosetta to remove signal contamination"
+    "## 2. Rosetta - Remove Signal Contamination\n",
+    "We'll now process the images with rosetta to remove signal contamination. This will give us a new set of compensated images."
    ]
   },
   {
@@ -140,15 +163,12 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# pick an informative name\n",
-    "output_folder = 'rosetta_output'\n",
-    "\n",
     "# create sub-folder to hold images and files from this set of multipliers\n",
     "output_folder_path = os.path.join(base_dir, output_folder)\n",
     "os.makedirs(output_folder_path)\n",
     "\n",
     "# compensate the data\n",
-    "rosetta_mat_path = os.path.join(base_dir, 'commercial_rosetta_matrix_v1.csv')\n",
+    "rosetta_mat_path = os.path.join(base_dir, rosetta_matrix)\n",
     "rosetta.compensate_image_data(raw_data_dir=img_out_dir, comp_data_dir=output_folder_path,comp_mat_path=rosetta_mat_path, \n",
     "                              raw_data_sub_folder='normalized', panel_info=panel, batch_size=1, norm_const=1)"
    ]
@@ -158,7 +178,7 @@
    "id": "1e0c72cb",
    "metadata": {},
    "source": [
-    "### Now that we've generated the compensated data, we'll generate stitched images to visualize what signal was removed"
+    "Now that we've generated the compensated data, we'll generate stitched images to visualize what signal was removed"
    ]
   },
   {
@@ -187,15 +207,23 @@
    "id": "7b2274b4",
    "metadata": {},
    "source": [
-    "### There will now be a folder named *stitched_with_Au* and *stitched_with_Noodle* within the *base_dir*. You can look through these stitched images to visualize what signal is being removed from the two most common source channels."
+    "There will now be a folder named `stitched_with_Au` and `stitched_with_Noodle` within the `base_dir`. You can look through these stitched images to visualize what signal is being removed from the two most common source channels."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d0307342",
+   "metadata": {},
+   "source": [
+    "## 3. Rosetta - Compensate the Whole Run"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "bd4f51e0-8bcc-4b5b-b2d3-6bc00140e0ca",
    "metadata": {},
    "source": [
-    "### Once you're satisfied that the Rosetta is working appropriately, you can use it to process your run. First select the run you want to process, and define the relevant top-level folders"
+    "Once you're satisfied that the Rosetta is working appropriately, you can use it to process your run. First select the run you want to process, and define the relevant top-level folders"
    ]
   },
   {
@@ -223,7 +251,7 @@
    "id": "7f6161fb-3050-44e1-ab9a-7b9c59675d89",
    "metadata": {},
    "source": [
-    "### Prior to running compensation, you'll need to extract your data if you haven't already"
+    "Prior to running compensation, you'll need to extract your data if you haven't already"
    ]
   },
   {
@@ -250,7 +278,7 @@
    "id": "c4df6224-9839-43f7-9122-62bca0668fbc",
    "metadata": {},
    "source": [
-    "### Then, you can compensate the data using rosetta"
+    "Then, you can compensate the data using rosetta"
    ]
   },
   {
@@ -272,7 +300,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3.8.13 ('toffy_env')",
    "language": "python",
    "name": "python3"
   },
@@ -287,6 +315,11 @@
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
    "version": "3.8.13"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "b6fb10fd04ae96ae15bad5607f85b9bdcc66c9c776108a7102620b79ac3bec79"
+   }
   }
  },
  "nbformat": 4,

diff --git a/templates/img/CD11c_post_rosetta_cropped.png b/templates/img/CD11c_post_rosetta_cropped.png
diff --git a/templates/img/CD11c_pre_rosetta_cropped.png b/templates/img/CD11c_pre_rosetta_cropped.png