diff --git a/templates/.DS_Store b/templates/.DS_Store new file mode 100644 index 00000000..352d86d6 Binary files /dev/null and b/templates/.DS_Store differ diff --git a/templates/4a_compensate_image_data.ipynb b/templates/4a_compensate_image_data.ipynb index f238b5fd..6a6d9c01 100644 --- a/templates/4a_compensate_image_data.ipynb +++ b/templates/4a_compensate_image_data.ipynb @@ -5,9 +5,25 @@ "id": "ca8b60ef", "metadata": {}, "source": [ + "# Image Compensation\n", "## This notebook is an example: create a copy before running it or you will get merge conflicts!" ] }, + { + "cell_type": "markdown", + "id": "237267d0", + "metadata": {}, + "source": [ + "Rosetta is the normalization process for your images produced by the MIBI. By normalizing the images you can reduce forms contamination that may show up.\n", + "\n", + "For example, we illustrate Pre and Post Rosetta processing on the CD11c channel.\n", + "\n", + "\n", + " \n", + " \n", + "
\n" + ] + }, { "cell_type": "code", "execution_count": null, @@ -25,65 +41,71 @@ "import pandas as pd\n", "from mibi_bin_tools import bin_files\n", "from toffy import rosetta\n", - "\n", "from ark.utils.io_utils import list_folders, list_files" ] }, { "cell_type": "markdown", - "id": "d1db8364", - "metadata": {}, - "source": [ - "### First, make a folder for evaluating rosetta normalization, and put the full path below" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7955e002", + "id": "95eb4672", "metadata": {}, - "outputs": [], "source": [ - "base_dir = 'path/to/base/dir'" + "## 1. Setup" ] }, { "cell_type": "markdown", - "id": "64aa3139", + "id": "d1db8364", "metadata": {}, "source": [ - "### Next, copy over the .bin files for the ~10 FOVs will you use for testing. In addition to the .bin files, make sure to copy over the .JSON files with the same name into this folder. Place them in a folder named *example_bins*.\n", + "First, make a folder for evaluating rosetta normalization, and put the full path below\n", + "\n", + "Next, copy the Rosetta Matrix `commercial_rosetta_matrix_v1.csv` from the `/files` directory of toffy into `base_dir`. \n", + "\n", + "**UPDATE LATER TO BE COMPATIBLE WITH THE NEW AUTOMATED PANEL ADJUSTMENT SCRIPT**\n", "\n", - "### For example, fov-1-scan-1.bin, fov-1-scan-1.json, fov-23-scan-1.bin, fov-23-scan-1.json, etc" + "Select a name for the `output_folder`.\n", + "\n", + "Select the commercial Rosetta Matrix you wish to use." ] }, { "cell_type": "code", "execution_count": null, - "id": "952e3323", + "id": "7955e002", "metadata": {}, "outputs": [], "source": [ + "base_dir = 'path/to/base/dir'\n", + "\n", "# this folder should contain the bins and JSONs for the ~10 fovs\n", - "test_bin_dir = os.path.join(base_dir, 'bin_files')" + "test_bin_dir = os.path.join(base_dir, 'example_bins')\n", + "\n", + "panel_file_name = 'example_panel_file.csv'\n", + "\n", + "# pick an informative name\n", + "output_folder = 'rosetta_output'\n", + "\n", + "rosetta_matrix = 'commercial_rosetta_matrix_v1.csv'" ] }, { "cell_type": "markdown", - "id": "e0647556", - "metadata": {}, - "source": [ - "### Next, copy the *commercial_rosetta_matrix_v1.csv* and the *example_panel_file.csv* files from the *files* directory of toffy into *base_dir*. Make sure to update the Target column of *example_panel_file.csv* with the details of your panel. For a complete description of panel formating, see the [panel description](https://github.com/angelolab/toffy#panel-format). Once you've updated the panel file, put the new name below. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "267afe31-8076-48d0-ac93-15de8009751b", + "id": "64aa3139", "metadata": {}, - "outputs": [], "source": [ - "panel_file_name = 'example_panel_file.csv'" + "Next, copy over the `.bin` files for the ~10 FOVs will you use for testing. In addition to the `.bin` files, make sure to copy over the `.json` files with the same name into this folder. Place them in a folder named `example_bins`.\n", + "\n", + "For example:\n", + "\n", + "```sh\n", + "path\\\\to\\\\base\\\\dir\n", + "└── example_bins\n", + " ├── fov-1-scan-1.bin\n", + " ├── fov-1-scan-1.json\n", + " ├── fov-23-scan-1.bin\n", + " ├── fov-23-scan-1.json\n", + " └── ...\n", + "```" ] }, { @@ -91,7 +113,7 @@ "id": "d0ab49c5", "metadata": {}, "source": [ - "### We'll then use this panel file to extract the images from the bin files\n" + "We'll then use this panel file to extract the images from the bin files.\n" ] }, { @@ -130,7 +152,8 @@ "id": "7239e333", "metadata": {}, "source": [ - "### We'll now process the images with rosetta to remove signal contamination" + "## 2. Rosetta - Remove Signal Contamination\n", + "We'll now process the images with rosetta to remove signal contamination. This will give us a new set of compensated images." ] }, { @@ -140,15 +163,12 @@ "metadata": {}, "outputs": [], "source": [ - "# pick an informative name\n", - "output_folder = 'rosetta_output'\n", - "\n", "# create sub-folder to hold images and files from this set of multipliers\n", "output_folder_path = os.path.join(base_dir, output_folder)\n", "os.makedirs(output_folder_path)\n", "\n", "# compensate the data\n", - "rosetta_mat_path = os.path.join(base_dir, 'commercial_rosetta_matrix_v1.csv')\n", + "rosetta_mat_path = os.path.join(base_dir, rosetta_matrix)\n", "rosetta.compensate_image_data(raw_data_dir=img_out_dir, comp_data_dir=output_folder_path,comp_mat_path=rosetta_mat_path, \n", " raw_data_sub_folder='normalized', panel_info=panel, batch_size=1, norm_const=1)" ] @@ -158,7 +178,7 @@ "id": "1e0c72cb", "metadata": {}, "source": [ - "### Now that we've generated the compensated data, we'll generate stitched images to visualize what signal was removed" + "Now that we've generated the compensated data, we'll generate stitched images to visualize what signal was removed" ] }, { @@ -187,7 +207,15 @@ "id": "7b2274b4", "metadata": {}, "source": [ - "### There will now be a folder named *stitched_with_Au* and *stitched_with_Noodle* within the *base_dir*. You can look through these stitched images to visualize what signal is being removed from the two most common source channels." + "There will now be a folder named `stitched_with_Au` and `stitched_with_Noodle` within the `base_dir`. You can look through these stitched images to visualize what signal is being removed from the two most common source channels." + ] + }, + { + "cell_type": "markdown", + "id": "d0307342", + "metadata": {}, + "source": [ + "## 3. Rosetta - Compensate the Whole Run" ] }, { @@ -195,7 +223,7 @@ "id": "bd4f51e0-8bcc-4b5b-b2d3-6bc00140e0ca", "metadata": {}, "source": [ - "### Once you're satisfied that the Rosetta is working appropriately, you can use it to process your run. First select the run you want to process, and define the relevant top-level folders" + "Once you're satisfied that the Rosetta is working appropriately, you can use it to process your run. First select the run you want to process, and define the relevant top-level folders" ] }, { @@ -223,7 +251,7 @@ "id": "7f6161fb-3050-44e1-ab9a-7b9c59675d89", "metadata": {}, "source": [ - "### Prior to running compensation, you'll need to extract your data if you haven't already" + "Prior to running compensation, you'll need to extract your data if you haven't already" ] }, { @@ -250,7 +278,7 @@ "id": "c4df6224-9839-43f7-9122-62bca0668fbc", "metadata": {}, "source": [ - "### Then, you can compensate the data using rosetta" + "Then, you can compensate the data using rosetta" ] }, { @@ -272,7 +300,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "Python 3.8.13 ('toffy_env')", "language": "python", "name": "python3" }, @@ -287,6 +315,11 @@ "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.13" + }, + "vscode": { + "interpreter": { + "hash": "b6fb10fd04ae96ae15bad5607f85b9bdcc66c9c776108a7102620b79ac3bec79" + } } }, "nbformat": 4, diff --git a/templates/img/CD11c_post_rosetta_cropped.png b/templates/img/CD11c_post_rosetta_cropped.png new file mode 100644 index 00000000..29358dcb Binary files /dev/null and b/templates/img/CD11c_post_rosetta_cropped.png differ diff --git a/templates/img/CD11c_pre_rosetta_cropped.png b/templates/img/CD11c_pre_rosetta_cropped.png new file mode 100644 index 00000000..8d0a6812 Binary files /dev/null and b/templates/img/CD11c_pre_rosetta_cropped.png differ