diff --git a/templates/.DS_Store b/templates/.DS_Store
new file mode 100644
index 00000000..352d86d6
Binary files /dev/null and b/templates/.DS_Store differ
diff --git a/templates/4a_compensate_image_data.ipynb b/templates/4a_compensate_image_data.ipynb
index f238b5fd..6a6d9c01 100644
--- a/templates/4a_compensate_image_data.ipynb
+++ b/templates/4a_compensate_image_data.ipynb
@@ -5,9 +5,25 @@
"id": "ca8b60ef",
"metadata": {},
"source": [
+ "# Image Compensation\n",
"## This notebook is an example: create a copy before running it or you will get merge conflicts!"
]
},
+ {
+ "cell_type": "markdown",
+ "id": "237267d0",
+ "metadata": {},
+ "source": [
+ "Rosetta is the normalization process for your images produced by the MIBI. By normalizing the images you can reduce forms contamination that may show up.\n",
+ "\n",
+ "For example, we illustrate Pre and Post Rosetta processing on the CD11c channel.\n",
+ "\n",
+ "
\n",
+ " | \n",
+ " | \n",
+ "
\n"
+ ]
+ },
{
"cell_type": "code",
"execution_count": null,
@@ -25,65 +41,71 @@
"import pandas as pd\n",
"from mibi_bin_tools import bin_files\n",
"from toffy import rosetta\n",
- "\n",
"from ark.utils.io_utils import list_folders, list_files"
]
},
{
"cell_type": "markdown",
- "id": "d1db8364",
- "metadata": {},
- "source": [
- "### First, make a folder for evaluating rosetta normalization, and put the full path below"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "7955e002",
+ "id": "95eb4672",
"metadata": {},
- "outputs": [],
"source": [
- "base_dir = 'path/to/base/dir'"
+ "## 1. Setup"
]
},
{
"cell_type": "markdown",
- "id": "64aa3139",
+ "id": "d1db8364",
"metadata": {},
"source": [
- "### Next, copy over the .bin files for the ~10 FOVs will you use for testing. In addition to the .bin files, make sure to copy over the .JSON files with the same name into this folder. Place them in a folder named *example_bins*.\n",
+ "First, make a folder for evaluating rosetta normalization, and put the full path below\n",
+ "\n",
+ "Next, copy the Rosetta Matrix `commercial_rosetta_matrix_v1.csv` from the `/files` directory of toffy into `base_dir`. \n",
+ "\n",
+ "**UPDATE LATER TO BE COMPATIBLE WITH THE NEW AUTOMATED PANEL ADJUSTMENT SCRIPT**\n",
"\n",
- "### For example, fov-1-scan-1.bin, fov-1-scan-1.json, fov-23-scan-1.bin, fov-23-scan-1.json, etc"
+ "Select a name for the `output_folder`.\n",
+ "\n",
+ "Select the commercial Rosetta Matrix you wish to use."
]
},
{
"cell_type": "code",
"execution_count": null,
- "id": "952e3323",
+ "id": "7955e002",
"metadata": {},
"outputs": [],
"source": [
+ "base_dir = 'path/to/base/dir'\n",
+ "\n",
"# this folder should contain the bins and JSONs for the ~10 fovs\n",
- "test_bin_dir = os.path.join(base_dir, 'bin_files')"
+ "test_bin_dir = os.path.join(base_dir, 'example_bins')\n",
+ "\n",
+ "panel_file_name = 'example_panel_file.csv'\n",
+ "\n",
+ "# pick an informative name\n",
+ "output_folder = 'rosetta_output'\n",
+ "\n",
+ "rosetta_matrix = 'commercial_rosetta_matrix_v1.csv'"
]
},
{
"cell_type": "markdown",
- "id": "e0647556",
- "metadata": {},
- "source": [
- "### Next, copy the *commercial_rosetta_matrix_v1.csv* and the *example_panel_file.csv* files from the *files* directory of toffy into *base_dir*. Make sure to update the Target column of *example_panel_file.csv* with the details of your panel. For a complete description of panel formating, see the [panel description](https://github.com/angelolab/toffy#panel-format). Once you've updated the panel file, put the new name below. "
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "267afe31-8076-48d0-ac93-15de8009751b",
+ "id": "64aa3139",
"metadata": {},
- "outputs": [],
"source": [
- "panel_file_name = 'example_panel_file.csv'"
+ "Next, copy over the `.bin` files for the ~10 FOVs will you use for testing. In addition to the `.bin` files, make sure to copy over the `.json` files with the same name into this folder. Place them in a folder named `example_bins`.\n",
+ "\n",
+ "For example:\n",
+ "\n",
+ "```sh\n",
+ "path\\\\to\\\\base\\\\dir\n",
+ "└── example_bins\n",
+ " ├── fov-1-scan-1.bin\n",
+ " ├── fov-1-scan-1.json\n",
+ " ├── fov-23-scan-1.bin\n",
+ " ├── fov-23-scan-1.json\n",
+ " └── ...\n",
+ "```"
]
},
{
@@ -91,7 +113,7 @@
"id": "d0ab49c5",
"metadata": {},
"source": [
- "### We'll then use this panel file to extract the images from the bin files\n"
+ "We'll then use this panel file to extract the images from the bin files.\n"
]
},
{
@@ -130,7 +152,8 @@
"id": "7239e333",
"metadata": {},
"source": [
- "### We'll now process the images with rosetta to remove signal contamination"
+ "## 2. Rosetta - Remove Signal Contamination\n",
+ "We'll now process the images with rosetta to remove signal contamination. This will give us a new set of compensated images."
]
},
{
@@ -140,15 +163,12 @@
"metadata": {},
"outputs": [],
"source": [
- "# pick an informative name\n",
- "output_folder = 'rosetta_output'\n",
- "\n",
"# create sub-folder to hold images and files from this set of multipliers\n",
"output_folder_path = os.path.join(base_dir, output_folder)\n",
"os.makedirs(output_folder_path)\n",
"\n",
"# compensate the data\n",
- "rosetta_mat_path = os.path.join(base_dir, 'commercial_rosetta_matrix_v1.csv')\n",
+ "rosetta_mat_path = os.path.join(base_dir, rosetta_matrix)\n",
"rosetta.compensate_image_data(raw_data_dir=img_out_dir, comp_data_dir=output_folder_path,comp_mat_path=rosetta_mat_path, \n",
" raw_data_sub_folder='normalized', panel_info=panel, batch_size=1, norm_const=1)"
]
@@ -158,7 +178,7 @@
"id": "1e0c72cb",
"metadata": {},
"source": [
- "### Now that we've generated the compensated data, we'll generate stitched images to visualize what signal was removed"
+ "Now that we've generated the compensated data, we'll generate stitched images to visualize what signal was removed"
]
},
{
@@ -187,7 +207,15 @@
"id": "7b2274b4",
"metadata": {},
"source": [
- "### There will now be a folder named *stitched_with_Au* and *stitched_with_Noodle* within the *base_dir*. You can look through these stitched images to visualize what signal is being removed from the two most common source channels."
+ "There will now be a folder named `stitched_with_Au` and `stitched_with_Noodle` within the `base_dir`. You can look through these stitched images to visualize what signal is being removed from the two most common source channels."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d0307342",
+ "metadata": {},
+ "source": [
+ "## 3. Rosetta - Compensate the Whole Run"
]
},
{
@@ -195,7 +223,7 @@
"id": "bd4f51e0-8bcc-4b5b-b2d3-6bc00140e0ca",
"metadata": {},
"source": [
- "### Once you're satisfied that the Rosetta is working appropriately, you can use it to process your run. First select the run you want to process, and define the relevant top-level folders"
+ "Once you're satisfied that the Rosetta is working appropriately, you can use it to process your run. First select the run you want to process, and define the relevant top-level folders"
]
},
{
@@ -223,7 +251,7 @@
"id": "7f6161fb-3050-44e1-ab9a-7b9c59675d89",
"metadata": {},
"source": [
- "### Prior to running compensation, you'll need to extract your data if you haven't already"
+ "Prior to running compensation, you'll need to extract your data if you haven't already"
]
},
{
@@ -250,7 +278,7 @@
"id": "c4df6224-9839-43f7-9122-62bca0668fbc",
"metadata": {},
"source": [
- "### Then, you can compensate the data using rosetta"
+ "Then, you can compensate the data using rosetta"
]
},
{
@@ -272,7 +300,7 @@
],
"metadata": {
"kernelspec": {
- "display_name": "Python 3 (ipykernel)",
+ "display_name": "Python 3.8.13 ('toffy_env')",
"language": "python",
"name": "python3"
},
@@ -287,6 +315,11 @@
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.13"
+ },
+ "vscode": {
+ "interpreter": {
+ "hash": "b6fb10fd04ae96ae15bad5607f85b9bdcc66c9c776108a7102620b79ac3bec79"
+ }
}
},
"nbformat": 4,
diff --git a/templates/img/CD11c_post_rosetta_cropped.png b/templates/img/CD11c_post_rosetta_cropped.png
new file mode 100644
index 00000000..29358dcb
Binary files /dev/null and b/templates/img/CD11c_post_rosetta_cropped.png differ
diff --git a/templates/img/CD11c_pre_rosetta_cropped.png b/templates/img/CD11c_pre_rosetta_cropped.png
new file mode 100644
index 00000000..8d0a6812
Binary files /dev/null and b/templates/img/CD11c_pre_rosetta_cropped.png differ