Skip to content

Commit

Permalink
Merge pull request #53 from BodenmillerGroup/imctools2
Browse files Browse the repository at this point in the history
Updates the example pipeline outline to CP4 and imctools v2
  • Loading branch information
votti authored Dec 15, 2020
2 parents 4515f28 + 656a1d0 commit c920c43
Show file tree
Hide file tree
Showing 8 changed files with 1,226 additions and 639 deletions.
56 changes: 36 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,42 @@
[![DOI](https://zenodo.org/badge/103582813.svg)](https://zenodo.org/badge/latestdoi/103582813)
# A flexible image segmentation pipeline for heterogeneous multiplexed tissue images based on pixel classification

I am currently remodelling the whole description/procedure and would be happy for any feedback!

#####
Please consider the current state of this repository as 'Beta'.
####

Notable changes to older version is change to CellProfiler 3 and that the ImcPluginsCP work now with any CP3 installation without special installation procedure.

The pipeline is based on CellProfiler (http://cellprofiler.org/, v1.3.5) for segmentation and Ilastik (http://ilastik.org/) for
for pixel classification. It is streamlined by using the specially developped imctools python package (https://github.com/BodenmillerGroup/imctools) as well as custom CellProfiler modules (https://github.com/BodenmillerGroup/ImcPluginsCP/tree/master-cp3, master-cp3 branch!).

This pipeline was developped in the Bodenmiller laboratory at the University of Zurich (http://www.bodenmillerlab.org/) to segment hundreds of highly multiplexed imaging mass cytometry (IMC) images. However we think it might also be usefull for other multiplexed imaging techniques.

The document to start can be found at 'scripts/imc_preprocessing.ipynb' (https://nbviewer.jupyter.org/github/BodenmillerGroup/ImcSegmentationPipeline/blob/development/scripts/imc_preprocessing.ipynb).
The pdf found in 'Documentation/201709_imctools_guide.pdf' is still conceptually valid, however the installation procedures described are outdated. Please follow the instructions in the imc_preprocessing.ipynb document!

This pipeline was presented at the 2019 Imaging Mass Cytometry User Group Meeting. [The slides can be downloaded here](https://drive.google.com/file/d/1ajPzlJ2CUj6sFYSOq0HR2dOJehHIlCJt/view). The slides briefly explain why we chose this approach to image segmentation and provide help to run the pipeline.

We freely share this pipeline in the hope that it will be usefull for others to perform high quality image segmentation and serve as a basis to develop more complicated open source IMC image processin17g workflows. In return we would like you to be considerate and give us and others feedback if you find a bug or issue and raise a GitHub Issue on the affected projects or on this page.

## Introduction
The pipeline is based on CellProfiler (http://cellprofiler.org/, tested v4.0.6) for segmentation and Ilastik
(http://ilastik.org/, tested v1.3.5)
for pixel classification. It is streamlined by using the specially developed imctools python
package (https://github.com/BodenmillerGroup/imctools, >v2.1) as well as custom CellProfiler modules
(https://github.com/BodenmillerGroup/ImcPluginsCP, release v4.2.1).

This pipeline was developed in the Bodenmiller laboratory at the University of Zurich (http://www.bodenmillerlab.org/)
to segment hundreds of highly multiplexed imaging mass cytometry (IMC) images.
However we think that this concept also works well for other multiplexed imaging modalities..

The document to start can be found at 'scripts/imc_preprocessing.ipynb'
(https://nbviewer.jupyter.org/github/BodenmillerGroup/ImcSegmentationPipeline/blob/development/scripts/imc_preprocessing.ipynb).
The pdf found in 'Documentation/201709_imctools_guide.pdf' is still conceptually valid, however the installation
procedures described are outdated. Please follow the instructions in the imc_preprocessing.ipynb document!

This pipeline was presented at the 2019 Imaging Mass Cytometry User Group Meeting.
[The slides can be downloaded here](https://drive.google.com/file/d/1ajPzlJ2CUj6sFYSOq0HR2dOJehHIlCJt/view).
The slides briefly explain why we chose this approach to image segmentation and provide help to run the pipeline.

We freely share this pipeline in the hope that it will be useful for others to perform high quality image segmentation
and serve as a basis to develop more complicated open source IMC image processin17g workflows. In return we would like
you to be considerate and give us and others feedback if you find a bug or issue and raise a GitHub Issue
on the affected projects or on this page.

## Changelog
- v1.0:
- Change to imctools v2:
- Changes the structure of the folder to the new format, changing the
naming of the .ome.tiff files

- Change to Cellprofiler v4:
- Requires the use of the ImcPluginsCP master branch

## Citation
d
If you use this workflow for your research, please cite us:
```
@misc{ImcSegmentationPipeline,
Expand Down
File renamed without changes.
151 changes: 151 additions & 0 deletions cp4_pipelines/1_prepare_ilastik.cppipe
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
CellProfiler Pipeline: http://www.cellprofiler.org
Version:5
DateRevision:406
GitHash:
ModuleCount:12
HasImagePlaneDetails:False

Images:[module_num:1|svn_version:'Unknown'|variable_revision_number:2|show_window:False|notes:['This module will prepare ilastik stacks for the ilastik cell classification pipeline.']|batch_state:array([], dtype=uint8)|enabled:True|wants_pause:False]
:
Filter images?:Images only
Select the rule criteria:and (extension does isimage) (directory doesnot containregexp "[\\\\/]\\.")

Metadata:[module_num:2|svn_version:'Unknown'|variable_revision_number:6|show_window:False|notes:['No metadata.']|batch_state:array([], dtype=uint8)|enabled:True|wants_pause:False]
Extract metadata?:Yes
Metadata data type:Text
Metadata types:{}
Extraction method count:1
Metadata extraction method:Extract from file/folder names
Metadata source:File name
Regular expression to extract from file name:^(?P<filename>.*)
Regular expression to extract from folder name:(?P<Date>[0-9]{4}_[0-9]{2}_[0-9]{2})$
Extract metadata from:All images
Select the filtering criteria:and (file does contain "")
Metadata file location:Elsewhere...|
Match file and image metadata:[]
Use case insensitive matching?:No
Metadata file name:
Does cached metadata exist?:No

NamesAndTypes:[module_num:3|svn_version:'Unknown'|variable_revision_number:8|show_window:False|notes:['Here only _ilastik.tiff files are selected.']|batch_state:array([], dtype=uint8)|enabled:True|wants_pause:False]
Assign a name to:Images matching rules
Select the image type:Grayscale image
Name to assign these images:DNA
Match metadata:[]
Image set matching method:Order
Set intensity range from:Image metadata
Assignments count:1
Single images count:0
Maximum intensity:255.0
Process as 3D?:No
Relative pixel spacing in X:1.0
Relative pixel spacing in Y:1.0
Relative pixel spacing in Z:1.0
Select the rule criteria:and (file does endwith "_ilastik.tiff")
Name to assign these images:Ilastik
Name to assign these objects:Cell
Select the image type:Color image
Set intensity range from:Image bit-depth
Maximum intensity:255.0

Groups:[module_num:4|svn_version:'Unknown'|variable_revision_number:2|show_window:False|notes:['No groups.']|batch_state:array([], dtype=uint8)|enabled:True|wants_pause:False]
Do you want to group your images?:No
grouping metadata count:1
Metadata category:None

SmoothMultichannel:[module_num:5|svn_version:'Unknown'|variable_revision_number:6|show_window:False|notes:['This filter will remove strong, single outlier pixels from the images, which sometimes occur in IMC images.']|batch_state:array([], dtype=uint8)|enabled:True|wants_pause:False]
Select the input image:Ilastik
Name the output image:IlastikFil
Select smoothing method:Remove single hot pixels
Calculate artifact diameter automatically?:Yes
Typical artifact diameter:16.0
Edge intensity difference:0.1
Clip intensities to 0 and 1?:Yes
Neighborhood filter size:3
Hot pixel threshold:50.0
Scale hot pixel threshold to image scale?:Yes

SummarizeStack:[module_num:6|svn_version:'Unknown'|variable_revision_number:1|show_window:False|notes:["This calculates an 'average' of all channels and multiplies it with 100.", 'Thus an average count of 0.1 (over all channels) will correspond to 10 in an uint16 image.', '', 'This allows to easily discriminate forground and background.', '']|batch_state:array([], dtype=uint8)|enabled:True|wants_pause:False]
Select the input image:IlastikFil
Conversion method:Python Function
Name the output image:ScaledMean
Input a Python function:lambda x, axis: np.mean(x, axis=axis)*100

StackImages:[module_num:7|svn_version:'Unknown'|variable_revision_number:2|show_window:False|notes:['Add the summary channel as first channel.']|batch_state:array([], dtype=uint8)|enabled:True|wants_pause:False]
Name the output image:IlastikExp
Hidden:2
Image name:ScaledMean
Image name:IlastikFil

Resize:[module_num:8|svn_version:'Unknown'|variable_revision_number:4|show_window:False|notes:['Scaling up the images 2x makes pixel classificaiton easier.']|batch_state:array([], dtype=uint8)|enabled:True|wants_pause:False]
Select the input image:IlastikExp
Name the output image:Ilastik2x
Resizing method:Resize by a fraction or multiple of the original size
Resizing factor:2
Width of the final image:100
Height of the final image:100
Interpolation method:Bilinear
Method to specify the dimensions:Manual
Select the image with the desired dimensions:None
Additional image count:0

CropImage:[module_num:9|svn_version:'Unknown'|variable_revision_number:4|show_window:False|notes:['This function crops random sections from the images, that are used for training.', '', 'The filename is used as a random seed.']|batch_state:array([], dtype=uint8)|enabled:True|wants_pause:False]
Select the input image:Ilastik2x
Name the output image:Ilastikcropped
W width:500
H height:500
Crop random or specified section?:Crop random section based on metadata.
X of upper left corner:0
Y of upper left corner:0
Optional Random Seed:\g<filename>
Additional image count:0

SaveImages:[module_num:10|svn_version:'Unknown'|variable_revision_number:15|show_window:False|notes:[]|batch_state:array([], dtype=uint8)|enabled:True|wants_pause:False]
Select the type of image to save:Image
Select the image to save:Ilastik2x
Select method for constructing file names:From image filename
Select image name for file prefix:Ilastik
Enter single file name:OrigBlue
Number of digits:4
Append a suffix to the image file name?:Yes
Text to append to the image name:_s2
Saved file format:h5
Output file location:Same folder as image|scaled
Image bit depth:16-bit integer
Overwrite existing files without warning?:Yes
When to save:Every cycle
Record the file and path information to the saved image?:No
Create subfolders in the output folder?:No
Base image folder:Elsewhere...|
How to save the series:T (Time)

SaveImages:[module_num:11|svn_version:'Unknown'|variable_revision_number:15|show_window:False|notes:['Saves the crop together with the coordinates.', 'This is important to track back were the random crops were exactly taken.']|batch_state:array([], dtype=uint8)|enabled:True|wants_pause:False]
Select the type of image to save:Image
Select the image to save:Ilastikcropped
Select method for constructing file names:From image filename
Select image name for file prefix:Ilastik
Enter single file name:OrigBlue
Number of digits:4
Append a suffix to the image file name?:Yes
Text to append to the image name:_x\g<Crop_Ilastikcropped_x>_y\g<Crop_Ilastikcropped_y>_w\g<Crop_Ilastikcropped_w>_h\g<Crop_Ilastikcropped_h>
Saved file format:h5
Output file location:Default Output Folder|crops
Image bit depth:16-bit integer
Overwrite existing files without warning?:Yes
When to save:Every cycle
Record the file and path information to the saved image?:Yes
Create subfolders in the output folder?:No
Base image folder:Elsewhere...|
How to save the series:T (Time)

CreateBatchFiles:[module_num:12|svn_version:'Unknown'|variable_revision_number:8|show_window:False|notes:[]|batch_state:array([], dtype=uint8)|enabled:False|wants_pause:False]
Store batch files in default output folder?:Yes
Output folder path:/mnt/f2160748-a937-44bd-aca8-3adb8a839612/Data/Analysis/cp4_segmentation_example/cpout
Are the cluster computers running Windows?:No
Hidden- in batch mode:No
Hidden- in distributed mode:No
Hidden- default input folder at time of save:/home/vitoz
Hidden- revision number:0
Hidden- from old matlab:No
Local root path:/home/vitoz
Cluster root path:/home/vitoz
Loading

0 comments on commit c920c43

Please sign in to comment.