Skip to content

Segmentation

Jump to bottom
Csaba Pinter edited this page Sep 22, 2017 · 3 revisions

Overview

Create Segmentation node that supports both volume and model representations of multiple segments (DICOM Segmentation Object) that can overlap, and support on-demand automatic conversion between

People: Andras, Csaba, Steve, Jc, Andriy

Plan

Segmentation plans archive

Potential requirements

  • grouping: colormap is just a flat list; ontologies
  • high-resolution images: 2GB-15GB from the scanner and usually downsampled/cropped to about 1GB
  • edit contours using spline curves and control points (see ITKSnap and others)
  • define open curves and surfaces (not just volumes)

Phase 2

  • CLI: GetRepresentation(string) for LabelmapVolumeNodes, always returns merged labelmap, but subclass Segmentation does conversion
  • RGBA colorwash volume for merged labelmap
  • Segmentation and segment selector widget: shows all segmentation objects and segments inside
  • Fractional labelmap support (conversion classes) #52
    • generated surface models would be smooth without additional surface post-processing (now we lose lots of information when we convert a mesh or straight line to a same-resolution mesh as the grayscale source volume), no (or much less) need for oversampling the source volume
    • store probabilistic segmentation results
    • paint and other segmentation tools could draw smooth contours
    • edit in oblique planes (maybe even non-linearly transformed): allowed by storing one segment per volume and fractional volume (therefore the volume can be interpolated quite smoothly in any direction)- Ontologies
    • Most of them proprietary, we need to define a subset of one we use locally (see FacetedVisualizer extension: filter search for ontology term)

Phase 1

  • Segmentation node

    • allow registration of representation classes, identified by string (actual implementation: no need for registering, just storing the representation name internally)
    • allow registration of converter classes for converting between representations (VTK filter)
    • automatic conversion using converters
    • implement vtkOrientedImageData to be able to store oriented images outside MRML
    • module widget (segment table, display settings, representations, copy/move/import/export)
    • storage (write and read to/from binary labelmap and closed surface master representations, perform the conversions using the stored parameters)

  • Display in slice view: as colorwash or contour (showing model intersections seems to be fast enough and provides nice smooth view)

    • Segmentation object would generate a merged volume on GetImageData()
      • displayable manager handles both scalar volumes (colored by colormap) or RGBA (color explicitly defined for each voxel) -> it is not the displayable manager, but the labelmap display code in slice layer logic that does it
    • Allow showing multiple segmentation objects at the same time: all segmentation objects would be rendered into a single labelmap layer; labelmap selector would only select one
      • Other modules (e.g., Editor) could add displayed labelmaps, too (later a special show/hide selector would be placed there instead of a simple node selector)

  • Display in 3D view:

    • Displayable manager would ask for polydata representation for each displayed segment from the segmentation node; one actor for each segment

  • Create new editor module: Segment Editor

    • Old description: Edit in editor:
      • Per-structure box: fill from segments
      • Allow mask for all the other contours (for "paint over" disabled drawing)
      • Temporarily clear segment image from segmentation object, move it to a new segmentation object, owned by the Editor (this allows to editing in a different representation; while keeping all segments in a segmentation to have the same representation)

  • Set name, color, tags (for grouping; tags can be put into hierarchies)

  • Show/hide individual contours and groups (differently in each viewer)

  • Transform handling: linear and deformable

  • Subject hierarchy: individual segment handling, conversions, extract representations, segment volume action

  • RT modules: DVH, DVHComparison, DoseComparison, ContourComparison, ContourMorphology

Phase 0

  • Create a new node for storing segmentation: vtkMRMLLabelMapVolumeNode. This node does not have any new functionality compared to its parent class, vtkMRMLScalarVolumeNode, but it has a different class. Instead of using LabelMap custom attribute to differentiate between label map/scalar volume, the class can be checked. This paves the way for adding custom vtkMRMLLabelMapVolumeNode-derived classes that are smarter (can store overlapping contours, etc). - Done (Andras) Topic branch: https://github.com/SlicerRt/Slicer/tree/segmentation-node (integrated into trunk)
  • Progress tracked by ticket #626 (done)

Final design

(What has not been implemented yet is in italics, the others are already working)

  • New vtkMRMLSegmentationNode node type

    • Subclass of also new vtkMRMLLabelmapVolumeNode

    • Contains a vtkSegmentation VTK object (see below)

    • Provides a merged labelmap representation containing all segments for labelmap display

      • Merged RGBA volume with all the per-segment volumes composited with their color

    • Manages storage (many segments may be stored more efficiently in a centralized way, e.g., in a single 4D volume instead of many separate 3D volumes)

    • Handles events

      • Master representation changed event for vtkSegmentation -> invalidate (remove) all non-master representations
      • Subject hierarchy UID added -> check if the segmentation has a DICOM reference to the instance UIDs of the subject hierarchy node with the new UIDs, and if yes and it's a volume, then set it as reference labelmap geometry
        • Handles the case when the referenced DICOM volume is loaded after the segmentation object. The scenario when the volume is loaded first needs to be handled in the DICOM loading of the segmentation

  • vtkSegmentation VTK object

    • Contains the segments (see below)
    • Stores master representation type (see below)
    • Allows setting and getting conversion parameters. A conversion parameter with a specific name may be present in multiple conversion rules, then they have the same value
    • Triggers automatic conversion by requesting presence of specific representation types

  • vtkSegment VTK object

    • Stores representations (which are vtkDataObject objects)
    • Properties
      • Name
      • DefaultColor
      • Tags: for easy grouping or indicating states (e.g. selected)
        • Hierarchy nodes only support node-level operation and they are also quite complicated and inefficient; tags could be organized in multiple hierarchies, sophisticated filtering rules could be created quite simply
    • For now they are identified by their index in the vtkSegment vector in vtkSegmentation

  • Conversion and logic

    • Implemented in pure VTK, as it is done similarly for forward/inverse transform computation and caching in vtkGeneralTransform.

    • Conversion graph is responsible for calculating the cheapest route for a specific conversion: nodes are the representations, edges are the conversion rules

    • vtkSegmentationConverter class: Each segmentation has an instance of this. It contains the conversion rule instances

    • vtkSegmentationConverterRule class

      • Base class for all converters. They need to be registered to the factory class that creates them for the converters.
      • Define a conversion algorithm for a conversion graph edge, specified by a "SorceRepresentation" and "TargetRepresentation"
      • Store the actual conversion parameters

    • Default conversion parameters: Reference image geometry (convenience functions for (de)serializing are available)

  • Representations

    • Master representation for storing the original data, all the others would be generated automatically; all segments in a segmentation object have the same master representation type!

      • The master representation is the one that is saved on disk
      • If it is modified, then the others are invalidated (if a non-master is changed, then it does not happen)

    • Default representations

      • Binary labelmap: labelmap having only 0 and 1 values (still stored in unsigned char volume)

        • Only contains an extent that covers the foreground, save a lot of memory for small segments
        • May be compressed and/or multi-resolution but then: the complete display pipeline has to be implemented to work directly on this special representationand additional compression/decompression would be needed for editing
        • VTK stencil is an example for compressed data storage and processing, but it's only binary data and only a few operations are implemented

      • Fractional labelmap: continuous (non-binary) per-segment volume

      • Planar contour: from RT structure sets or from other user-defined contours on slices - contours may be open

      • Closed surface: needed for nice 3D rendering

      • Allow adding more representations for additional storage as compressed, lower-resolution, etc. (e.g., for real-time update of 3D rendering of the segmentation results)

Clone this wiki locally