Master tests (#149)

* fixed generate meta tests * fixed inference script tests * fixed metrics tests * fixed preprocessing (masks, tile, mp aux) * fixed evaluation metrics tests * lots of issues with inference. fixed 2d so far * fixed 2.5d inference, still working on 3d * 3 remaining errors in 3d inference * debugging 3d inference * debugging 3d inference * debugging 3d inference, fixed overlap shape * fixed 5d tiling bug * fixed 3d inference! * fixed stitch tests * fixed dataset tests * fixed dataset w mask tests * fixed inference dataset tests * fixed plot utils tests * added flatfield tests * fixed gen mask tests * working on tiling tests * fixed tile nonuni tests * fixed uniform tile tests * fixed aux utils tests * debugging image utils * updated pandas version to avoid attribute error in pandas * fixed image utils tests * fixed mask utils tests * fixed mp utils tests * debugging tile utils * fixed tile utils tests * newer mpl version * updated skimage, debugging flatfield * sort output of os.listdir * Inference features (progressbar, define prediction folder name, metrics in figures) (#147) * minor fix for preprocessing_script * changed the layout of sub panels in predicted figures * removed redundant slice margin adjustment * disable the check for modelcheckpoint monitor * Added flags to save predicted images in image directory or model directory * Added data normalization options to preprocessing * updated image_inference script with data normalization * bug fix * updated tests * updated tests and 3d preprocessing * added dataset otsu mask type * added script for pooling multiple datasets * created meta_utils; added multi-processing option to meta_generator; estimate dataset z-score parameters from foreground images only * added function to sample values at block centers * added blocks_meta.csv * bug fix * updated functions to sample pixels and compute zscore parameters * Fixed the bug caused by mixed numpy datatypes; make data normalization backward compatible * updated metrics_script to new normalization options added get_pp_config function to preprocess_utils * unzscore prediction before computing SSIM; added multi-threading to tiling * unzscore predition for 3D inference * bug fix * turned off normalization for reading 3D target images for computing metrics * change output dtype to float32 * bug fix * made metrics_script backward compatible * Rename "workers" to "num_workers" in the training config * fixed tests * fixed tests * added pool config file * made Maskgenerator backward compatible * Add large 2D inference * bug fix * computed metrics stats for single FOV * update inference_script.py * bug fix * generate mask meta for user supplied masks; add watershed * update config files * update notebook * update conda env yaml * edit config * edit notebook * Add README for the notebook * edit comment * update comment blocks * update comment blocks * update README.md * update notebook * update README.md * fix plots not displayed issue * update README.md * adding a shell script for setup * shell script for setup * update setup script * update README.md * update notebook * edited the image translation exercies for clarity, added TOC, and added jupyterlab to conda environment config. * update README * update the paths to data to avoid conflict with 04_image_translation repo * update instructions * move README to course repo, clean up the notebook * fix typos * update paths for backup tiles * bux fix * update plotting * add config for 2.5D model; add warning for too small min_fraction * fix margin issue with 2.5D inference * fix indexing issues in uniform tiling * bug fix * bug fix * bug fix * bug fix * bug fix * fix plotting bug * fix tiling z indexing bug * fix inference 2.5D model bug * fix inference 2.5D model bug, cleaned * inference: add progressbar, colorbar for figure-target, decrease margin in figure * fix figure metric assignment * fix inference input middle slice selection * plot multiple inputs in figures * add tqdm to requirements.txt file * inference refactoring * changed pd version, convert nan to none when reading meta * fixed generate meta tests * fixed inference script tests * fixed metrics tests * fixed preprocessing (masks, tile, mp aux) * fixed evaluation metrics tests * lots of issues with inference. fixed 2d so far * fixed 2.5d inference, still working on 3d * 3 remaining errors in 3d inference * debugging 3d inference * debugging 3d inference * debugging 3d inference, fixed overlap shape * fixed 5d tiling bug * fixed 3d inference! * fixed stitch tests * fixed dataset tests * fixed dataset w mask tests * fixed inference dataset tests * fixed plot utils tests * added flatfield tests * fixed gen mask tests * working on tiling tests * fixed tile nonuni tests * fixed uniform tile tests * fixed aux utils tests * debugging image utils * updated pandas version to avoid attribute error in pandas * fixed image utils tests * fixed mask utils tests * fixed mp utils tests * debugging tile utils * fixed tile utils tests * newer mpl version * updated skimage, debugging flatfield * sort output of os.listdir * changed pd version, convert nan to none when reading meta * making tests compatible with progress bar changes, still debugging 3d * removed requirement to run xy metrics, fixed tests for 3d inference * fixed plot utils test * sorted glob output Co-authored-by: Johanna Rahm <48733135+JohannaRahm@users.noreply.github.com>
mehta-lab · May 10, 2022 · 59d599a · 59d599a
1 parent b9d6045
commit 59d599a
Show file tree

Hide file tree

Showing 42 changed files with 913 additions and 543 deletions.
diff --git a/README.md b/README.md
@@ -1,4 +1,4 @@
-[![Build Status](https://github.com/czbiohub/microDL/workflows/build/badge.svg)]
+![Build Status](https://github.com/czbiohub/microDL/workflows/build/badge.svg)
 [![Code Coverage](https://codecov.io/gh/czbiohub/microDL/branch/master/graphs/badge.svg)](https://codecov.io/gh/czbiohub/microDL)
 
 # microDL

diff --git a/micro_dl/cli/inference_script.py b/micro_dl/cli/inference_script.py
@@ -91,7 +91,6 @@ def run_inference(config_fname,
         inference_inst.run_prediction()
 
 
-
 if __name__ == '__main__':
     args = parse_args()
     # Get GPU ID and memory fraction

diff --git a/micro_dl/cli/metrics_script.py b/micro_dl/cli/metrics_script.py
@@ -93,11 +93,13 @@ def compute_metrics(model_dir,
         (see evaluation_metrics)
     :param bool test_data: Uses test indices in split_samples.json,
     otherwise all indices
+    :param str name_parser: Type of name parser (default or parse_idx_from_name)
     """
     # Load config file
     config_name = os.path.join(model_dir, 'config.yml')
     with open(config_name, 'r') as f:
         config = yaml.safe_load(f)
+
     preprocess_config = preprocess_utils.get_preprocess_config(config['dataset']['data_dir'])
     # Load frames metadata and determine indices
     frames_meta = pd.read_csv(os.path.join(image_dir, 'frames_meta.csv'))
@@ -117,7 +119,6 @@ def compute_metrics(model_dir,
     else:
         test_ids = np.sort(np.unique(frames_meta[split_idx_name]))
 
-
     # Find other indices to iterate over than split index name
     # E.g. if split is position, we also need to iterate over time and slice
     test_meta = pd.read_csv(os.path.join(model_dir, 'test_metadata.csv'))

diff --git a/micro_dl/cli/preprocess_script.py b/micro_dl/cli/preprocess_script.py
@@ -172,6 +172,7 @@ def generate_masks(params_dict,
     mask_channel = mask_processor_inst.get_mask_channel()
     return mask_dir, mask_channel
 
+
 def generate_zscore_table(params_dict,
                           norm_dict,
                           mask_dir):
@@ -205,18 +206,21 @@ def tile_images(params_dict,
                 tile_dict,
                 resize_flag,
                 flat_field_dir,
+                tiles_exist=False,
                 ):
     """
     Tile images.
 
     :param dict params_dict: dict with keys: input_dir, output_dir, time_ids,
      channel_ids, pos_ids, slice_ids, int2strlen, uniform_struct, num_workers
     :param dict tile_dict: dict with tiling related keys: tile_size, step_size,
-     image_format, depths. optional: min_fraction, mask_channel, mask_dir,
+     image_format, depths, min_fraction. Optional: mask_channel, mask_dir,
      mask_depth, tile_3d
     :param bool resize_flag: indicator if resize related params in preprocess_config
      passed to pre_process()
     :param str/None flat_field_dir: dir with flat field correction images
+    :param bool tiles_exist: If tiling weights after other channels, make sure
+     previous tiles are not erased
     :return str tile_dir: dir with tiled images
     """
     # Check tile args
@@ -227,6 +231,10 @@ def tile_images(params_dict,
     hist_clip_limits = None
     if 'hist_clip_limits' in tile_dict:
         hist_clip_limits = tile_dict['hist_clip_limits']
+    # Set default minimum fraction to 0
+    min_fraction = 0.
+    if 'min_fraction' in tile_dict:
+        min_fraction = tile_dict['min_fraction']
     # setup tiling keyword arguments
     kwargs = {'input_dir': params_dict['input_dir'],
               'output_dir': params_dict['output_dir'],
@@ -241,11 +249,11 @@ def tile_images(params_dict,
               'hist_clip_limits': hist_clip_limits,
               'flat_field_dir': flat_field_dir,
               'num_workers': params_dict['num_workers'],
-              'int2str_len': params_dict['int2strlen'],
               'tile_3d': tile_3d,
               'int2str_len': params_dict['int2strlen'],
-              'min_fraction': tile_dict['min_fraction'],
+              'min_fraction': min_fraction,
               'normalize_im': params_dict['normalize_im'],
+              'tiles_exist': tiles_exist,
               }
 
     if params_dict['uniform_struct']:
@@ -352,18 +360,19 @@ def pre_process(preprocess_config):
                         normalize_channels,
                     )
 
-    req_params_dict = {'input_dir': input_dir,
-                   'output_dir': output_dir,
-                   'slice_ids': slice_ids,
-                   'time_ids': time_ids,
-                   'pos_ids': pos_ids,
-                   'channel_ids': channel_ids,
-                   'uniform_struct': uniform_struct,
-                   'int2strlen': int2str_len,
-                   'normalize_channels': normalize_channels,
-                   'num_workers': num_workers,
-                   'normalize_im': normalize_im,
-                   }
+    req_params_dict = {
+        'input_dir': input_dir,
+        'output_dir': output_dir,
+        'slice_ids': slice_ids,
+        'time_ids': time_ids,
+        'pos_ids': pos_ids,
+        'channel_ids': channel_ids,
+        'uniform_struct': uniform_struct,
+        'int2strlen': int2str_len,
+        'normalize_channels': normalize_channels,
+        'num_workers': num_workers,
+        'normalize_im': normalize_im,
+       }
 
     # -----------------Estimate flat field images--------------------
     flat_field_dir = None
@@ -432,7 +441,7 @@ def pre_process(preprocess_config):
                 flat_field_dir=flat_field_dir,
                 str_elem_radius=str_elem_radius,
                 mask_type=mask_type,
-                mask_channel=mask_channel,
+                mask_channel=None,
                 mask_ext=mask_ext,
             )
         elif 'mask_dir' in preprocess_config['masks']:
@@ -443,11 +452,12 @@ def pre_process(preprocess_config):
             mask_meta_fname = None
             if 'csv_name' in preprocess_config['masks']:
                 mask_meta_fname = preprocess_config['masks']['csv_name']
-            mask_meta = \
-            meta_utils.mask_meta_generator(mask_dir,
-                                             name_parser='parse_sms_name',
-                                             )
+            mask_meta = meta_utils.mask_meta_generator(
+                mask_dir,
+                name_parser='parse_sms_name',
+            )
             frames_meta = aux_utils.read_meta(req_params_dict['input_dir'])
+            # Automatically assign existing masks the next available channel number
             mask_meta['channel_idx'] += (frames_meta['channel_idx'].max() + 1)
             # use the first mask channel as the default mask for tiling
             mask_channel = int(mask_meta['channel_idx'].unique()[0])
@@ -531,6 +541,7 @@ def pre_process(preprocess_config):
                 tile_dict=weight_tile_config,
                 resize_flag=resize_flag,
                 flat_field_dir=None,
+                tiles_exist=True,
             )
         preprocess_config['tile']['tile_dir'] = tile_dir
 

diff --git a/micro_dl/deprecated/gen_mask_seg.py b/micro_dl/deprecated/gen_mask_seg.py
@@ -216,7 +216,6 @@ def tile_mask_stack(self,
                     cropped_image_data = tile_utils.crop_at_indices(
                         input_image=cur_mask,
                         crop_indices=crop_indices_dict[fname],
-                        isotropic=isotropic
                     )
                 else:
                     cropped_image_data = tile_utils.tile_image(

diff --git a/micro_dl/inference/evaluation_metrics.py b/micro_dl/inference/evaluation_metrics.py
@@ -89,22 +89,32 @@ def ssim_metric(target,
                 prediction,
                 mask=None,
                 win_size=21):
-    """SSIM of target and prediction
+    """
+    Structural similarity indiex (SSIM) of target and prediction.
+    Window size is not passed into function so make sure tiles
+    are never smaller than default win_size.
 
     :param np.array target: ground truth array
     :param np.array prediction: model prediction
+    :param np.array/None mask: Mask
     :param int win_size: window size for computing local SSIM
     :return float/list ssim and ssim_masked
     """
     if mask is None:
-        cur_ssim = ssim(target, prediction,
-                        win_size=win_size,
-                        data_range=target.max() - target.min())
+        cur_ssim = ssim(
+            target,
+            prediction,
+            win_size=win_size,
+            data_range=target.max() - target.min(),
+        )
         return cur_ssim
     else:
-        cur_ssim, cur_ssim_img = ssim(target, prediction,
-                                      data_range=target.max() - target.min(),
-                                      full=True)
+        cur_ssim, cur_ssim_img = ssim(
+            target,
+            prediction,
+            data_range=target.max() - target.min(),
+            full=True,
+        )
         cur_ssim_masked = np.mean(cur_ssim_img[mask])
         return [cur_ssim, cur_ssim_masked]
 
@@ -320,6 +330,8 @@ def estimate_xyz_metrics(self,
             metrics_row,
             ignore_index=True,
         )
+        print('metrics xyz')
+        print(self.metrics_xyz)
 
     def estimate_xy_metrics(self,
                             target,