Home

Process Flow

1. convert geotiff_to_tiff.py

2. a. plo_bbox.py

b. identify_bad_labels.py

3. a. train_test_split.py

b. split_geojson.py

4. get_data_stat.py

5. a. wv_util.py

b. process_wv_ms_train.py - to format in a way that tensorflow requires + data augmentation

6. aug_util.py (list the diff augs)

7. run_train.sh

8. a. run_eval.sh

b. export_model.sh

9. a. infer_detections.sh

b. cal_measure.sh (outputs csv file)

10. a. check_model.py - place ground truth side by side with predictions

b. create_detections.py

Data preparation

• labels.geojson: with bounding boxes location, uid, class type (damaged or non-damaged buildings), and tiff id (2048 x 2048 tiff names)

• a folder of tif chips of the size 2048 * 2048. Every chip contains at least one bbox.

1. Convert Geotiff to tiff

2. Manual inspect and remove bad bounding boxes

• Plot bounding boxes over big tiffs with bbox uids. Manual record the uids of bad labels
• Delete uids of bad labels from geojson, form a new geojson label file

3. Train-test split

• Split images into training, validation and test data
• Split geojson file to form training label geojson file and test label geojson file

4. Get statistics of training and testing data

• number of training big chips:
• number of test big chips:
• number of training bboxes for class 1:
• number of training bboxes for class 2:
• number of test bboxes for class 1:
• number of test bboxes for class 2:

5. Chip big tifs into smaller tifs

• Chip 2048 tif into smaller tifs
• discard black small tifs and their bboxes on the fly for train and val
• automatically removing cloud-covered bboxes on the fly for train and val
• remove small chips that do not contain any bboxes

6. augmentation

• Augment damaged buildings various combinations of augmentation techiniques

7. convert into TF-record

• convert training and validation images and bboxes to TF-record formats.
• create .pbtxt file as label map (1: damaged buildings, 2: non-damaged buildings)

8. Training

https://github.com/tensorflow/models/blob/1f562faacb0f0a083be9bcd7f9e72d94bb23e310/research/object_detection/g3doc/running_locally.md

9. Inference and scoring

• Create TF record for test data
• Generate detection results using tensorflow inference code. The results are written in a TF record file
• Compute metrics using TF record file with tensorflow code, see here: https://github.com/tensorflow/models/blob/1f562faacb0f0a083be9bcd7f9e72d94bb23e310/research/object_detection/g3doc/oid_inference_and_evaluation.md

10. Visualization

• Piece together detections on small tiffs to make 2048 x 2048 tiff files.
• Plot ground truth and predictions side by side in one image.