Operational Vehicle Maintenance using AR/VR Developed for the Smart India Hackathon 2019, The world's Largest Hackathon with nearly 784 Teams for across the country Participating.
This project won the First Prize at the Hackathon.This project was modified to make it suitable to work for the Operational vehicle maintainance. we use the base model of Matterport Masked RCNN for which I express my gratitude for making it open source https://github.com/matterport/Mask_RCNN
A Mask R-CNN model requires the user to annotate the images and identify the region of damage. The annotation tool I used is the VGG Image Annotator — v 1.0.6. You can use the html version available at http://www.robots.ox.ac.uk/~vgg/software/via/via-1.0.6.html . Using this tool you can create a polygon mask as shown below:
Once you have created all the annotations, you can download the annotation and save it in a json format. You can look at my images and annotations on my repository here.
After the Data Collection and annotation part, we move onto the training aspect of the Mask-RCNN. Without going into the details of how the training takes place, I have provided a simple python command in order to do the training. Please utilize a GPU if you don't want to wait till Eternity!!
there are three ways you can train the model
- From scratch
foo@bar :~$ python3 custom.py train --dataset=/path/to/datasetfolder
- Transfer Learning using COCO weights as the starting point( This is highly recommended for use!)
foo@bar :~$ python3 custom.py train --dataset=/path/to/datasetfolder --weights=coco
- Resuming a previous Training
foo@bar :~$ python3 custom.py train --dataset=/path/to/datasetfolder --weights=last
For this specific purpose, I have an entire Directory dedicated to provide many functionalities. These are :-
- To run a tornado server,
server3.py - to run the inference without the server using command line
evaluate.py - A utility module to import and use the mask R-CNN
predicfunc.py
Along with this , there are few Jupyter files provided in the source by Matterport in order to better understand how the entire inference works. Also note that mask R-CNN without the gpu take a very long time. For our model it took nearly 18 seconds to make a prediction!!
