A project by Jeremy Diaz (https://github.com/jdiaz4302) and Kristin Robinson (https://github.com/CUKrob)
Artistic style transfer is a recent method for applying artistic styles to everyday pictures, such as making a photograph look like it was painted by Vincent van Gogh (Figure 1C). A foundational paper in this field and the approach which this project is based off is "A Neural Algorithm of Artistic Style" by Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge.
The goal of this project is to see if natural disasters (such as fires and floods) can be effectively used as artistic styles. That is, can we visually predict what a natural disaster may look like?
We gathered before and after images for 28 flooded locations. One image was used as a "style" image (representing a typical flood) (Figure 2) to apply to other location's before images. We generated output images which serve as compromises between before-image content and style-image style, and then compared these output images (via the same unweighted loss functions) to the true after images.
Artistic style transfer utilizes a pretrained convolutional neural network, therefore there is no "training"; parameters are not tuned, only hyperparameters (such as loss function tradeoff and neural style depth) are. We found the hyperparameter combinations which minimize the loss/error between output images and 20 post-fire images, then used this combination to evaluate performance on 7 test set images. The model-generated outputs of which are below.
Overall the results are pretty impressive! Notably, however, the model will not "add water". Our model primarily gives the scenes a muddy and eroded appearance, suggesting that we are more reasonably predicting the aftermath (rather than the presence) of a flood. Due to this, the predictions are less impression on locations which lack abundant plant life.
This project was competed using the PyTorch (http://pytorch.org/) deep learning framework and was inspired by the above-mentioned arxiv paper. Our neural style tranfer code was immensely helped by the following documentation: http://pytorch.org/tutorials/advanced/neural_style_tutorial.html