Earlier in March 2017, there was a lot of news coverage about an incident in the social media. Journalist Kurt Eichenwald received a tweet that stated "YOU DESERVE A SEIZURE FOR YOUR POSTS". The Journalist who suffered from epilepsy, got a seizure by looking at the image.
This incident has shown how social media could potentially be used as a terror weapon to inflict immediate physical pain and suffering to unsuspecting victims. Email, websites with user generated content are equally vulnerable to such attacks. In 2008 a hacker defaced the website of the Epilepsy Foundation and inserted flashing images which caused multiple health incidents to unsuspecting victims.
It is important for sites with user generated content to acknowledge this and take preventive action. Machine Learning can be applied to automatically detect images causing strobe behavior and block them.
TensorFlow is a an open source library for building machine learning models. In our case we will use an existing model and retrain it to detect seizure inducing images.This is called transfer learning. The popular Inception v3 model trained on ImageNet is used as the base model.
- TensorFlow is installed on docker. the image used is gcr.io/tensorflow/tensorflow:latest-devel
- Use Fatkun Batch Image downloader for chrome to search and download images of flashing strobe lighting.
- Follow the instructions in Google Code Labs to retrain your model.
- With 500 Iterations, Final test accuracy was 86.8%
- With 4000 Iterations, Final test accuracy improved to 91.5% and that is the model uploaded here as retrained_graph.pb
- Download test images to validate the trained model. Since I used google images to batch download training images, I decided to use bing image search for the test images. Couple of test images are uploaded under the
test_imagesfolder.
The animated gif needs to be converted to a jpg image for the model to work on it. This can be easily accomplished in the command line using ImageMagick.