Adds EfficientNet B0 Model to training scripts

[TechnikEmpire]

Commit adds newly published EFNet B0 model to training scripts. No published model though. My experience is that this initial hub module has some issues, because I get worse classification results with this model than the Mobilenet model and that simply shouldn't be. Efnet is state of the art for accuracy. Anyway we'll want to keep an eye for efficientnet-lite b0 hub modules because they drastically reduce computation without sacrificing much accuracy.

I'm also attaching a newly trained mobilenet model zip. This is because I used another neural network to de-noise the training set and then re-trained. The "sexy" category had a few thousand files that were full blown pornography, and the pornography category had several thousand items that were not pornography at all.

I used a test set (that I'll also attach) to compare against. I notice in my work that "homework" image results resulted in a very high number of false positives with the old model. In fact the accuracy on the attached small validation set was ~88% with the old model. The accuracy with this newly attached model is ~94% on the same small validation set, a ~6% increase.

If nothing else, this demonstrates how drastically a noisy/poorly labelled training set can impact learning.

The newly attached models' overall validation accuracy is 93.10%.

See:

Relying on driver to perform ptx compilation. This message will be only logged once.
Epoch 1/5
4320/4320 [==============================] - 1875s 434ms/step - loss: 1.0400 - accuracy: 0.8958 - val_loss: 0.9698 - val_accuracy: 0.9217
Epoch 2/5
4320/4320 [==============================] - 1824s 422ms/step - loss: 0.9493 - accuracy: 0.9352 - val_loss: 0.9528 - val_accuracy: 0.9280
Epoch 3/5
4320/4320 [==============================] - 1840s 426ms/step - loss: 0.9076 - accuracy: 0.9529 - val_loss: 0.9562 - val_accuracy: 0.9243
Epoch 4/5
4320/4320 [==============================] - 1809s 419ms/step - loss: 0.8761 - accuracy: 0.9655 - val_loss: 0.9525 - val_accuracy: 0.9249
Epoch 5/5
4320/4320 [==============================] - 1810s 419ms/step - loss: 0.8533 - accuracy: 0.9737 - val_loss: 0.9460 - val_accuracy: 0.9277

Second round of fine-tuning with LR decreased
Epoch 1/2
4320/4320 [==============================] - 1854s 429ms/step - loss: 0.8360 - accuracy: 0.9801 - val_loss: 0.9393 - val_accuracy: 0.9322
Epoch 2/2
4320/4320 [==============================] - 1825s 423ms/step - loss: 0.8206 - accuracy: 0.9884 - val_loss: 0.9458 - val_accuracy: 0.9310

As you can see, I did a second round of fine-tuning, dropping the LR even further than before and it paid off by rounding us up a percentage point. Additional experimentation seems to indicate that this is the peak we can achieve with the cleaned training data and fine tuning for this network architecture.

[TechnikEmpire]

https://technikempire.com/mobilenet_v2_140_224.zip

"Homework" Google Images small validation set, in case you're interested in seeing what I meant with the old model vs new model and confirming my report.

https://technikempire.com/homework_val_small.zip

[GantMan]

Looks great!

[GantMan]

Do you have a public URL to the better training data?

[TechnikEmpire]

I'm a little nervous about distributing it, especially because it's just a huge mix bag of random images. I could provide you with the process I used, actually I could even give you the program I wrote to do the cleanup.

[GantMan]

Sounds good. Want ot sync up on a Slack like community.infinite.red ? or on Twitter?

[TechnikEmpire]

Slack would work.