Pre Trained Models

Search Space

As part of the project, we conducted two large grid searches on the Azure Machine Learning platform under a variety of hyperparameters and model classes. The following parameter grid was used for both of the searches:

SearchSpace(
              model_name="vitb16r224",
              # According to: https://learn.microsoft.com/en-us/azure/machine-learning/reference-automl-images-hyperparameters
              layers_to_freeze=Choice([0, 5, 9]),
              # 0 for no weighted loss, 1 for weighted loss with sqrt.(class_weights), 2 for weighted loss with class_weights.
              weighted_loss=Choice([0,2]),
              learning_rate=Choice([0.0001, 0.01, 0.01]),
              number_of_epochs=30,
              optimizer="sgd",
        )
SearchSpace(
          model_name=Choice(["seresnext", "resnet50", "resnet152"]),
          # According to: https://learn.microsoft.com/en-us/azure/machine-learning/reference-automl-images-hyperparameters
          layers_to_freeze=Choice([0, 2]),
          # 0 for no weighted loss, 1 for weighted loss with sqrt.(class_weights), 2 for weighted loss with class_weights.
          weighted_loss=Choice([0,2]),
          learning_rate=Choice([0.0001, 0.001, 0.01]),
          optimizer= Choice(["sgd","adamw"]),
          number_of_epochs=30)

Results

The following are some of the best-performing models from both grid searches along with their results and the hyperparameters they were trained at. A file containing raw records of many performance metrics is available here

Run Name	Description	Hyperparameters	Accuracy
Bubbly_fox_qvpztcz	Best performing transformer model for grid search 1	hyperparameters: {"model": {"layers_to_freeze": 0, "learning_rate": 0.01, "model_name": "vitb16r224", "optimizer": "sgd", "weighted_loss": 0}}	Min: 0.67167, Max: 0.70648, Last: 0.70375
Bright_tongue_rqvcf6v9	High-performing relatively small resnet model (50) for grid search 1	hyperparameters: {"model": {"early_stopping": "True", "layers_to_freeze": 0, "learning_rate": 0.01, "model_name": "resnet50", "optimizer": "sgd", "weighted_loss": 0}}	Min: 0.44915, Max: 0.70307, Last: 0.69215
Salmon_stem_t4p5j6wr	Best performing residual network model from grid search 2	hyperparameters: {"model": {"layers_to_freeze": 2, "learning_rate": 0.0001, "model_name": "seresnext", "optimizer": "adamw", "weighted_loss": 0}}	Min: 0.66826, Max: 0.70375, Last: 0.68874
Keen_kitchen_r3wfc48z	Best overall and transformer-based from grid search 2	hyperparameters : {"model": {"layers_to_freeze": 0, "learning_rate": 0.01, "model_name": "vitb16r224", "optimizer": "sgd", "weighted_loss": 0}}	Min: 0.64642, Max: 0.70990, Last: 0.68328
Upbeat_quill_450nd2gd	Best weighted loss model. Best with approximately balanced confusion matrix	hyperparameters: {"model": {"layers_to_freeze": 0, "learning_rate": 0.001, "model_name": "resnet50", "number_of_epochs": 30, "optimizer": "adamw", "weighted_loss": 2}}	Min: 0.29693, Max: 0.62253, Last: 0.60614

Models

Archives containing the trained versions of the models described above can be accessed here.

Each archive has the following structure:

• Each archive in the model folder is named after its corresponding training run and contains three files.
• The model.ps file contains the model weights in Microsoft's format. The file scoring_file_v_1_0_0.py stores an interface to the trained model, this is primarily intended to be used when deploying the model via AzureML but could also be used by a human.
• The conda_env.yml file contains a specification of the python environment necessary to use the scoring_file_v_1_0_0.py file.

Deploying models via Azure is a somewhat involved process that is well documented by Microsoft.