Use ResNet-12 pre-training on my dataset

[zhouyizhuo]

@Tsingularity @daviswer
Dear Authors
I would like to ask how does 'val_acc' change during 'pre-training'?I am using the 'ResNet-12' model you provided to pre-train on my own dataset. Its 'train_acc' is good, but 'val_acc' is particularly poor and has not improved with pre-training. Is this the gap between normal training and small sample validation?

The content of the train.bash file is as follows:

The log file is as follows:

The introduction of my personal dataset is as follows:
1. train_classes: 382 train_images: ~16w
2. val_classes: 191 val_images: ~10w
3. test_classes: 191 test_images: ~8w

What might be the reason?
I am still a novice in this field, and any reply from you will be helpful to me!

As a supplement: I also tried to use the pre-trained model directly for testing on my dataset, but the results were not very good either.(But it is normal on other datasets)

[daviswer]

Hi @zhouyizhuo - I'm not entirely sure how to read your dataset info (16 training classes and 382 total images?) but this appears to be a VERY small dataset, given the way training accuracy jumps and then immediately saturates, with val accuracy not following along. For reference, the fairly small CUB dataset is 200 classes and 12k images total. While few-shot models are able to generalize to new classes with very limited reference data, the pretraining data still needs to be fairly large. It could be that your dataset is simply too small for ResNet12. I'd also check that you're normalizing your image inputs the same way we are here, given that the pretrained models are performing at chance on your dataset but normally on others