the accuracy is too low in inat18

[fantastice]

When I run inat18, I found that the following error is reported, so I commented out “self.class_map = class_map”, which is located at line 392 of dataset, and after running 200 rounds, I found that the accuracy is too low.The gpu I'm using is 4090t and the batchsize is 64

[rangwani-harsh]

Hi @fantastice,

Thanks for your effort, for the large datasets like iNat-18 we used a large effective batch size of 2048 across GPUs, to simulate that you can use accumulate gradient for large number of steps (8) and batch size (256) if possible on your gpu.

Further we would like to add that we used a cosine learning rate, hence 200 rounds might not give same performance as 1000 rounds we had in the paper.

Please let us know if you had run any more experiments.

Thanks

[fantastice]

Thank you very much for your answer, as I only have one gpu I had to set the batch_size to 64 to get the program to run. I will go ahead and try to run through 1000 rounds and if I have any good news I'll be sure to let you know as soon as I can. Thanks again and good luck with your work life!

[rangwani-harsh]

To increase the batch size with just 1 gpu:

You can increase the --accum_iter param in the following line:

DeiT-LT/sh/train_inat.sh

Line 20 in 5898778

--accum-iter 4 \

So the effective batch size would be batch_size x accum_iter. Maybe try with --accum_iter 16 (or 32), this will make the training slower but the batch size 1024 (or 2048).

Looking forward to your response.

[fantastice]

Thank you very much for your advice. I will run inat18 again according to the method you provided. If I have any good news, I will let you know immediately.Merry Christmas to you on Christmas Day tomorrow!