FileNotFoundError: [Errno 2] No such file or directory: 'data/imagenet/classnames.txt'

[JiuqingDong]

Dear Authors,

Could you tell me how can I find this file 'data/imagenet/classnames.txt'?
when I run the command, there is a error:FileNotFoundError: [Errno 2] No such file or directory: 'data/imagenet/classnames.txt'

https://github.com/xmartlabs/caffeflow/blob/master/examples/imagenet/imagenet-classes.txt
Can I use this one?

[AtsuMiyai]

@JiuqingDong Hi, thanks for your interest in our LoCoOp!
I follow the dataset setup of https://github.com/KaiyangZhou/CoOp/blob/main/DATASETS.md
You can download imagenet-classes.txt via https://drive.google.com/file/d/1-61f_ol79pViBFDG_IDlUQSwoLcn2XXF/view

[JiuqingDong]

Thank you for your reply.

Yesterday, I used this file to evaluate the 16-shot pre-trained models you released. I got the average result as follows. Three results denote Seed 1, 2, and 3.

Then I used the file you gave me. I got the same results. However, these results are inconsistent with those shown in Table 1 of your paper. I'm not sure if I did something wrong.

I use the inference command 'CUDA_VISIBLE_DEVICES=0 bash scripts/locoop/eval.sh data imagenet vit_b16_ep50 1 output/imagenet/LoCoOp/vit_b16_ep50_16shots/nctx16_cscFalse_ctpend/seed1'

[AtsuMiyai]

Oh, it is strange. Could you give me the log file?

Seed1 result is:
MCM avg. FPR:0.3443744680851064, AUROC:0.9273575242446808, AUPR:0.9846419915753499
GL-MCM avg. FPR:0.2815491134751773, AUROC:0.9361394410159575, AUPR:0.9862216511729003

[JiuqingDong]

log.txt

Here is the log file for Seed 1. Thank you so much.

[AtsuMiyai]

What is the performance using CoOp's ckpt?
https://drive.google.com/drive/folders/1eV3uJxxQ0hvY3JltwuXds_Qhr77QLfRe?usp=sharing

[JiuqingDong]

log__.txt
This is the log file of CoOp.pth

The results I obtained seem to be inconsistent with those published in the paper. Did you modify the code?

[AtsuMiyai]

I think I do not change my code. (I'm going to check now, just to be sure.)
I don't think our paper score is wrong. For example, CoOp's OOD detection results on ImageNet are presented in another paper https://arxiv.org/pdf/2306.06048.pdf. And, the result (Fig.5) is very close to mine.

Could you check the Dataset setup, etc.? Besides, you can MCM repo to check your datasets are correct. https://github.com/deeplearning-wisc/MCM

[AtsuMiyai]

@JiuqingDong Sorry for the delay reply.
Could you check you really use https://drive.google.com/file/d/1-61f_ol79pViBFDG_IDlUQSwoLcn2XXF/view?
Because this file you give and mine are different, so it is strange to have the same results.

[JiuqingDong]

Dear Author,

I confirm that I used the file you gave me.

The OOD datasets are downloaded from the MCM repository.
The Texture dataset includes more than 5640 images, while the other three datasets include 10,000 images for each. Imagenet-1k include 1,281,167 images.
I will download and re-test it again. Thank you!

May I ask one more question that:
How to use your repository to implement CoOp? For example, comment out some modules or hyperparameters.

Sincerely,

[JiuqingDong]

Thank you very much. When I totally re-tested(download the code, install the env...), I got a result that is consistent with yours. Maybe I did something wrong by mistake before.

By the way, How to use your repository to implement CoOp? For example, comment out some modules or hyperparameters.

[AtsuMiyai]

I'm glad to hear it!
As for the implementation of CoOp, you can comment like # out self.lambda_value * loss_en in https://github.com/AtsuMiyai/LoCoOp/blob/master/trainers/locoop.py#L299 (or in https://github.com/AtsuMiyai/LoCoOp/blob/master/trainers/locoop.py#L317)

[JiuqingDong]

I truly appreciate your timely help.