checkpoints didn't perform as what paper shown

[languandong]

Excellent work!
As for VQA-RAD dataset of the VQA task, I download the checkpoint you provided and run the test script.
The effect was shown below.

But paper's

[dongzizhu]

Hi,
I'm also reimplementing their experiments. Could you share where did you get their dataset? I did find the VQA-RAD dataset according to the original RAD paper, but the train-val-test split is not clear. BTW I'm getting a much worse performance than yours.

Thx!

[languandong]

Hi, I'm also reimplementing their experiments. Could you share where did you get their dataset? I did find the VQA-RAD dataset according to the original RAD paper, but the train-val-test split is not clear. BTW I'm getting a much worse performance than yours.

Thx!

I get the train-val-test from a previous work MVEF https://github.com/aioz-ai/MICCAI19-MedVQA.
It seems that all the work later was divided according to this

[dongzizhu]

I get the train-val-test from a previous work MVEF https://github.com/aioz-ai/MICCAI19-MedVQA. It seems that all the work later was divided according to this

Yes yes, I'm also using the dataset here. Thx! Also, I think your results 0.659-0.849-0.7738 are close enough to their reported results :). Good luck!

[VcRenOne]

Excellent work! As for VQA-RAD dataset of the VQA task, I download the checkpoint you provided and run the test script. The effect was shown below.

But paper's

Hi dalao, I'm reproducing this experiment on my own dataset, and it works, but without any val results or any test results. Would like to ask you how to reproduce the code, thank you very much!!

It gets stuck in this spot (rank loop) every time
Also, I can't find any saved ckpt file

[Victory0622]

我从之前的工作 MVEF https://github.com/aioz-ai/MICCAI19-MedVQA获得了 train-val-test 。看来后来的工作都是按照这个来划分的

是的，我一直在使用这里的数据集。谢谢！另外，我认为您的结果 0.659-0.849-0.7738 与他们报告的结果足够接近:)。祝你运好！

Hi, could you please share with me the split dataset including training, validation, and testing datasets? I have downloaded the previous work MVEF, but it only contains training and testing datasets. When I tried to divide it into training, validation, and testing datasets, I got worse results.

[Shmily17]

嗨，我也在重新实现他们的实验。你能分享一下你从哪里得到他们的数据集吗？我确实根据原始 RAD 论文找到了 VQA-RAD 数据集，但 train-val-test 拆分尚不清楚。顺便说一句，我的表现比你差得多。
感谢！

我从以前的工作 MVEF https://github.com/aioz-ai/MICCAI19-MedVQA 中获得了 train-val-test。看来后来所有的工作都是按照这个划分的

Hello, I also encountered the same problem, the final accuracy rate is only 26%, please share the training set, validation set and test set, thank you from the bottom of my heart! Have a great day

[dongzizhu]

Hi @Shmily17 @Victory0622

I think the reason you are getting a worse performance is because the label is messed up. I don't remember how I solved this, maybe I retrained the model or did some kind of label matching in the test process. But their code did work out. Good luck!