Question about frozen parameters.

[zsc000722]

Hi, congratulations to your excellent work! I was recurrenting your work with the following config:

optimizer : {
type: AdamW,
part: only_new,
kwargs: {
lr : 0.0005,
weight_decay : 0.05
}}
which means, only the new parts of params could be updated while training. But I tried to calculate the number of trainable parameters like the following picture:

As shown in the picture that all the parameters in the network, no matter newly inserted part or the part that should be frozen seems both trainable. I am wondering that am I doing something wrong? Or is there anything I don't know about frozen parameters? I'll be appreciate it if you can reply to my problems. Thanks a lot!

[zyh16143998882]

Sorry for the delayed response as I am very busy these days.

You can refer to the add_weight_decay function in the following path, path: https://github.com/zyh16143998882/ICCV23-IDPT/blob/main/tools/builder.py.

Only parameters that are added to the params list (including no_decay and decay) are updated during training, other parameters are not updated during training. So you only need to count all the parameters that are in the params list.