Making the decay parameters(dv,du) learnable and separate du, dv for different layers?

[naveedunjum]

Is there a way to make the decay parameters learnable? I want to train a network that can also adjust the decay parameters for the individual neurons.

Also is it possible to use different du, dv settings for different layers?

[bamsumit]

@naveedunjum Yes and Yes.

1. To make the decay parameters learnable, you can set requires_grad = True in the neuron parameter.
2. You can use different neuron parameters for different layers and thus have different u_decay and v_decay.

[naveedunjum]

@bamsumit That's great. I thought only the threshold was learnable. And how do I check the learned parameters? I tried the net.blocks[0].neuron.voltage_decay and net.blocks[0].neuron.threshold before and after training with requires_grad=True, don't see much difference.