[Bug] make ZeroRedundancyOptimizer support param-wise settings

[nijkah]

Some configurations such as mask-rcnn_swin-t-p4-w7_fpn_1x_coco.py requires paramwise_cfg to give different parameters to some modules.

However, torch.distributed.optim.ZeroRedundancyOptimizer only recently started supporting the feature for multiple param groups. (Link)
(It supports in torch.__version__ >= 1.12.0).

The key implementation is

• Save parameters as List[torch.Tensor] in ZeroRedundancyOptimizer._all_params
• Give original parameters to the inner optimizer(optimizer_class) as Optimizer.__init__(self, params, defaults)

We can copy-and-paste its __init__ logic to support param-wise settings with torch.__version__ < 1.12.0.
However, I think this kind of solution is quite messy and vulnerable to change in torch.distributed.optim.ZeroRedundancyOptimizer.

I kindly ask for any good ideas to handle this!