This utility provides a PyTorch implementation of the GDN non-linearity based on the papers:
"Density Modeling of Images using a Generalized Normalization Transformation"
Johannes Ballé, Valero Laparra, Eero P. Simoncelli
https://arxiv.org/abs/1511.06281
"End-to-end Optimized Image Compression"
Johannes Ballé, Valero Laparra, Eero P. Simoncelli
https://arxiv.org/abs/1611.01704
The implementation is based on the available Tensorflow implementation un the contrib package (https://www.tensorflow.org/api_docs/python/tf/contrib/layers/gdn)
The GDN layer can be used as a normal non-linearity in PyTorch but must be instantiated with the number of channels at the application and the torch device where it will be used. The GDN layer supports 4-d inputs (batch of images) or 5-d inputs (batch of videos). The 5-d input is handled by unfolding the sequence dimension.
device = torch.device('cuda')
n_ch = 8
gdn = GDN(n_ch, device)
input = torch.randn(1, 8, 32, 32).to(device)
output = gdn(input)
In an example application, the normal GDN should be used with convolutions in an Encoder, and the inverse GDN should be used in the decoder with transposed convolutions.
Other parameters that can be used with the GDN are:
gdn = GDN(8, device
inverse = True,
beta_min=1-e6,
gamma_init=.1,
reparam_offset=2**-18
)