1608.06993 - Densely Connected Convolutional Networks
♥ DL , CV , CNN, Recognition
- One of CVPR2017 best paper, a new CNN archetecture
- Advantages:
- Alleviate gradient vanishing
- strengthen feature propagation
- encourage feature reuse
- reduce number of parameters
- Can be trained as similar steps in ResNet
- Limitations: from DPN
- Width of the densely connected path linearly increases as the depth rises
- This may cause the number of parameters to grow quadratically compared with the residual networks if the implementation is not specifically optimized
- Archetecture:
- Difference bwtween ResNet and DenseNet:
- ResNet adds the input features to the output features through the residual path:
$x(l) = H_l(l-1) + x_{l-1}$ - DenseNet uses a densely connected path to concatenate the input features with the output features :
$x(l) = H_l([x_0,...,x_{l-1}])$ - This enables each micro-block to receive raw information from all previous micro-blocks
- ResNet adds the input features to the output features through the residual path:
- Preactivation, i.e BN->ReLU->1x1Conv->BN->ReLU->3x3Conv
- To make pooling easier(the dimensions may increase too fast), halve feature dimension using conv before pooling
- Growth rate k(the number of 3x3 kernels after each part):
- After each part, the dimension of next part will increase by k
- Larger k means more information will be accessed, while the computational complexity will be increased
- In this paper k = 32/48
- Difference bwtween ResNet and DenseNet:
- Implementation: