Non-Linearity for Convolution

Why non-linearity is applied only for Conv2d layer and not other layers when building a CNN model?

when we get a feature map corresponding to a filter we apply ReLU non linear activation and then apply maxpool or avg pool so we do not need ReLU here because the we already have applied the activation func before passing it to pooling layers.
In case of max pool it chooses max value present inside a kxk window which has values already passed through a ReLU function so the output of this is already a ReLU of something.
Same logic applies with avg pool since we take avg of all non -ve values(since we had applied ReLU) so the avg is also non -ve and applying ReLU to a non -ve value gives the same value.

1 Like