Should we not normalize accross rows?

[Gaurav_Kandel]

Should this not be accross rows i.e normalization of all the activated value of each layer