Should we not normalize accross rows?

Should this not be accross rows i.e normalization of all the activated value of each layer