Initialization of parameters in sigmoid neuron class

Hi Team

Professor used ‘self.w = np.random.randn(1, X.shape[1]) ‘ to initialize w in sigmoid neuron class defination while in perception class defination, he used ’ self.w = np.ones(X.shape[1])’ to initialize w.

Q1. what is the thought process behind this? from what I understand till now, we can initialize parameters in any way and the algorithm will take care of it.

Q2. any specific reason to initialize using normally distributed random numbers?

Thank you Team in advance.

Hi Saurav,

  1. The main idea behind this is considering the dimension of the layer.
  2. We don’t know about the minima, hence we try to randomize the initialization process, There are other great initialization methods for Neural Networks, that will be covered in upcoming lectures.

You may also want to refer this thread: Perceptron weight Initialization

1 Like