Sigmoid neuron code

Hi Team,

In Sigmoid Neuron class fit function, it seems that we’re first initializing w and b values. Then for all input values we compute dw and db values and reinitialize w and b. Then at last we’re computing loss for all the inputs.

While in theory classes, it’s been explained that we first compute loss on that w and b values. Based upon that we reinitalise w and b values.

Two questions here:-

  1. In fit function on what basis we’re reinitializing the w and b values as we’re computing loss at the end? As per the algorithm we should first compute loss and on that basis we should reinitalize w and b values, right?
  2. Even if we calculate loss at the end for all the inputs, how we’re using that all reinitialized w and b values that we got at every iteration while computing loss?

Attached the screenshot of both fit function and learning algo. Please check and explain in detail.

Thanks

Shweta

Hi shweta!

First of all, as you can see we initialize our parameters To none. Then, as we begin our learning algo, we randomly initialize our parameters, Because we cannot compute the loss with a none value.

Randomly initializing just means that we are selecting a point on the loss curve, from where we will descend to the minimum of it, through a PATH, Defined by our LOSS function.

So, initializing the parameters is just random selection of beginning point. But the path traversal is via loss function optimization.

As for you second question, we are updating the parameters in every iteration of dataset through gradient descent algo, which is indeed the optimized path to take.

For this, you need to calculate loss for every epoch you do. And then plot.

I hope you get that.

Thank you.

Yeah I got that. Thanks a lot for the detailed explanation :slight_smile: