Doubt in Feed Forward Network

Why in FFN, while updating the weight, the new weight is divided by m.
image
Previously we are not dividing
image

Hi @abhash1997,
Which notebook is it? can you please share the specific name of the notebook?

In 0318_FeedForwardNetwork_new.ipynb , the fit method of class FirstFFNetwork

dw1 is divided by m to normalize the value of dw1 over m training examples

Then why in previous SigmoidNeuron ,we are not dividing in notebook “0228_SigmoidNeuron.ipynb”…?
image

See its a choice, you can either use the sum of the gradients or average of the gradients to update weights. Both have their own consequences. It’s not that if you take sum, your model won’t converge.
Here’s a more elaborate answer: https://stats.stackexchange.com/questions/183840/sum-or-average-of-gradients-in-mini-batch-gradient-decent

2 Likes

sir never discussed this divide by m concept in class. will someone provide more insight why we are doing this divide thing

Hi Saurav,
It’s just a matter of choice to use it. Please refer to the shared article by databaaz.

1 Like

Hi Saurav,
It’s just a matter of choice to use it. Please refer to the shared article by databaaz.

1 Like