In the deep learning course, in the 10th module that is sequence models, we are practically shown how to implements RNNs, LSTMs and GRUs. So in the RNN part of the practical videos, we set up the full training setup which will effectively be used for any of the three. I am attaching a snippet of the code.
In the training setup, we’re looping through the number of batches which are 10 in this case. I am unable to understand what the following code in the loop means:
for i in range(n_batches):
loss_arr[i+1] = (loss_arr[i]*i + train(net, opt, criterion, batch_size))/(i + 1)
It’d be great if someone could clear as to why the loss is being multiplied (loss_arr[i] * i) and why we’re dividing and adding the following to it (train(net, opt, criterion, batch_size)). And also why are we dividing the whole by i + 1 at each step.
Thanks in advance.