BPTT implementation doubt

How do we implement BPTT from scratch using python for a simple RNN

Generally, we don’t implement BPTT (Back-Propagation Through Time) from scratch since it can be a little overwhelming to calculate all the gradients through all time-steps.

However, if you want to know how, check this:
Building RNN BPTT from scratch using numpy

Or if you just want to learn how to write an RNN from scratch without using PyTorch’s nn.RNN() and perform gradient descent without using optimizer.step(), you can check this:
NLP from scratch: Classifying names with Character-level RNN

1 Like

yaa thanks a lot…But could you explain what those deltas are in the below screenshot …Remaining things i was able to understand…


You mean this? ≜

It just means that 'LHS symbol/function' is defined by 'RHS expression'.