About Feed Forward Network

I have created a generic vectorized algorithm for a feed forward network, as one of the exercise in DL-105. However i am not able to get satisfied result while implementing, can you please guide me through this. I’m attaching my developed code. kindly help me out by verifying this code is it correct or have to do any changes… I’m getting the op but its with high loss


class FFN_Genclass:

def __init__(self, inputs, outputs, hidden_neurons=[1]):

   self.ni = inputs.shape\[1\]

   self.no = outputs.shape\[1\]

   self.hl = len(hidden\_neurons)

   self.hn = hidden\_neurons

   self.sizes = \[self.ni\] + self.hn + \[self.no\]

   self.W = {}

   self.B = {}

   for i in range(self.hl + 1):

       self.W\[i+1\] = np.random.randn(self.sizes\[i\], self.sizes\[i+1\])

       self.B\[i+1\] = np.random.randn(1, self.sizes\[i+1\])

def sigmoid(self, X):

   return 1.0/(1.0 + np.exp(-X))

def softmax(self, X):

   exps = np.exp(X)

   return exps / np.sum(exps, axis=1).reshape(-1,1)

def forward_pass(self,X):

   self.H = {}

   self.A = {}

   self.H\[0\] = X

   for i in range(self.hl):

       self.A\[i+1\] = np.matmul(self.H\[i\],self.W\[i+1\]) + self.B\[i+1\]

       self.H\[i+1\] = self.sigmoid(self.A\[i+1\])

   self.A\[self.hl+1\] = np.matmul(self.H\[self.hl\], self.W\[self.hl + 1\])

   self.H\[self.hl+1\] = self.softmax(self.A\[self.hl+1\])

   self.OP\_H = self.H\[self.hl+1\]

   return self.OP\_H

def grad_sigmoid(self, X):

   return X\*(1-X)  

def grad(self,X,Y):


   n = len(self.sizes) - 1

   self.DH = {}

   self.DA = {}

   self.DW = {}

   self.DB = {}

   self.DA\[n\] = self.OP\_H - Y

   for k in range(n,0,-1):

       self.DW\[k\] = np.matmul(self.H\[k-1\].T,self.DA\[k\])

       self.DB\[k\] = np.sum(self.DA\[k\], axis=0).reshape(1,-1)

       self.DH\[k-1\] = np.matmul(self.DA\[k\],self.W\[k\].T)

       self.DA\[k-1\] = np.multiply(self.DH\[k-1\], self.sigmoid(self.H\[k-1\]))

def fit(self,X,Y,epochs=1,learning_rate=1,display_loss=False):

   DW = {}

   DB = {}

   if display\_loss:

       loss = {}

   m = X.shape\[0\]

   for i in tqdm\_notebook(range(epochs), total=epochs, unit="epoch"):


       for j in range(self.hl + 1):

           self.W\[j+1\] -= (learning\_rate \* self.DW\[j+1\]/m)

           self.B\[j+1\] -= (learning\_rate \* self.DB\[j+1\]/m)

       if display\_loss:

           y\_pred = self.predict(X)

           loss\[i\] = log\_loss(np.argmax(Y, axis=1), y\_pred)

   if display\_loss:





def predict(self, X):

   Y\_pred = self.forward\_pass(X)

   return np.array(Y\_pred).squeeze()


Hi Rohit,
Can you also share a snapshot of results ?
Is it that your loss is not converging, or not changing at all?

yes, it is converging but not with satisfactory result. I’m attaching the op’s

if i change the learning rate its changing drastically!

Hi, Thanks for this.
I guess that’s due to the network. Can you try training with a network having more hidden layers, and try to fine tune it with more number of neurons?

Hi, Thank you.
I surely will try with a variety of network architecture.
so, is the model / algorithm i have built correct ? , no error in calculation of weights, FP or in BP right !

1 Like