Error in the initialisation activation function for creating 3 layered NNetwork

for init_method in [‘random’, ‘zeros’, ‘xavier’, ‘he’]:

for activation_function in ['tanh']:

    print(init_method, activation_function)

    model = FFnetwork(init_method=init_method, activation_function=activation_function)

    model.fit(X_train, y_oh_train, epochs=100, eta=0.5, algo="NAG", display_loss=True)

    post_process(plot_scale=0.05)

    print('\n--\n')

The above is the code snippet and i have attached the error screenshot please help me with the error. As the same set of codes are working fine for all other initialization function except for the “tanh” function

KeyError: ‘W2’
The error screenshot, only tells, there is no key ‘W2’ in 'param'.
You can verify in 'fit' function if 'temp_params' actually contains 'W2' before calling 'grad'.

I have checked there W2 is present there but then also it is showing the same error.

Can you share link to your notebook?

1 Like

Yeah here is the link for the Notebook

https://colab.research.google.com/drive/15gRRvbe-ztO6q6zSufKGcowqaSfhE_K0?usp=sharing


Can you check share permissions (as reported in the error) and regenerate the link?

1 Like

Sorry for the delayed reply i was stuck with some work.
here is the fresh link please do check and tell me if the problem do exist.
https://colab.research.google.com/drive/15gRRvbe-ztO6q6zSufKGcowqaSfhE_K0?usp=sharing

When your code is failing while accessing params['W2'], this is the value stored in params (no W2):

params: {‘W1’: array([[1.76405235, 0.40015721],
[0.97873798, 2.2408932 ]]), ‘B1’: array([[ 1.86755799, -0.97727788]])}

ok so what should i do i am not getting it.

        elif algo == "NAG":
            temp_params = {}
            for i in range(1,self.num_layers+1):
                print("debug: i", i)
                self.update_params["v_w"+str(i)]=gamma*self.prev_update_params["v_w"+str(i)]
                self.update_params["v_b"+str(i)]=gamma*self.prev_update_params["v_b"+str(i)]
                temp_params["W"+str(i)]=self.params["W"+str(i)]-self.update_params["v_w"+str(i)]
                temp_params["B"+str(i)]=self.params["B"+str(i)]-self.update_params["v_b"+str(i)]
                self.grad(X,Y,temp_params)

In loop above, for i=1, only temp_params[‘W1’] will be initialised and then you call self.grad.
Note temp_params[‘W2’] will be initialised only when i=2. So when i=1 and self.grad is called, it throws error trying to access params[‘W2’].

Depending on what you are doing, consider moving call to self.grad function out of the loop, so that it is called after all required variables are initialised.