Xavier Initialisation when using nn.Sequential in PyTorch

General Discussions / Queries Deep Learning
#1

How can we do xavier initialisation with nn.sequential block used for making architecture of neural network ?
I was trying to do like this :

    def __init__(self):
        super().__init__()
        
        torch.manual_seed(1)
        self.net = nn.Sequential(
            nn.Linear(2,15),
            nn.Sigmoid(),
            nn.Linear(15,1)
        )

        init.xavier_normal_(self.weights)
1 Like
#2

This answer should help:

Basically, if you want to initialize all layers of net with Xavier init, do:

for p in net.parameters():
    torch.nn.init.xavier_uniform_(p)