What is the difference between model.train() and model.eval() in RNN training function using python?
Can you please share the relevant code extract of train and eval that you are referring to?
If you refer this canonical example: https://pytorch.org/tutorials/intermediate/char_rnn_classification_tutorial.html
Here it is explained: (see last line in italics)
Evaluating the Results
To see how well the network performs on different categories, we will create a confusion matrix, indicating for every actual language (rows) which language the network guesses (columns). To calculate the confusion matrix a bunch of samples are run through the network with
evaluate() , which is the same as
train() minus the backprop.
I am not referring to training and evaluation process of the Sequential model.I am just referring to the special methods taht are called on the model we defined.Can find from below screenshots
If we look at this documentation, eval is equivalent to calling train with mode = False. Hope this helps.
[docs] def train(self, mode=True): r"""Sets the module in training mode. This has any effect only on certain modules. See documentations of particular modules for details of their behaviors in training/evaluation mode, if they are affected, e.g. :class:`Dropout`, :class:`BatchNorm`, etc. Args: mode (bool): whether to set training mode (``True``) or evaluation mode (``False``). Default: ``True``. Returns: Module: self """ self.training = mode for module in self.children(): module.train(mode) return self [docs] def eval(self): r"""Sets the module in evaluation mode. This has any effect only on certain modules. See documentations of particular modules for details of their behaviors in training/evaluation mode, if they are affected, e.g. :class:`Dropout`, :class:`BatchNorm`, etc. This is equivalent with :meth:`self.train(False) <torch.nn.Module.train>`. Returns: Module: self """ return self.train(False)
These are the two phases in pytorch.
model.train() is used to switch the model to training mode, whereas
model.eval() for evaluation mode.
Refer this for more clarity.
People use it when they need certain layers during training but don’t need it during evaluations
model.train() will enable layers like dropout during training while
model.eval() will disable these layers which are not needed during evaluation