torch.nn.Module

torch.nn.Module is the base class for all neural networks modules.

Some classes that inherit from torch.nn.Module include

torch.nn.Sequential
torch.nn.ModuleList
torch.nn.ModuleDict
torch.nn.Transformer
torch.nn.Embedding
torch.nn.LayerNorm
torch.nn.Conv1d (via torch.nn.modules.conv._ConvNd)
torch.nn.Dropout (via torch.nn.modules.dropout._DropoutNd)
torch.nn.Container
PreTrainedModel (transformers library)

Torch's documentation refers to torch.nn.Module as a container.

Relation to torch.nn.modules.module.Module

torch.nn.Module and torch.nn.modules.module.Module are the same class:

import torch
print(torch.nn.Module.__module__ + '.' + torch.nn.Module.__name__)
#
# torch.nn.modules.module.Module

Members

Members of a torch.nn.Module object include

`add_module()`	Add a child module to the module. See also `children()` and `apply()`
`_apply()`
`apply()`	Applies a function recursively to every submodule (as returned by `children()`).
`bfloat16()`
`buffers()`
`_call_impl()`
`call_super_init`	`bool` object
`children()`	See also `add_module()`
`cpu()`	Compare with `to()`
`cuda()`	Moves all model parameters and buffers to the GPU. Compare with `cpu()`
`double()`
`dump_patches`	`bool` object
`eval()`	Put the model in evaluation mode, equivalent to `self.train(False)`.
`extra_repr()`
`float()`
`forward()`	Defines the computation performed at every call. This method should be overridden by all subclasses that inherit from `torch.nn.Module`. See also `register_forward_hook()` and `register_forward_pre_hook()`
`_get_backward_hooks()`
`_get_backward_pre_hooks()`
`get_buffer()`
`get_extra_state()`
`_get_name()`
`get_parameter()`	Returns the parameter identified by the given target.
`get_submodule()`
`half()`
`ipu()`
`_load_from_state_dict()`
`load_state_dict()`
`_maybe_warn_non_full_backward_hook()`
`modules()`
`named_buffers()`
`named_children()`	Returns an iterator over the immediate children of the module. The iterator returns tuples with the child module's name and the child module itself. See also `named_modules()`
`_named_members()`
`named_modules()`	See also `named_children()`
`named_parameters()`
`parameters()`	Returns an iterator over the module's parameters (typically needed by an optimizer). See also `torch.nn.Parameter`.
`register_backward_hook()`	Deprecated in favor of `register_full_backward_hook()`
`register_buffer()`
`register_forward_hook()`	Registers a hook function which is called every time `forward()` has computed an output.
`register_forward_pre_hook()`	Registers a hook function which is called before `forward()` in invoked.
`register_full_backward_hook()`	Registers a hook function which is called every time the module's gradients are computed.
`register_full_backward_pre_hook()`
`register_load_state_dict_post_hook()`
`_register_load_state_dict_pre_hook()`
`register_module()`
`register_parameter()`
`_register_state_dict_hook()`
`register_state_dict_pre_hook()`
`_replicate_for_data_parallel()`
`requires_grad_()`
`_save_to_state_dict()`
`set_extra_state()`
`share_memory()`
`_slow_forward()`
`state_dict()`
`T_destination`	?
`to()`
`to_empty()`
`train()`	See also `eval()`
`training`	A boolean which corresponds to the module's mode (training or evaluation). This member is created (and initialized) in the module's `__init__` method.
`type()`
`_version`	`int` object
`xpu()`
`zero_grad()`

Creating neural networks

In order to create a neural network, a class needs to be created that inherits from torch.nn.Module.

The layers of the neural network are defined in the class's __init__ method (constructor).

The class then also needs a forward() method which specifies how the data «flows» through the network.

class RenesNeuralNetwork(torch.nn.Module):

   def __init__(self):

     #
     # Call to parent class's __init__() function so that assigning submodules (self.xx = xyzmodule)
     # are automatically registered (which is useful for their parameters to be converted in calls
     # to .to() etc.)
     #
       super(RenesNeuralNetwork, self).__init__()

     #
     # After calling the parent class' __init__() function, the
     # submodules can be added:
     #
       self.flatten = torch.nn.Flatten()

       self.layers  = torch.nn.Sequential(
           …
       )

   def forward(self, x):

       x = self.flatten(x)
       return self.layers(x)

See also this torch Hello World example.

Parameters and state_dict

A module's learnable parameters (i.e. weights and biases) can be accessed with the model's .parameters() method (which returns a generator object).

state_dict is a dictionary that maps each layer to its parameter tensor and are typically used to save or load models.