3.1.6. unit_scaling.DepthModuleList

class unit_scaling.DepthModuleList(modules: Iterable[Module])

A torch.nn.ModuleList that automatically configures the depth for sake of scaling. Note that this does not track depth changes caused by modifications after initial construction.

ModuleList can be indexed like a regular Python list, but modules it contains are properly registered, and will be visible by all Module methods.

Parameters:

modules (iterable?) – an iterable of modules to add

append(module: Module) → ModuleList

Append a given module to the end of the list.

Parameters:

module (nn.Module) – module to append

extend(modules: Iterable[Module]) → Self

Append modules from a Python iterable to the end of the list.

Parameters:

modules (iterable) – iterable of modules to append

insert(index: int, module: Module) → None

Insert a given module before a given index in the list.

Parameters:

• index (int) – index to insert.

• module (nn.Module) – module to insert