3.1.24.1. unit_scaling.parameter.Parameter

unit_scaling.parameter.Parameter(data: Tensor, mup_type: Literal['weight', 'bias', 'norm', 'output'], mup_scaling_depth: int | None = None) → Parameter

Construct a u-μP parameter object, an annotated torch.nn.Parameter.

The returned parameter also supports the ParameterData protocol:

p = uu.Parameter(torch.zeros(10), mup_type=”weight”) assert p.mup_type == “weight” assert p.mup_scaling_depth is None