3.1.22.1. unit_scaling.functional.add

unit_scaling.functional.add(input: Tensor | int | float, other: Tensor | int | float, constraint: str | None = 'to_output_scale', alpha: int = 1, out: Tensor | None = None) → Tensor[source]

Applies a unit-scaled addition.

Adds other, scaled by alpha, to input.

\[\text{{out}}_i = \text{{input}}_i + \text{{alpha}} \times \text{{other}}_i\]

Supports broadcasting to a common shape, type promotion, and integer, float, and complex inputs.

Parameters:

  • input (Tensor) – the input tensor.

  • other (Tensor or Number) – the tensor or number to add to input.

  • constraint (Optional[str]?) – The name of the constraint function to be applied to the outputs & input gradients. In this case, the constraint name must be one of: [None, ‘gmean’, ‘hmean’, ‘amean’, ‘to_output_scale’, ‘to_left_grad_scale’, to_right_grad_scale] (see unit_scaling.constraints for details on these constraint functions). Defaults to gmean.

Examples

>>> a = torch.randn(4)
>>> a
tensor([ 0.0202,  1.0985,  1.3506, -0.6056])
>>> torch.add(a, 20)
tensor([ 20.0202,  21.0985,  21.3506,  19.3944])

>>> b = torch.randn(4)
>>> b
tensor([-0.9732, -0.3497,  0.6245,  0.4022])
>>> c = torch.randn(4, 1)
>>> c
tensor([[ 0.3743],
        [-1.7724],
        [-0.5811],
        [-0.8017]])
>>> torch.add(b, c, alpha=10)
tensor([[  2.7695,   3.3930,   4.3672,   4.1450],
        [-18.6971, -18.0736, -17.0994, -17.3216],
        [ -6.7845,  -6.1610,  -5.1868,  -5.4090],
        [ -8.9902,  -8.3667,  -7.3925,  -7.6147]])