3.1.21.1. unit_scaling.core.functional

Core functionality for implementing unit_scaling.functional.

Functions

logarithmic_interpolation(alpha, lower, upper)	Interpolate between lower and upper with logarithmic spacing (constant ratio).
rms(x[, dims, keepdim, eps])	Compute the RMS \(\sqrt{\mathrm{mean}(x^2) + \epsilon}\) of a tensor.
scale_elementwise(f, output_scale, ...[, ...])	Transforms an element-wise function into a scaled version.
transformer_residual_scaling_rule([...])	Compute the residual tau ratios for the default transformer rule.