3.1.23.2. unit_scaling.optim.lr_scale_func_adam
- unit_scaling.optim.lr_scale_func_adam(param: ParameterData) float[source]
Calculate the LR scaling factor for
torch.optim.Adamandtorch.optim.AdamW.
Calculate the LR scaling factor for torch.optim.Adam
and torch.optim.AdamW.