DeRF Normalization

Derf normalization layer.

Applies the error function (erf) as a smooth, saturating non-linearity with learnable affine parameters.

class src.model.norm.derf.Derf(*args: Any, **kwargs: Any)

Bases: Module

Derf normalization layer.

Applies the error function (erf) as a smooth, saturating non-linearity with learnable affine parameters. The formulation is:

y = gamma * erf(alpha * x + s) + beta

where alpha and s are scalar parameters controlling the slope and shift of the erf, and gamma, beta are per-dimension scale and bias.

Parameters:

dim – Number of features in the input (normalized dimension).

alpha

Learnable scalar slope parameter, initialized to 1.0.

s

Learnable scalar shift parameter, initialized to 0.0.

gamma

Learnable per-dimension scale of shape (dim,), initialized to 1.

beta

Learnable per-dimension bias of shape (dim,), initialized to 0.

forward(x: torch.Tensor) → torch.Tensor

Apply Derf normalization.

Parameters:

x – Input tensor of shape (..., dim).

Returns:

Tensor of same shape as x, transformed by the erf non-linearity with learnable affine parameters.