ttnn.xielu

ttnn.xielu(input_tensor: ttnn.Tensor, *, alpha_p: float = 0.8, alpha_n: float = 0.8, memory_config: ttnn.MemoryConfig = None, output_tensor: ttnn.Tensor = None) → ttnn.Tensor

Applies xIELU (Expanded Integral of the Exponential Linear Unit) to input_tensor element-wise. This is a custom piecewise trainable activation function derived from “Deriving Activation Functions Using Integration” paper: https://arxiv.org/abs/2411.13010

With beta = 0.5 and eps = -1e-6:

x > 0 : alpha_p * x^2 + beta * x x <= 0: alpha_n * (expm1(minimum(x, eps))) - (alpha_n * x) + 0.5 * x

Parameters:

input_tensor (ttnn.Tensor) – the input tensor.

Keyword Arguments:

  • alpha_p (float, optional) – Alpha positive constant. Defaults to 0.8.

  • alpha_n (float, optional) – Alpha negative constant. Defaults to 0.8.

  • memory_config (ttnn.MemoryConfig, optional) – Memory configuration for the operation. Defaults to None.

  • output_tensor (ttnn.Tensor, optional) – preallocated output tensor. Defaults to None.

Returns:

ttnn.Tensor – the output tensor.

Note

Supported dtypes and layouts:

Dtypes

Layouts

BFLOAT16, FLOAT32

TILE, ROW_MAJOR