ttnn.mish

ttnn.mish(input_tensor: ttnn.Tensor, *, fast_and_approximate_mode: bool = False, memory_config: ttnn.MemoryConfig = None, output_tensor: ttnn.Tensor = None, sub_core_grids: ttnn.CoreRangeSet = None) → ttnn.Tensor

Performs the element-wise Mish activation function on the input_tensor. Mish is a smooth, non-monotonic self-regularizing activation function that allows small negative outputs to pass through.

\[\begin{split}\mathrm{{output\_tensor}}_i &= \mathrm{{input\_tensor}}_i \cdot \tanh(\mathrm{{softplus}}(\mathrm{{input\_tensor}}_i)) \\ &= \mathrm{{input\_tensor}}_i \cdot \tanh\left(\ln(1 + e^{{\mathrm{{input\_tensor}}_i}})\right)\end{split}\]
Parameters:

input_tensor (ttnn.Tensor)the input tensor.

Keyword Arguments:

  • fast_and_approximate_mode (bool, optional)Use the fast and approximate mode. Defaults to False.

  • memory_config (ttnn.MemoryConfig, optional)Memory configuration for the operation. Defaults to None.

  • output_tensor (ttnn.Tensor, optional)preallocated output tensor. Defaults to None.

  • sub_core_grids (ttnn.CoreRangeSet, optional)sub core grids for the operation. Defaults to None.

Returns:

ttnn.Tensorthe output tensor.

Note

Supported dtypes and layouts:

Dtypes

Layouts

BFLOAT16, BFLOAT8_B, FLOAT32

TILE, ROW_MAJOR

Example

# Create a tensor with random values
tensor = ttnn.rand([2, 2], dtype=ttnn.bfloat16, layout=ttnn.TILE_LAYOUT, device=device)

# Compute Mish activation function
output = ttnn.mish(tensor)
logger.info(f"Mish: {output}")