ttnn.abs (tt::ttnn::AbsOp)

Eltwise absolute.

Eltwise absolute operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.add (tt::ttnn::AddOp)

Eltwise add.

Eltwise add operation.

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.all_gather (tt::ttnn::AllGatherOp)

All gather op.

Tensor All Gather operation

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
all_gather_dim::mlir::IntegerAttr32-bit signed integer attribute
cluster_axis::mlir::IntegerAttr32-bit unsigned integer attribute
num_links::mlir::IntegerAttr32-bit unsigned integer attribute

Operands:

OperandDescription
inputranked tensor of any type values
deviceTTNN device

Results:

ResultDescription
resultranked tensor of any type values

ttnn.all_reduce (tt::ttnn::AllReduceOp)

All reduce op.

Tensor All Reduce operation

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
reduce_type::mlir::tt::ReduceTypeAttr
TT Reduce Type{{% markdown %}}Enum cases: * sum (`Sum`) * mean (`Mean`) * max (`Max`) * min (`Min`) * std (`Std`) * var (`Var`){{% /markdown %}}
cluster_axis::mlir::IntegerAttr32-bit unsigned integer attribute
num_links::mlir::IntegerAttr32-bit unsigned integer attribute

Operands:

OperandDescription
inputranked tensor of any type values
deviceTTNN device

Results:

ResultDescription
resultranked tensor of any type values

ttnn.alloc (tt::ttnn::AllocOp)

Alloc op.

Tensor Alloc operation

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
address::mlir::IntegerAttr64-bit signless integer attribute
size::mlir::IntegerAttr64-bit signless integer attribute
buffer_type::mlir::tt::ttnn::BufferTypeAttr
TTNN Buffer Type{{% markdown %}}Enum cases: * dram (`DRAM`) * l1 (`L1`) * system_memory (`SystemMemory`) * l1_small (`L1Small`) * trace (`Trace`){{% /markdown %}}

Results:

ResultDescription
resultranked tensor of any type values

ttnn.arange (tt::ttnn::ArangeOp)

Arange operation.

Tensor arange operation.

Produces a (1, 1, 1, N)-shaped tensor with values from start to end (exclusive) with a step size of step.

Examples: %0 = "ttnn.arange"() {start = 0 : i64, end = 5 : i64 step = 1 : i64} : () -> tensor<1x1x1x5xi64> // %0: [[[[0, 1, 2, 3, 4]]]]

%1 = "ttnn.arange"() {start = 0 : i64, end = 10 : i64, step = 2 : i64} : () -> tensor<1x1x1x5xf32> // %1: [[[[0.0, 2.0, 4.0, 6.0, 8.0]]]]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
start::mlir::IntegerAttr64-bit signless integer attribute
end::mlir::IntegerAttr64-bit signless integer attribute
step::mlir::IntegerAttr64-bit signless integer attribute
dtype::mlir::tt::DataTypeAttr
TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}}
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
deviceTTNN device

Results:

ResultDescription
resultranked tensor of any type values

ttnn.argmax (tt::ttnn::ArgMaxOp)

Argmax reduction op.

Determine the indices of the maximum values along a specified dimension of a tensor or over all elements in a tensor.

Parameters:

  • input: The input tensor.
  • dim: Specifies the dimension along which the argmax is applied.
  • use_multicore: Whether to use multiple cores or not.

IR usage: // Input tensor of shape (128, 28, 28, 64) %input = ... : tensor<128x28x28x64xbf16>

%empty = "ttnn.empty"(%0) <{dtype = #tt.supportedDataTypes, ....}> : -> tensor<128x28x28xi32> %4 = "ttnn.argmax"(%input, %empty) <{dim = 3 : i32, use_multicore = false}> : (tensor<128x28x28xbf16>, tensor<128x28x28xi32) -> tensor<128x28x28xi32>

Example: input: [[1, 5, 3], [2, 4, 6]]

// Computing along dim 0 output: [1, 0, 1]

// Computing along dim 1 output: [1, 2]

// Computing for entire tensor output: 5

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
dim::mlir::IntegerAttr32-bit signless integer attribute
use_multicore::mlir::BoolAttrbool attribute
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.atan (tt::ttnn::AtanOp)

Eltwise arctangent op.

Performs an elementwise arctangent (atan) operation on the input tensor. This operation computes the inverse tangent of each element, returning values in the range [-π/2, π/2]. Supports floating-point tensor types.

Example:

%input = tensor<4xf32> {1.0, 0.5, 0.0, -1.0}
%result = "ttir.atan"(%input) : (tensor<4xf32>) -> tensor<4xf32>

Given the input [1.0, 0.5, 0.0, -1.0], the result would be approximately: [0.785, 0.464, 0.0, -0.785] (values in radians).

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.bitwise_and (tt::ttnn::BitwiseAndOp)

Eltwise bitwise AND.

Performs element-wise bitwise AND of two tensors lhs and rhs and produces a result tensor.

Example: // %lhs: [[1, 2], [3, 4]] // %rhs: [[5, 6], [7, 8]] %result = "ttnn.bitwise_and"(%lhs, %rhs) : (tensor<2x2xi32>, tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[1, 2], [3, 0]]

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.bitwise_not (tt::ttnn::BitwiseNotOp)

Eltwise bitwise NOT.

Performs element-wise NOT of tensor operand and produces a result tensor.

Example: // Bitwise operation with with integer tensors // %operand: [[1, 2], [3, 4]] %result = "ttnn.bitwise_not"(%operand) : (tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[-2, -3], [-4, -5]]

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.bitwise_or (tt::ttnn::BitwiseOrOp)

Eltwise bitwise OR.

Performs element-wise bitwise OR of two tensors lhs and rhs and produces a result tensor.

Example: // %lhs: [[1, 2], [3, 4]] // %rhs: [[5, 6], [7, 8]] %result = "ttnn.bitwise_or"(%lhs, %rhs) : (tensor<2x2xi32>, tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[5, 6], [7, 12]]

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.bitwise_xor (tt::ttnn::BitwiseXorOp)

Eltwise bitwise XOR.

Performs element-wise bitwise XOR of two tensors lhs and rhs and produces a result tensor.

Example: // %lhs: [[1, 2], [3, 4]] // %rhs: [[5, 6], [7, 8]] %result = "ttnn.bitwise_xor"(%lhs, %rhs) : (tensor<2x2xi32>, tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[4, 4], [4, 12]]

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.cbrt (tt::ttnn::CbrtOp)

Eltwise cubic root.

Eltwise cubic root operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.ceil (tt::ttnn::CeilOp)

Eltwise ceil.

Eltwise ceil operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.clamp_scalar (tt::ttnn::ClampScalarOp)

Clamp op.

Clamp tensor values to a specified range.

Example: min: 2.000000+00 input: [[0, 1, 2, 3, 4, 5, 6, 7]] max: 5.000000+00

"ttnn.clamp_scalar"(%arg0) <{max = 2.000000e+00 : f32, min = 5.000000e+00 : f32}> -> %out = [[2, 2, 2, 3, 4, 5, 5, 5]]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
min::mlir::FloatAttr32-bit float attribute
max::mlir::FloatAttr32-bit float attribute

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultvariadic of ranked tensor of any type values

ttnn.clamp_tensor (tt::ttnn::ClampTensorOp)

Clamp op.

Clamp tensor values to a specified range using min/max as tensor.

Example: min: [[2, 2, 2, 3, 3, 3, 0, 0]] input: [[0, 1, 2, 3, 4, 5, 6, 7]] max: [[5, 5, 5, 9, 9, 9, 6, 6]]

"ttnn.clamp_tensor"(%input, %min, %max) %out: [[2, 2, 2, 3, 4, 5, 6, 6]]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values
minranked tensor of any type values
maxranked tensor of any type values

Results:

ResultDescription
resultvariadic of ranked tensor of any type values

ttnn.collective_permute (tt::ttnn::CollectivePermuteOp)

Collective permute op.

Collective permute op. This operation ingests a multi-device tensor spread across multi-devices and will shuffle the data according to source_target_pairs [['src', 'dest']].

Example: For a 1x2 mesh, the following will take the device shard living in device 0 and move it to device 1. The device shard living in device 1 will move to device 0. %source_target_pairs: [[0, 1], [1, 0]]

In the case of missing 'dest', the device shard living on that device will contain values of 0. For example, device shard living in device 0 will contain 0 values. %source_target_pairs: [[0, 1]]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
source_target_pairs::mlir::DenseIntElementsAttr64-bit signless integer elements attribute

Operands:

OperandDescription
inputranked tensor of any type values
deviceTTNN device

Results:

ResultDescription
resultranked tensor of any type values

ttnn.concat (tt::ttnn::ConcatOp)

Concat op.

Concat tensors along a given dimension.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
dim::mlir::IntegerAttr32-bit signed integer attribute
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.constant (tt::ttnn::ConstantOp)

Constant op.

Produces tensor filled with given constant value.

Examples: %0 = "ttnn.constant"() {value = dense<[[3, 4, 2], [1, 7, 8]]> : tensor<2x3xui16>} : () -> tensor<2x3xui16> // %0: [[3, 4, 2], [1, 7, 8]] %1 = "ttnn.constant"() {value = dense<[0.2, 1.3]> : tensor<2xf32>} : () -> tensor<2xf32> // %1: [0.2, 1.3]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
value::mlir::ElementsAttrconstant vector/tensor attribute

Results:

ResultDescription
resultranked tensor of any type values

ttnn.construct_tensor (tt::ttnn::ConstructTensorOp)

Construct empty tensor on host op.

Invoke tensor ctor to create an empty tensor on host.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
shape::mlir::tt::ttnn::ShapeAttr
TTNN Shape attribute{{% markdown %}} TTNN shape attribute {{% /markdown %}}
dtype::mlir::tt::DataTypeAttr
TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}}
layout::mlir::tt::ttnn::LayoutAttr
TTNN Layout{{% markdown %}}Enum cases: * row_major (`RowMajor`) * tile (`Tile`) * invalid (`Invalid`){{% /markdown %}}

Results:

ResultDescription
resultranked tensor of any type values

ttnn.conv2d (tt::ttnn::Conv2dOp)

Conv2d operation.

Applies a 2D convolution over an input image composed of several input planes.

Inputs:

  • input (AnyRankedTensor): expected in the following flattened format (1, 1, N * H_in * W_in, C) where:
    • N is the batch size
    • H_in is the height of the input planes
    • W_in is the width of the input planes
    • C is the number of channels
  • weight (AnyRankedTensor): expected in the following format (O, C/G, K_H, K_W).
  • bias (Optional): expected in the following format (1, 1, 1, O) where:
    • C is the number of input channels
    • O is the number of output channels
    • G is the number of groups
    • K_H is the height of the kernel
    • K_W is the width of the kernel

Attributes:

  • in_channels (i32): The number of input channels.
  • out_channels (i32): The number of output channels.
  • batch_size (i32): The batch size.
  • input_height (i32): The input height.
  • input_width (i32): The input width.
  • kernel_size (array<2xi32>): [K_H, K_W] where K_H is the kernel height and K_W is the kernel width.
  • stride (array<2xi32>): [sH, sW] where sH is stride for height and sW is stride for width.
  • padding (array<2xi32>): [pH, pW] where pH is padding for height (top/bottom) and pW is padding for width (left/right).
  • dilation (array<2xi32>): [dH, dW] where dH is dilation for height and dW is dilation for width.
  • groups (i32): Number of blocked connections from input channels to output channels. Input and output channels must both be divisible by groups.

Outputs:

  • result (AnyRankedTensor): returned in the following flattened format (1, 1, N * H_out * W_out, O) where:
    • H_out = (H_in + 2 * pH - dH * (K_H - 1) - 1) / sH + 1
    • W_out = (W_in + 2 * pW - dW * (K_W - 1) - 1) / sW + 1

Example: %input = ttir.empty() : () -> tensor<1x1x1024x64xbf16> %weight = ttir.empty() : () -> tensor<64x64x3x3xbf16> %bias = ttir.empty() : () -> tensor<1x1x1x64xbf16> %device = "ttnn.get_device"() <{mesh_shape = #ttnn<mesh_shape 1x1>}> : () -> !ttnn.device %0 = "ttnn.conv2d"(%input, %weight, %bias, %device) <{ in_channels = 64: i32, out_channels = 64: i32, batch_size = 1: i32, input_height = 32: i32, input_width = 32: i32, kernel_size = array<i32: 3, 3>, stride = array<i32: 1, 1>, padding = array<i32: 0, 0>, dilation = array<i32: 1, 1>, groups = 1: i32 }> : (tensor<1x1x1024x64xbf16>, tensor<64x64x3x3xbf16>, tensor<1x1x1x64xbf16>, !ttnn.device) -> tensor<1x1x900x64xbf16>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
in_channels::mlir::IntegerAttr32-bit signless integer attribute
out_channels::mlir::IntegerAttr32-bit signless integer attribute
batch_size::mlir::IntegerAttr32-bit signless integer attribute
input_height::mlir::IntegerAttr32-bit signless integer attribute
input_width::mlir::IntegerAttr32-bit signless integer attribute
kernel_size::mlir::DenseI32ArrayAttri32 dense array attribute
stride::mlir::DenseI32ArrayAttri32 dense array attribute
padding::mlir::DenseI32ArrayAttri32 dense array attribute
dilation::mlir::DenseI32ArrayAttri32 dense array attribute
groups::mlir::IntegerAttr32-bit signless integer attribute
conv2d_config::mlir::tt::ttnn::Conv2dConfigAttr
TTNN Conv2dConfig attribute{{% markdown %}} TTNN conv2d config attribute {{% /markdown %}}

Operands:

OperandDescription
inputranked tensor of any type values
weightranked tensor of any type values
biasranked tensor of any type values
deviceTTNN device

Results:

ResultDescription
resultranked tensor of any type values

ttnn.conv_transpose2d (tt::ttnn::ConvTranspose2dOp)

ConvTranspose2d operation.

Applies a 2D transposed convolution operator over an input image composed of several input planes.

Inputs:

  • input AnyRankedTensor: expected in the following format (N, H_in, W_in, C) where:

    • N is the batch size
    • H_in is the height of the input planes
    • W_in is the width of the input planes
    • C is the number of channels
  • weight AnyRankedTensor: expected in the following format (C, O/G, K_H, K_W).

  • bias Optional: expected in the following format (1, 1, 1, O) where:

    • C is the number of input channels
    • O is the number of output channels
    • G is the number of groups
    • K_H is the height of the kernel
    • K_W is the width of the kernel
  • output AnyRankedTensor: expected in the following format (N, H_out, W_out, O) where:

    • H_out = (H_in - 1) * stride[0] - 2 * padding[0] + dilation[0] * (K_H - 1) + output_padding[0] + 1
    • W_out = (W_in - 1) * stride[1] - 2 * padding[1] + dilation[1] * (K_W - 1) + output_padding[1] + 1

Attributes:

  • in_channels i32: The number of input channels.
  • out_channels i32: The number of output channels.
  • batch_size i32: The batch size.
  • input_height i32: The input height.
  • input_width i32: The input width.
  • kernel_size array<2xi32>: The kernel size.
  • stride array<2xi32>: Controls the stride for the cross-correlation.
  • padding array<2xi32>: Controls the amount of implicit zero padding on both sides for dilation * (kernel_size - 1) - padding number of points.
  • output_padding array<2xi32>: Controls the additional size added to one side of the output shape.
  • dilation array<2xi32>: Controls the spacing between the kernel points
  • groups i32: Controls the connections between inputs and outputs. Must be divisible by input and output channels.

Example: // %input: tensor<3x8x8x256xbf16> // %weight: tensor<256x256x3x3xbf16> // %bias: tensor<1x1x1x256xbf16> // %output: tensor<3x10x10x256xbf16> %0 = "ttnn.conv_transpose2d"(%input, %weight, %bias, %output, %device) <{ batch_size = 3: i32, dilation = array<i32: 1, 1>, groups = 1: i32, in_channels = 256: i32, input_height = 8: i32, input_width = 8: i32, kernel_size = array<i32: 3, 3>, out_channels = 256: i32, output_padding = array<i32: 0, 0>, padding = array<i32: 0, 0>, stride = array<i32: 1, 1> > : (tensor<3x8x8x256xbf16>, tensor<256x256x3x3xbf16>, tensor<1x1x1x256xbf16>, tensor<3x10x10x256xbf16>) -> tensor<3x10x10x256xbf16>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
in_channels::mlir::IntegerAttr32-bit signless integer attribute
out_channels::mlir::IntegerAttr32-bit signless integer attribute
batch_size::mlir::IntegerAttr32-bit signless integer attribute
input_height::mlir::IntegerAttr32-bit signless integer attribute
input_width::mlir::IntegerAttr32-bit signless integer attribute
kernel_size::mlir::DenseI32ArrayAttri32 dense array attribute
stride::mlir::DenseI32ArrayAttri32 dense array attribute
padding::mlir::DenseI32ArrayAttri32 dense array attribute
output_padding::mlir::DenseI32ArrayAttri32 dense array attribute
dilation::mlir::DenseI32ArrayAttri32 dense array attribute
groups::mlir::IntegerAttr32-bit signless integer attribute

Operands:

OperandDescription
inputranked tensor of any type values
weightranked tensor of any type values
biasranked tensor of any type values
deviceTTNN device

Results:

ResultDescription
resultranked tensor of any type values

ttnn.cos (tt::ttnn::CosOp)

Eltwise cosine.

Eltwise cosine operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.deallocate (tt::ttnn::DeallocateOp)

Deallocate op.

Tensor Deallocate operation

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
force::mlir::BoolAttrbool attribute

Operands:

OperandDescription
inputranked tensor of any type values

ttnn.divide (tt::ttnn::DivideOp)

Eltwise divide.

Eltwise divide operation.

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.embedding_bw (tt::ttnn::EmbeddingBackwardOp)

Embedding backward op.

Embedding backward operation. Generates the gradient of the embedding operation with respect to the input.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
dtype::mlir::tt::DataTypeAttr
TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}}
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
inputranked tensor of any type values
weightranked tensor of any type values
in_gradientranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.embedding (tt::ttnn::EmbeddingOp)

Embedding op.

Embedding operation.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputranked tensor of any type values
weightranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.empty (tt::ttnn::EmptyOp)

Empty op.

Tensor empty operation

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
shape::mlir::tt::ttnn::ShapeAttr
TTNN Shape attribute{{% markdown %}} TTNN shape attribute {{% /markdown %}}
dtype::mlir::tt::DataTypeAttr
TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}}
layout::mlir::tt::ttnn::LayoutAttr
TTNN Layout{{% markdown %}}Enum cases: * row_major (`RowMajor`) * tile (`Tile`) * invalid (`Invalid`){{% /markdown %}}
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
deviceTTNN device

Results:

ResultDescription
resultranked tensor of any type values

ttnn.eq (tt::ttnn::EqualOp)

Eltwise equal to.

Eltwise equal to operation.

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.exp (tt::ttnn::ExpOp)

Eltwise exponential.

Eltwise exponential operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.expm1 (tt::ttnn::Expm1Op)

Eltwise unary op.

Performs element-wise exponential minus one operation on operand tensor and stores the result in the output tensor.

Example: %a: [[0, 1], [0, 0]] "ttnn.exmp1"(%a, %out) -> %out: [[0, 1.71828], [0, 0]]

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.fill_cache (tt::ttnn::FillCacheOp)

Fill static cache tensor.

Fills the cache tensor in-place with values from input at batch_offset.

Interfaces: MemoryEffectOpInterface (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{MemoryEffects::Write on ::mlir::SideEffects::DefaultResource}

Attributes:

AttributeMLIR TypeDescription
batch_offset::mlir::IntegerAttr32-bit signless integer attribute

Operands:

OperandDescription
cacheranked tensor of any type values
inputranked tensor of any type values

ttnn.floor (tt::ttnn::FloorOp)

Eltwise floor op.

Eltwise floor operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.from_device (tt::ttnn::FromDeviceOp)

FromDevice op.

This op retrieves the input tensor from the given device.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.full (tt::ttnn::FullOp)

Full op.

Tensor full operation

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
fillValue::mlir::FloatAttr32-bit float attribute

Operands:

OperandDescription
deviceTTNN device

Results:

ResultDescription
resultranked tensor of any type values

ttnn.gelu (tt::ttnn::GeluOp)

Eltwise GELU.

Eltwise GELU operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.get_device (tt::ttnn::GetDeviceOp)

Get Device op.

This op returns the current runtime device.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, InferTypeOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
mesh_shape::mlir::tt::ttnn::MeshShapeAttr
TTNN Mesh Shape{{% markdown %}} TTNN mesh shape {{% /markdown %}}

Results:

ResultDescription
deviceTTNN device

ttnn.ge (tt::ttnn::GreaterEqualOp)

Eltwise greater than or equal to.

Eltwise greater than or equal to operation.

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.gt (tt::ttnn::GreaterThanOp)

Eltwise greater than.

Eltwise greater than operation.

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.isfinite (tt::ttnn::IsFiniteOp)

Eltwise isfinite op.

Eltwise isfinite operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.leaky_relu (tt::ttnn::LeakyReluOp)

Eltwise leaky relu operation.

The Leaky ReLU (Rectified Linear Unit) operation computes an element-wise activation function over its input tensor. It is defined as:

y = x if x > 0 y = parameter * x if x <= 0

where parameter is a small, user-defined constant that determines the slope for negative inputs.

Attributes:

  • parameter (float): The slope for negative values.

Inputs:

  • input (Tensor): The input tensor to be activated.

Outputs:

  • output (Tensor): The tensor after applying the Leaky ReLU activation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
parameter::mlir::FloatAttr32-bit float attribute

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.le (tt::ttnn::LessEqualOp)

Eltwise less than or equal to.

Eltwise less than or equal to operation.

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.lt (tt::ttnn::LessThanOp)

Eltwise less than.

Eltwise less than operation.

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.linear (tt::ttnn::LinearOp)

Linear transformation of inputs.

Produces the matmul of tensors a and b with optional addition with bias.

Example: // %a = [[1., 2.]], [2., 1.]] // %b = [[0., 1.], [1., 0.]] // %bias = [[1.]] "ttnn.linear"(%a, %b, %bias, %result) : (tensor<2x2xf16>, tensor<2x2xf16>, tensor<1xf16>, tensor<2x2xf16>) -> tensor<2x2xf16> // %result = [[3., 2.], [2., 3.]]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
transpose_a::mlir::BoolAttrbool attribute
transpose_b::mlir::BoolAttrbool attribute

Operands:

OperandDescription
aranked tensor of any type values
branked tensor of any type values
biasranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.log1p (tt::ttnn::Log1pOp)

Eltwise log1p operation.

Performs element-wise logarithm plus one operation on operand tensor and puts the result in the output tensor.

Example: %a: [0.0, -0.999, 7.0, 6.38905621, 15.0] "ttnn.logp1"(%a, %out) -> %out: [0.0, -6.90776825, 2.07944155, 2.0, 2.77258873]

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.log (tt::ttnn::LogOp)

Eltwise logarithm.

Eltwise logarithm operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.logical_and (tt::ttnn::LogicalAndOp)

Eltwise logical and.

Eltwise logical and operation.

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.logical_not (tt::ttnn::LogicalNotOp)

Eltwise logical not op.

Eltwise logical not operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.logical_or (tt::ttnn::LogicalOrOp)

Eltwise logical or.

Eltwise logical or operation.

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.logical_xor (tt::ttnn::LogicalXorOp)

Eltwise logical xor.

Eltwise logical xor operation.

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.matmul (tt::ttnn::MatmulOp)

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
transpose_a::mlir::BoolAttrbool attribute
transpose_b::mlir::BoolAttrbool attribute

Operands:

OperandDescription
aranked tensor of any type values
branked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.max (tt::ttnn::MaxOp)

Max reduction op.

Max reduction op.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
keep_dim::mlir::BoolAttrbool attribute
dim_arg::mlir::ArrayAttr32-bit integer array attribute

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.max_pool2d (tt::ttnn::MaxPool2dOp)

Applies a 2D max pooling over an input signal composed of several input planes.

Applies a 2D max pooling over an input signal composed of several input planes.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
batch_size::mlir::IntegerAttr32-bit signed integer attribute
input_height::mlir::IntegerAttr32-bit signed integer attribute
input_width::mlir::IntegerAttr32-bit signed integer attribute
channels::mlir::IntegerAttr32-bit signed integer attribute
kernel_size::mlir::DenseI32ArrayAttri32 dense array attribute
stride::mlir::DenseI32ArrayAttri32 dense array attribute
padding::mlir::DenseI32ArrayAttri32 dense array attribute
dilation::mlir::DenseI32ArrayAttri32 dense array attribute
ceil_mode::mlir::BoolAttrbool attribute

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.maximum (tt::ttnn::MaximumOp)

Eltwise maximum OP.

Calculates maximum of input tensors' values element-wise and stores result in output tensor.

Example: %lhs: [[3, 2, 7], [1, 4, 4]] %rhs: [[1, 4, 2], [1, 2, 3]] "ttnn.maximum"(%lhs, %rhs, %out) -> %out: [[3, 4, 7], [1, 4, 4]]

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.mean (tt::ttnn::MeanOp)

Mean reduction op.

Mean reduction op.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
keep_dim::mlir::BoolAttrbool attribute
dim_arg::mlir::ArrayAttr32-bit integer array attribute

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.mesh_shard (tt::ttnn::MeshShardOp)

Mesh shard op.

Tensor Mesh Shard operation

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
shard_direction::mlir::tt::MeshShardDirectionAttr
TT MeshShardDirection{{% markdown %}}Enum cases: * full_to_shard (`FullToShard`) * shard_to_full (`ShardToFull`){{% /markdown %}}
shard_type::mlir::tt::MeshShardTypeAttr
MeshShard shard_type attribute in TT dialect{{% markdown %}} Define sharded tensor data of mesh_shard op. - Identity: input and output tensors are pre-sharded (same data) and no sharding is required. - Replicate: all of the devices has full tensor (same data). - Maximal: one or part of the devcices has full tensor (same data). - Devices: all or part of the devices has sharded (partial) tensor (different data). {{% /markdown %}}
shard_shape::mlir::DenseI64ArrayAttri64 dense array attribute
shard_dims::mlir::DenseI64ArrayAttri64 dense array attribute

Operands:

OperandDescription
inputranked tensor of any type values
deviceTTNN device

Results:

ResultDescription
resultranked tensor of any type values

ttnn.min (tt::ttnn::MinOp)

Min reduction op.

This op computes the minimum of all elements of the tensor or along specified dimension.

Example: input: [[1, 5, 3], [4, 2, 6]]

// Computing along dim 0 output: [1, 2, 3]

// Computing along dim 1 output: [1, 2]

// Computing for entire tensor output: 1

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
keep_dim::mlir::BoolAttrbool attribute
dim_arg::mlir::ArrayAttr32-bit integer array attribute

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.minimum (tt::ttnn::MinimumOp)

Eltwise minimum OP.

Calculates minimum of input tensors' values element-wise and stores result in output tensor.

Example: %lhs: [[3, 2, 7], [1, 4, 4]] %rhs: [[1, 4, 2], [1, 2, 3]] "ttnn.minimum"(%lhs, %rhs, %out) -> %out: [[1, 2, 2], [1, 2, 3]]

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.moreh_cumsum (tt::ttnn::MorehCumSumOp)

Moreh cummulative sum op.

Computes the cumulative sum of elements of a tensor along specified dimension.

Example: input: [[1, 2, 3], [4, 5, 6]]

// Cumulative sum along dim=0: output: [[1, 2, 3], [5, 7, 9]]

// Cumulative sum along dim=1: output: [[1, 3, 6], [4, 9, 15]]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
dim::mlir::IntegerAttr64-bit signless integer attribute
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.multiply (tt::ttnn::MultiplyOp)

Eltwise multiply.

Eltwise multiply operation.

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.neg (tt::ttnn::NegOp)

Eltwise negate.

Eltwise negate operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.ne (tt::ttnn::NotEqualOp)

Eltwise not equal to.

Eltwise not equal to operation.

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.ones (tt::ttnn::OnesOp)

Creates a tensor filled with ones.

Tensor operation to create a tensor filled with ones.

Given a ShapeAttr shape, produces a tensor with the same shape, filled with ones.

Example: %0 = "ttnn.ones"() <{shape = array<i32:64, 28, 28>}> : () -> tensor<64x28x28xbf16> // %0: [[[1, 1, 1, ..., 1], [1, 1, 1, ..., 1], ..., [1, 1, 1, ..., 1]]]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
shape::mlir::tt::ttnn::ShapeAttr
TTNN Shape attribute{{% markdown %}} TTNN shape attribute {{% /markdown %}}
dtype::mlir::tt::DataTypeAttr
TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}}
layout::mlir::tt::ttnn::LayoutAttr
TTNN Layout{{% markdown %}}Enum cases: * row_major (`RowMajor`) * tile (`Tile`) * invalid (`Invalid`){{% /markdown %}}
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
deviceTTNN device

Results:

ResultDescription
resultranked tensor of any type values

ttnn.pad (tt::ttnn::PadOp)

Pad op.

Pad input tensor by padding the input_shape to output_shape using the provided value.

The padding attribute must be a sequence of integers that is twice the size as the rank of the input. Each pair of integers in the padding attribute represents the amount of padding to add to the low and high of that dimension. I.e: an input tensor of shape <1x30x30x64xf32> with padding attribute <0, 0, 1, 1, 1, 1, 0, 0> will return a tensor of shape <1x32x32x64xf32>, and so will a padding attribute of <0, 0, 0, 2, 0, 2, 0, 0>.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
padding::mlir::DenseI32ArrayAttri32 dense array attribute
value::mlir::FloatAttr32-bit float attribute
use_multicore::mlir::BoolAttrbool attribute
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.permute (tt::ttnn::PermuteOp)

Permute operation.

Permute input tensor dimensions.

Attributes:

  • permutation array: The permutation of the input tensor dimensions.

Example: %a = ttir.empty() : () -> tensor<2x3x4xi32> %0 = "ttir.permute"(%a) {permutation = array<i64: 1, 2, 0>} : (tensor<2x3x4xi32>) -> tensor<3x4x2xi32>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
permutation::mlir::DenseI64ArrayAttri64 dense array attribute
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}
pad_value::mlir::FloatAttr32-bit float attribute

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.pow (tt::ttnn::PowerOp)

Eltwise power OP.

Performs element-wise exponentiation of lhs tensor by rhs tensor and produces a result tensor. Tensors must be of same shape.

Example:

  %result = "ttnn.pow"(%lhs, %rhs) : (tensor<6xf64>, tensor<6xf64>) -> tensor<6xf64>

  %lhs: [-2.0, -0.0, -36.0, 5.0, 3.0, 10000.0]
  %rhs: [2.0, 2.0, 1.1, 2.0, -1.0, 10.0]
  %result: [4.0, 0.0, -nan, 25.0, 0.333333343, inf]

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.prod (tt::ttnn::ProdOp)

Product reduction op.

This op computes the product of all elements of the tensor (full product) or along a specific dimension.

Example: input: [[1, 2, 3], [4, 5, 6]]

// Computing along dim 0 output: [4, 10, 18]

// Computing along dim 1 output: [6, 120]

// Computing full product output: 720

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
all_dimensions::mlir::BoolAttrbool attribute
keep_dim::mlir::BoolAttrbool attribute
dim_arg::mlir::IntegerAttr64-bit signless integer attribute
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.reciprocal (tt::ttnn::ReciprocalOp)

Eltwise reciprocal.

Eltwise reciprocal operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.reduce_scatter (tt::ttnn::ReduceScatterOp)

Reduce scatter op.

Tensor Reduce Scatter operation

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
reduce_type::mlir::tt::ReduceTypeAttr
TT Reduce Type{{% markdown %}}Enum cases: * sum (`Sum`) * mean (`Mean`) * max (`Max`) * min (`Min`) * std (`Std`) * var (`Var`){{% /markdown %}}
scatter_dim::mlir::IntegerAttr32-bit signed integer attribute
cluster_axis::mlir::IntegerAttr32-bit unsigned integer attribute
num_links::mlir::IntegerAttr32-bit unsigned integer attribute

Operands:

OperandDescription
inputranked tensor of any type values
deviceTTNN device

Results:

ResultDescription
resultranked tensor of any type values

ttnn.relu (tt::ttnn::ReluOp)

Eltwise ReLU.

Eltwise ReLU operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.remainder (tt::ttnn::RemainderOp)

Eltwise remainder.

Performs element-wise remainder of dividend lhs and divisor rhs tensors and produces a result tensor.

Example:

// %lhs: [17, -17, 17, -17] // %rhs: [3, 3, -3, -3] %result = "ttnn.remainder"(%lhs, %rhs) : (tensor<4xi64>, tensor<4xi64>) -> tensor<4xi64> // %result: [2, -2, 2, -2]

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.repeat_interleave (tt::ttnn::RepeatInterleaveOp)

Repeat interleave op.

Repeats elements of a tensor along a specified dimension. It allows for flexible repetition patterns, where each element can be repeated a different number of times. This is particularly useful for tasks that require duplicating elements in a non-uniform manner.

Parameters:

  • input: The input tensor.
  • repeats: Specifies the number of repetitions for each element, each element is repeated that number of times.
  • dim: The dimension along which to repeat values.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
repeats::mlir::IntegerAttr32-bit unsigned integer attribute
dim::mlir::IntegerAttr32-bit signed integer attribute
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.repeat (tt::ttnn::RepeatOp)

Repeat op.

Returns a new tensor filled with repetition of input tensor according to number of times specified in repeat_dims.

Parameters:

  • input_tensor (ttnn.Tensor): the input tensor.
  • repeat_dims (number): The number of repetitions for each element.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
repeat_dims::mlir::tt::ttnn::ShapeAttr
TTNN Shape attribute{{% markdown %}} TTNN shape attribute {{% /markdown %}}

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.reshape (tt::ttnn::ReshapeOp)

Reshape op.

Reshape tensor.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
shape::mlir::ArrayAttr32-bit integer array attribute
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.rsqrt (tt::ttnn::RsqrtOp)

Eltwise rsqrt.

Eltwise rsqrt operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.scatter (tt::ttnn::ScatterOp)

Scatter op.

Embeds the values of the 'update' tensor into 'input' at the given index and puts the value in the 'output' tensor.

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.sigmoid (tt::ttnn::SigmoidOp)

Eltwise sigmoid.

Eltwise sigmoid operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.sign (tt::ttnn::SignOp)

Eltwise sign operation.

Returns the sign of the operand element-wise and produces a result tensor.

Example: %a: [[3, -2, 0], [1, -4, 4]] "ttnn.sign"(%a, %out) -> %out: [[1, -1, 0], [1, -1, 1]]

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.sin (tt::ttnn::SinOp)

Eltwise sine.

Eltwise sine operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.slice (tt::ttnn::SliceOp)

Slice op.

Extract a portion of a tensor based on the specified start (begins), stop (ends), and step indices for each dimension.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
begins::mlir::ArrayAttr32-bit integer array attribute
ends::mlir::ArrayAttr32-bit integer array attribute
step::mlir::ArrayAttr32-bit integer array attribute

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.softmax (tt::ttnn::SoftmaxOp)

Softmax op.

Softmax operation.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
dimension::mlir::IntegerAttr32-bit signed integer attribute

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.sqrt (tt::ttnn::SqrtOp)

Eltwise sqrt.

Eltwise sqrt operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.subtract (tt::ttnn::SubtractOp)

Eltwise subtract.

Eltwise subtract operation.

Traits: AlwaysSpeculatableImplTrait, TwoOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.sum (tt::ttnn::SumOp)

Sum reduction op.

Sum reduction op.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
keep_dim::mlir::BoolAttrbool attribute
dim_arg::mlir::ArrayAttr32-bit integer array attribute

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.tan (tt::ttnn::TanOp)

Eltwise tan op.

Eltwise tan operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.tanh (tt::ttnn::TanhOp)

Eltwise tanh op.

Eltwise tanh operation.

Traits: AlwaysSpeculatableImplTrait, OneOperand

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.to_dtype (tt::ttnn::ToDTypeOp)

ToDType op.

This op converts the data type of the input tensor based on the given data type on the host.

Args:

  • :attr:input: the ttnn.Tensor
  • :attr:dtype: ttnn data type.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
dtype::mlir::tt::DataTypeAttr
TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}}

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.to_device (tt::ttnn::ToDeviceOp)

ToDevice op.

This op sends the input tensor to the given device with the given memory config.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
inputranked tensor of any type values
deviceTTNN device

Results:

ResultDescription
resultranked tensor of any type values

ttnn.to_layout (tt::ttnn::ToLayoutOp)

ToLayout op.

This op wraps all layout information gathered from ttir.toLayout. It is used/updated by the optimizer to perform optimizations, and later broken down into specific memory/layout operations (toDevice, toMemoryConfig etc.). Currently in the TTNN backend, we use this op solely for tilize/untilize, therefore marking all other attrs as optional. Once ttnn::to_layout supports other attrs, we can remove the optional tag.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
layout::mlir::tt::ttnn::LayoutAttr
TTNN Layout{{% markdown %}}Enum cases: * row_major (`RowMajor`) * tile (`Tile`) * invalid (`Invalid`){{% /markdown %}}
dtype::mlir::tt::DataTypeAttr
TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}}
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
inputranked tensor of any type values
deviceTTNN device

Results:

ResultDescription
resultranked tensor of any type values

ttnn.to_memory_config (tt::ttnn::ToMemoryConfigOp)

ToMemoryConfig op.

This op converts the memory config of the input tensor based on the given memory config. It handles:

  • Dram to L1
  • L1 to Dram
  • Interleaved to sharded
  • Sharded to interleaved
  • Sharded to sharded (reshard)

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.transpose (tt::ttnn::TransposeOp)

Transpose op.

Transpose tensor along two given dimensions.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
dim0::mlir::IntegerAttr32-bit signed integer attribute
dim1::mlir::IntegerAttr32-bit signed integer attribute

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.typecast (tt::ttnn::TypecastOp)

Typecast op.

This op converts the data type of the input tensor based on the given data type. It handles:

  • conversions of data types.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
dtype::mlir::tt::DataTypeAttr
TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}}

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.update_cache (tt::ttnn::UpdateCacheOp)

Update static cache tensor.

Updates the cache tensor in-place with values from input at update_index and batch_offset.

Interfaces: MemoryEffectOpInterface (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{MemoryEffects::Write on ::mlir::SideEffects::DefaultResource}

Attributes:

AttributeMLIR TypeDescription
batch_offset::mlir::IntegerAttr32-bit signless integer attribute

Operands:

OperandDescription
cacheranked tensor of any type values
inputranked tensor of any type values
update_indexranked tensor of any type values

ttnn.upsample (tt::ttnn::UpsampleOp)

Upsample 2D op.

Upsample 2D operation. Input tensor is assumed to be in NHWC format.

Attributes:

  • scale_factor (si32 | array): The scale factor for upsampling in H and W dimensions respectively.
  • mode (str): The upsampling algorithm. Currently only "nearest" and "bilinear" are supported. Default is "nearest".

Example: // %a: tensor<10x64x32xbf16> %0 = "ttnn.upsample"(%a) <{scale_factor = array<i32: 2, 4>}> : (tensor<10x64x32x3xbf16>) -> tensor<10x128x128x3xbf16>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
scale_factor::mlir::Attribute32-bit signed integer attribute or i32 dense array attribute
mode::mlir::StringAttrstring attribute
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
inputranked tensor of any type values

Results:

ResultDescription
resultranked tensor of any type values

ttnn.where (tt::ttnn::WhereOp)

Eltwise where.

Eltwise where operation.

Traits: AlwaysSpeculatableImplTrait, ThreeOperands

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

OperandDescription
inputsvariadic of ranked tensor of any type values

Results:

ResultDescription
resultsvariadic of ranked tensor of any type values

ttnn.zeros (tt::ttnn::ZerosOp)

Creates a tensor filled with zeros.

Tensor operation to create a tensor filled with zeros.

Given a ShapeAttr shape, produces a tensor with the same shape, filled with zeros.

Example: %0 = "ttnn.zeros"() <{shape = array<i32:64, 28, 28>}> : () -> tensor<64x28x28xbf16> // %0: [[[0, 0, 0, ..., 0], [0, 0, 0, ..., 0], ..., [0, 0, 0, ..., 0]]]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

AttributeMLIR TypeDescription
shape::mlir::tt::ttnn::ShapeAttr
TTNN Shape attribute{{% markdown %}} TTNN shape attribute {{% /markdown %}}
dtype::mlir::tt::DataTypeAttr
TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}}
layout::mlir::tt::ttnn::LayoutAttr
TTNN Layout{{% markdown %}}Enum cases: * row_major (`RowMajor`) * tile (`Tile`) * invalid (`Invalid`){{% /markdown %}}
memory_config::mlir::tt::ttnn::MemoryConfigAttr
TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}}

Operands:

OperandDescription
deviceTTNN device

Results:

ResultDescription
resultranked tensor of any type values