tt-mlir documentation

ttnn.abs (tt::ttnn::AbsOp)

Eltwise absolute.

Eltwise absolute operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.add (tt::ttnn::AddOp)

Eltwise add.

Eltwise add operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.all_gather (tt::ttnn::AllGatherOp)

All gather op.

Tensor All Gather operation

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.all_reduce (tt::ttnn::AllReduceOp)

All reduce op.

Tensor All Reduce operation

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.alloc (tt::ttnn::AllocOp)

Alloc op.

Tensor Alloc operation

Traits: AlwaysSpeculatableImplTrait, TT_CreationOpTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Results:

ttnn.arange (tt::ttnn::ArangeOp)

Arange operation.

Tensor arange operation.

Produces a (1, 1, 1, N)-shaped tensor with values from start to end (exclusive) with a step size of step.

Examples: %0 = "ttnn.arange"() {start = 0 : i64, end = 5 : i64 step = 1 : i64} : () -> tensor<1x1x1x5xi64> // %0: [[[[0, 1, 2, 3, 4]]]]

%1 = "ttnn.arange"() {start = 0 : i64, end = 10 : i64, step = 2 : i64} : () -> tensor<1x1x1x5xf32> // %1: [[[[0.0, 2.0, 4.0, 6.0, 8.0]]]]

Traits: AlwaysSpeculatableImplTrait, TT_CreationOpTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.argmax (tt::ttnn::ArgMaxOp)

Argmax reduction op.

Determine the indices of the maximum values along a specified dimension of a tensor or over all elements in a tensor.

Parameters:

• input: The input tensor.
• dim: Specifies the dimension along which the argmax is applied.
• use_multicore: Whether to use multiple cores or not.

IR usage: // Input tensor of shape (128, 28, 28, 64) %input = ... : tensor<128x28x28x64xbf16>

%empty = "ttnn.empty"(%0) <{dtype = #tt.supportedDataTypes, ....}> : -> tensor<128x28x28xi32> %4 = "ttnn.argmax"(%input, %empty) <{dim = 3 : i32, use_multicore = false}> : (tensor<128x28x28xbf16>, tensor<128x28x28xi32) -> tensor<128x28x28xi32>

Example: input: [[1, 5, 3], [2, 4, 6]]

// Computing along dim 0 output: [1, 0, 1]

// Computing along dim 1 output: [1, 2]

// Computing for entire tensor output: 5

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.atan2 (tt::ttnn::Atan2Op)

Eltwise atan2 OP.

Performs element-wise atan2 operation on lhs and rhs tensor and produces a result tensor.

Example:

  // %lhs: [0.0, 1.0, -1.0]
  // %rhs: [1.0, 0.0, 0.0]
  %result = "ttnn.atan2"(%lhs, %rhs) : (tensor<3xf64>, tensor<3xf64>) -> tensor<3xf64>
  // %result: [0.0, 1.57079637, -1.57079637] // [0.0, pi/2, -pi/2]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.atan (tt::ttnn::AtanOp)

Eltwise arctangent op.

Performs an elementwise arctangent (atan) operation on the input tensor. This operation computes the inverse tangent of each element, returning values in the range [-π/2, π/2]. Supports floating-point tensor types.

Example:

%input = tensor<4xf32> {1.0, 0.5, 0.0, -1.0}
%result = "ttir.atan"(%input) : (tensor<4xf32>) -> tensor<4xf32>

Given the input [1.0, 0.5, 0.0, -1.0], the result would be approximately: [0.785, 0.464, 0.0, -0.785] (values in radians).

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.bitwise_and (tt::ttnn::BitwiseAndOp)

Eltwise bitwise AND.

Performs element-wise bitwise AND of two tensors lhs and rhs and produces a result tensor.

Example: // %lhs: [[1, 2], [3, 4]] // %rhs: [[5, 6], [7, 8]] %result = "ttnn.bitwise_and"(%lhs, %rhs) : (tensor<2x2xi32>, tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[1, 2], [3, 0]]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.bitwise_not (tt::ttnn::BitwiseNotOp)

Eltwise bitwise NOT.

Performs element-wise NOT of tensor operand and produces a result tensor.

Example: // Bitwise operation with with integer tensors // %operand: [[1, 2], [3, 4]] %result = "ttnn.bitwise_not"(%operand) : (tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[-2, -3], [-4, -5]]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.bitwise_or (tt::ttnn::BitwiseOrOp)

Eltwise bitwise OR.

Performs element-wise bitwise OR of two tensors lhs and rhs and produces a result tensor.

Example: // %lhs: [[1, 2], [3, 4]] // %rhs: [[5, 6], [7, 8]] %result = "ttnn.bitwise_or"(%lhs, %rhs) : (tensor<2x2xi32>, tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[5, 6], [7, 12]]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.bitwise_xor (tt::ttnn::BitwiseXorOp)

Eltwise bitwise XOR.

Performs element-wise bitwise XOR of two tensors lhs and rhs and produces a result tensor.

Example: // %lhs: [[1, 2], [3, 4]] // %rhs: [[5, 6], [7, 8]] %result = "ttnn.bitwise_xor"(%lhs, %rhs) : (tensor<2x2xi32>, tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[4, 4], [4, 12]]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.cbrt (tt::ttnn::CbrtOp)

Eltwise cubic root.

Eltwise cubic root operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.ceil (tt::ttnn::CeilOp)

Eltwise ceil.

Eltwise ceil operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.clamp_scalar (tt::ttnn::ClampScalarOp)

Clamp op.

Clamp tensor values to a specified range.

Example: min: 2.000000+00 input: [[0, 1, 2, 3, 4, 5, 6, 7]] max: 5.000000+00

"ttnn.clamp_scalar"(%arg0) <{max = 2.000000e+00 : f32, min = 5.000000e+00 : f32}> -> %out = [[2, 2, 2, 3, 4, 5, 5, 5]]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.clamp_tensor (tt::ttnn::ClampTensorOp)

Clamp op.

Clamp tensor values to a specified range using min/max as tensor.

Example: min: [[2, 2, 2, 3, 3, 3, 0, 0]] input: [[0, 1, 2, 3, 4, 5, 6, 7]] max: [[5, 5, 5, 9, 9, 9, 6, 6]]

"ttnn.clamp_tensor"(%input, %min, %max) %out: [[2, 2, 2, 3, 4, 5, 6, 6]]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.collective_permute (tt::ttnn::CollectivePermuteOp)

Collective permute op.

Collective permute op. This operation ingests a multi-device tensor spread across multi-devices and will shuffle the data according to source_target_pairs [['src', 'dest']].

Example: For a 1x2 mesh, the following will take the device shard living in device 0 and move it to device 1. The device shard living in device 1 will move to device 0. %source_target_pairs: [[0, 1], [1, 0]]

In the case of missing 'dest', the device shard living on that device will contain values of 0. For example, device shard living in device 0 will contain 0 values. %source_target_pairs: [[0, 1]]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.concat (tt::ttnn::ConcatOp)

Concat op.

Concat tensors along a given dimension.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.constant (tt::ttnn::ConstantOp)

Constant op.

Produces tensor filled with given constant value.

Examples: %0 = "ttnn.constant"() {value = dense<[[3, 4, 2], [1, 7, 8]]> : tensor<2x3xui16>} : () -> tensor<2x3xui16> // %0: [[3, 4, 2], [1, 7, 8]] %1 = "ttnn.constant"() {value = dense<[0.2, 1.3]> : tensor<2xf32>} : () -> tensor<2xf32> // %1: [0.2, 1.3]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Results:

ttnn.construct_tensor (tt::ttnn::ConstructTensorOp)

Construct empty tensor on host op.

Invoke tensor ctor to create an empty tensor on host.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Results:

ttnn.conv2d (tt::ttnn::Conv2dOp)

Conv2d operation.

Applies a 2D convolution over an input image composed of several input planes.

Inputs:

• input (AnyRankedTensor): expected in the following flattened format (1, 1, N * H_in * W_in, C) where:
  • N is the batch size
  • H_in is the height of the input planes
  • W_in is the width of the input planes
  • C is the number of channels
• weight (AnyRankedTensor): expected in the following format (O, C/G, K_H, K_W).
• bias (Optional): expected in the following format (1, 1, 1, O) where:
  • C is the number of input channels
  • O is the number of output channels
  • G is the number of groups
  • K_H is the height of the kernel
  • K_W is the width of the kernel

Attributes:

• in_channels (i32): The number of input channels.
• out_channels (i32): The number of output channels.
• batch_size (i32): The batch size.
• input_height (i32): The input height.
• input_width (i32): The input width.
• kernel_size (array<2xi32>): [K_H, K_W] where K_H is the kernel height and K_W is the kernel width.
• stride (array<2xi32>): [sH, sW] where sH is stride for height and sW is stride for width.
• padding (array<2xi32>): [pH, pW] where pH is padding for height (top/bottom) and pW is padding for width (left/right).
• dilation (array<2xi32>): [dH, dW] where dH is dilation for height and dW is dilation for width.
• groups (i32): Number of blocked connections from input channels to output channels. Input and output channels must both be divisible by groups.

Outputs:

• result (AnyRankedTensor): returned in the following flattened format (1, 1, N * H_out * W_out, O) where:
  • H_out = (H_in + 2 * pH - dH * (K_H - 1) - 1) / sH + 1
  • W_out = (W_in + 2 * pW - dW * (K_W - 1) - 1) / sW + 1

Example: %input = ttir.empty() : () -> tensor<1x1x1024x64xbf16> %weight = ttir.empty() : () -> tensor<64x64x3x3xbf16> %bias = ttir.empty() : () -> tensor<1x1x1x64xbf16> %device = "ttnn.get_device"() <{mesh_shape = #ttnn<mesh_shape 1x1>}> : () -> !ttnn.device %0 = "ttnn.conv2d"(%input, %weight, %bias, %device) <{ in_channels = 64: i32, out_channels = 64: i32, batch_size = 1: i32, input_height = 32: i32, input_width = 32: i32, kernel_size = array<i32: 3, 3>, stride = array<i32: 1, 1>, padding = array<i32: 0, 0>, dilation = array<i32: 1, 1>, groups = 1: i32 }> : (tensor<1x1x1024x64xbf16>, tensor<64x64x3x3xbf16>, tensor<1x1x1x64xbf16>, !ttnn.device) -> tensor<1x1x900x64xbf16>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.conv_transpose2d (tt::ttnn::ConvTranspose2dOp)

ConvTranspose2d operation.

Applies a 2D transposed convolution operator over an input image composed of several input planes.

Inputs:

• input AnyRankedTensor: expected in the following format (N, H_in, W_in, C) where:

  • N is the batch size
  • H_in is the height of the input planes
  • W_in is the width of the input planes
  • C is the number of channels

• weight AnyRankedTensor: expected in the following format (C, O/G, K_H, K_W).

• bias Optional: expected in the following format (1, 1, 1, O) where:

  • C is the number of input channels
  • O is the number of output channels
  • G is the number of groups
  • K_H is the height of the kernel
  • K_W is the width of the kernel

• output AnyRankedTensor: expected in the following format (N, H_out, W_out, O) where:

  • H_out = (H_in - 1) * stride[0] - 2 * padding[0] + dilation[0] * (K_H - 1) + output_padding[0] + 1
  • W_out = (W_in - 1) * stride[1] - 2 * padding[1] + dilation[1] * (K_W - 1) + output_padding[1] + 1

Attributes:

• in_channels i32: The number of input channels.
• out_channels i32: The number of output channels.
• batch_size i32: The batch size.
• input_height i32: The input height.
• input_width i32: The input width.
• kernel_size array<2xi32>: The kernel size.
• stride array<2xi32>: Controls the stride for the cross-correlation.
• padding array<2xi32>: Controls the amount of implicit zero padding on both sides for dilation * (kernel_size - 1) - padding number of points.
• output_padding array<2xi32>: Controls the additional size added to one side of the output shape.
• dilation array<2xi32>: Controls the spacing between the kernel points
• groups i32: Controls the connections between inputs and outputs. Must be divisible by input and output channels.

Example: // %input: tensor<3x8x8x256xbf16> // %weight: tensor<256x256x3x3xbf16> // %bias: tensor<1x1x1x256xbf16> // %output: tensor<3x10x10x256xbf16> %0 = "ttnn.conv_transpose2d"(%input, %weight, %bias, %output, %device) <{ batch_size = 3: i32, dilation = array<i32: 1, 1>, groups = 1: i32, in_channels = 256: i32, input_height = 8: i32, input_width = 8: i32, kernel_size = array<i32: 3, 3>, out_channels = 256: i32, output_padding = array<i32: 0, 0>, padding = array<i32: 0, 0>, stride = array<i32: 1, 1> > : (tensor<3x8x8x256xbf16>, tensor<256x256x3x3xbf16>, tensor<1x1x1x256xbf16>, tensor<3x10x10x256xbf16>) -> tensor<3x10x10x256xbf16>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.cos (tt::ttnn::CosOp)

Eltwise cosine.

Eltwise cosine operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.deallocate (tt::ttnn::DeallocateOp)

Deallocate op.

Tensor Deallocate operation

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

ttnn.dequantize (tt::ttnn::DequantizeOp)

Dequantize operation.

Applies dequantization to the input tensor.

Inputs:

• input AnyRankedTensor: The input tensor to be dequantized.
• scale f32: The scale factor.
• zero_point i32: The zero point value.
• axis Optional: The axis along which quantization is applied.
• output_dtype Optional<TT_DataTypeAttr>: The data type of the output tensor.
• memory_config Optional<TTNN_MemoryConfigAttr>: The memory configuration for the output tensor.

output[i] = (input[i] - zero_point) * scale

Example:

%input = ttir.empty() : () -> tensor<64x128x!quant.uniform<i32:f32, 0.1>>
%output = ttir.empty() : () -> tensor<64x128xf32>
%dequantized = "ttnn.dequantize"(%input, %output) : (tensor<64x128x!quant.uniform<i32:f32, 0.1>, tensor<64x128xf32>) -> tensor<64x128xf32>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.divide (tt::ttnn::DivideOp)

Eltwise divide.

Eltwise divide operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.embedding_bw (tt::ttnn::EmbeddingBackwardOp)

Embedding backward op.

Embedding backward operation. Generates the gradient of the embedding operation with respect to the input.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.embedding (tt::ttnn::EmbeddingOp)

Embedding op.

Embedding operation.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttnn.empty (tt::ttnn::EmptyOp)

Empty op.

Tensor empty operation

Traits: AlwaysSpeculatableImplTrait, TT_CreationOpTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.eq (tt::ttnn::EqualOp)

Eltwise equal to.

Eltwise equal to operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.exp (tt::ttnn::ExpOp)

Eltwise exponential.

Eltwise exponential operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.expm1 (tt::ttnn::Expm1Op)

Eltwise unary op.

Performs element-wise exponential minus one operation on operand tensor and stores the result in the output tensor.

Example: %a: [[0, 1], [0, 0]] "ttnn.exmp1"(%a, %out) -> %out: [[0, 1.71828], [0, 0]]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.fill_cache (tt::ttnn::FillCacheOp)

Fill static cache tensor.

Fills the cache tensor in-place with values from input at batch_offset.

Interfaces: MemoryEffectOpInterface (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{MemoryEffects::Write on ::mlir::SideEffects::DefaultResource}

Attributes:

Operands:

ttnn.floor (tt::ttnn::FloorOp)

Eltwise floor op.

Eltwise floor operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.from_device (tt::ttnn::FromDeviceOp)

FromDevice op.

This op retrieves the input tensor from the given device.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttnn.full (tt::ttnn::FullOp)

Full op.

Tensor full operation

Traits: AlwaysSpeculatableImplTrait, TT_CreationOpTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.gelu (tt::ttnn::GeluOp)

Eltwise GELU.

Eltwise GELU operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.get_device (tt::ttnn::GetDeviceOp)

Get Device op.

This op returns a submesh carved out from the parent runtime device. Mesh shape and mesh offset define the size and offset of the submesh.

Traits: AlwaysSpeculatableImplTrait, TT_DuplicateConstEvalTrait

Interfaces: ConditionallySpeculatable, InferTypeOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Results:

ttnn.ge (tt::ttnn::GreaterEqualOp)

Eltwise greater than or equal to.

Eltwise greater than or equal to operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.gt (tt::ttnn::GreaterThanOp)

Eltwise greater than.

Eltwise greater than operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.isfinite (tt::ttnn::IsFiniteOp)

Eltwise isfinite op.

Eltwise isfinite operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.leaky_relu (tt::ttnn::LeakyReluOp)

Eltwise leaky relu operation.

The Leaky ReLU (Rectified Linear Unit) operation computes an element-wise activation function over its input tensor. It is defined as:

y = x if x > 0 y = parameter * x if x <= 0

where parameter is a small, user-defined constant that determines the slope for negative inputs.

Attributes:

• parameter (float): The slope for negative values.

Inputs:

• input (Tensor): The input tensor to be activated.

Outputs:

• output (Tensor): The tensor after applying the Leaky ReLU activation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.le (tt::ttnn::LessEqualOp)

Eltwise less than or equal to.

Eltwise less than or equal to operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.lt (tt::ttnn::LessThanOp)

Eltwise less than.

Eltwise less than operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.linear (tt::ttnn::LinearOp)

Linear transformation of inputs.

Produces the matmul of tensors a and b with optional addition with bias.

Example: // %a = [[1., 2.]], [2., 1.]] // %b = [[0., 1.], [1., 0.]] // %bias = [[1.]] "ttnn.linear"(%a, %b, %bias, %result) : (tensor<2x2xf16>, tensor<2x2xf16>, tensor<1xf16>, tensor<2x2xf16>) -> tensor<2x2xf16> // %result = [[3., 2.], [2., 3.]]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.log1p (tt::ttnn::Log1pOp)

Eltwise log1p operation.

Performs element-wise logarithm plus one operation on operand tensor and puts the result in the output tensor.

Example: %a: [0.0, -0.999, 7.0, 6.38905621, 15.0] "ttnn.logp1"(%a, %out) -> %out: [0.0, -6.90776825, 2.07944155, 2.0, 2.77258873]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.log (tt::ttnn::LogOp)

Eltwise logarithm.

Eltwise logarithm operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.logical_and (tt::ttnn::LogicalAndOp)

Eltwise logical and.

Eltwise logical and operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.logical_not (tt::ttnn::LogicalNotOp)

Eltwise logical not op.

Eltwise logical not operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.logical_or (tt::ttnn::LogicalOrOp)

Eltwise logical or.

Eltwise logical or operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.logical_xor (tt::ttnn::LogicalXorOp)

Eltwise logical xor.

Eltwise logical xor operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.matmul (tt::ttnn::MatmulOp)

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.max (tt::ttnn::MaxOp)

Max reduction op.

Max reduction op.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.max_pool2d (tt::ttnn::MaxPool2dOp)

Applies a 2D max pooling over an input signal composed of several input planes.

Applies a 2D max pooling over an input signal composed of several input planes.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.maximum (tt::ttnn::MaximumOp)

Eltwise maximum OP.

Calculates maximum of input tensors' values element-wise and stores result in output tensor.

Example: %lhs: [[3, 2, 7], [1, 4, 4]] %rhs: [[1, 4, 2], [1, 2, 3]] "ttnn.maximum"(%lhs, %rhs, %out) -> %out: [[3, 4, 7], [1, 4, 4]]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.mean (tt::ttnn::MeanOp)

Mean reduction op.

Mean reduction op.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.mesh_shard (tt::ttnn::MeshShardOp)

Mesh shard op.

Tensor Mesh Shard operation

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.min (tt::ttnn::MinOp)

Min reduction op.

This op computes the minimum of all elements of the tensor or along specified dimension.

Example: input: [[1, 5, 3], [4, 2, 6]]

// Computing along dim 0 output: [1, 2, 3]

// Computing along dim 1 output: [1, 2]

// Computing for entire tensor output: 1

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.minimum (tt::ttnn::MinimumOp)

Eltwise minimum OP.

Calculates minimum of input tensors' values element-wise and stores result in output tensor.

Example: %lhs: [[3, 2, 7], [1, 4, 4]] %rhs: [[1, 4, 2], [1, 2, 3]] "ttnn.minimum"(%lhs, %rhs, %out) -> %out: [[1, 2, 2], [1, 2, 3]]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.moreh_cumsum (tt::ttnn::MorehCumSumOp)

Moreh cummulative sum op.

Computes the cumulative sum of elements of a tensor along specified dimension.

Example: input: [[1, 2, 3], [4, 5, 6]]

// Cumulative sum along dim=0: output: [[1, 2, 3], [5, 7, 9]]

// Cumulative sum along dim=1: output: [[1, 3, 6], [4, 9, 15]]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.multiply (tt::ttnn::MultiplyOp)

Eltwise multiply.

Eltwise multiply operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.neg (tt::ttnn::NegOp)

Eltwise negate.

Eltwise negate operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.ne (tt::ttnn::NotEqualOp)

Eltwise not equal to.

Eltwise not equal to operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.ones (tt::ttnn::OnesOp)

Creates a tensor filled with ones.

Tensor operation to create a tensor filled with ones.

Given a ShapeAttr shape, produces a tensor with the same shape, filled with ones.

Example: %0 = "ttnn.ones"() <{shape = array<i32:64, 28, 28>}> : () -> tensor<64x28x28xbf16> // %0: [[[1, 1, 1, ..., 1], [1, 1, 1, ..., 1], ..., [1, 1, 1, ..., 1]]]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait, TT_CreationOpTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.pad (tt::ttnn::PadOp)

Pad op.

Pad input tensor by padding the input_shape to output_shape using the provided value.

The padding attribute must be a sequence of integers that is twice the size as the rank of the input. Each pair of integers in the padding attribute represents the amount of padding to add to the low and high of that dimension. I.e: an input tensor of shape <1x30x30x64xf32> with padding attribute <0, 0, 1, 1, 1, 1, 0, 0> will return a tensor of shape <1x32x32x64xf32>, and so will a padding attribute of <0, 0, 0, 2, 0, 2, 0, 0>.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.permute (tt::ttnn::PermuteOp)

Permute operation.

Permute input tensor dimensions.

Attributes:

• permutation array: The permutation of the input tensor dimensions.

Example: %a = ttir.empty() : () -> tensor<2x3x4xi32> %0 = "ttir.permute"(%a) {permutation = array<i64: 1, 2, 0>} : (tensor<2x3x4xi32>) -> tensor<3x4x2xi32>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.pow (tt::ttnn::PowOp)

Eltwise power OP.

Performs element-wise exponentiation of lhs tensor by rhs tensor and produces a result tensor. Tensors must be of same shape.

Example:

  %result = "ttnn.pow"(%lhs, %rhs) : (tensor<6xf64>, tensor<6xf64>) -> tensor<6xf64>

  %lhs: [-2.0, -0.0, -36.0, 5.0, 3.0, 10000.0]
  %rhs: [2.0, 2.0, 1.1, 2.0, -1.0, 10.0]
  %result: [4.0, 0.0, -nan, 25.0, 0.333333343, inf]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.prepare_conv2d_weights (tt::ttnn::PrepareConv2dWeightsOp)

Prepares conv2d weights so that they can be consumed by the conv2d op.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.prod (tt::ttnn::ProdOp)

Product reduction op.

This op computes the product of all elements of the tensor (full product) or along a specific dimension.

Example: input: [[1, 2, 3], [4, 5, 6]]

// Computing along dim 0 output: [4, 10, 18]

// Computing along dim 1 output: [6, 120]

// Computing full product output: 720

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.quantize (tt::ttnn::QuantizeOp)

Quantize operation.

Applies quantization to the input tensor.

Inputs:

• input AnyRankedTensor: The input tensor to be quantized.
• scale f32: The scale factor.
• zero_point i32: The zero point value.
• axis Optional: The axis along which quantization is applied.
• output_dtype Optional<TT_DataTypeAttr>: The data type of the output tensor.
• memory_config Optional<TTNN_MemoryConfigAttr>: The memory configuration for the output tensor.

output[i] = (input[i] / scale) + zero_point

Example:

%input = ttir.empty() : () -> tensor<64x128xf32>
%output = ttir.empty() : () -> tensor<64x128x!quant.uniform<i32:f32, 0.1>>
%quantized = "ttir.quantize"(%input, %output) : (tensor<64x128xf32>, tensor<64x128x!quant.uniform<i32:f32, 0.1>>) -> tensor<64x128x!quant.uniform<i32:f32, 0.1>>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.reciprocal (tt::ttnn::ReciprocalOp)

Eltwise reciprocal.

Eltwise reciprocal operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.reduce_scatter (tt::ttnn::ReduceScatterOp)

Reduce scatter op.

Tensor Reduce Scatter operation

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.relu (tt::ttnn::ReluOp)

Eltwise ReLU.

Eltwise ReLU operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.remainder (tt::ttnn::RemainderOp)

Eltwise remainder.

Performs element-wise remainder of dividend lhs and divisor rhs tensors and produces a result tensor.

Example:

// %lhs: [17, -17, 17, -17] // %rhs: [3, 3, -3, -3] %result = "ttnn.remainder"(%lhs, %rhs) : (tensor<4xi64>, tensor<4xi64>) -> tensor<4xi64> // %result: [2, -2, 2, -2]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.repeat_interleave (tt::ttnn::RepeatInterleaveOp)

Repeat interleave op.

Repeats elements of a tensor along a specified dimension. It allows for flexible repetition patterns, where each element can be repeated a different number of times. This is particularly useful for tasks that require duplicating elements in a non-uniform manner.

Parameters:

• input: The input tensor.
• repeats: Specifies the number of repetitions for each element, each element is repeated that number of times.
• dim: The dimension along which to repeat values.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.repeat (tt::ttnn::RepeatOp)

Repeat op.

Returns a new tensor filled with repetition of input tensor according to number of times specified in repeat_dims.

Parameters:

• input_tensor (ttnn.Tensor): the input tensor.
• repeat_dims (number): The number of repetitions for each element.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.requantize (tt::ttnn::RequantizeOp)

Requantize operation.

Applies requantization to the input tensor.

Inputs:

• input AnyRankedTensor: The input tensor to be requantized.
• in_scale f32: The input scale factor.
• in_zero_point i32: The input zero point value.
• out_scale f32: The output scale factor.
• out_zero_point i32: The output zero point value.
• axis Optional: The axis along which quantization is applied.
• output_dtype Optional<TT_DataTypeAttr>: The data type of the output tensor.
• memory_config Optional<TTNN_MemoryConfigAttr>: The memory configuration for the output tensor.

output[i] = round((input[i] - input_zero_point) * (input_scale / output_scale)) + output_zero_point

Example:

%input = ttir.empty() : () -> tensor<64x128x!quant.uniform<i32:f32, 0.1>>
%output = ttir.empty() : () -> tensor<64x128x!quant.uniform<i32:f32, 0.2>>
%requantized = "ttnn.requantize"(%input, %output) : (tensor<64x128x!quant.uniform<i32:f32, 0.1>, tensor<64x128x!quant.uniform<i32:f32, 0.2>>) -> tensor<64x128x!quant.uniform<i32:f32, 0.2>>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.reshape (tt::ttnn::ReshapeOp)

Reshape op.

Reshape tensor.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.rsqrt (tt::ttnn::RsqrtOp)

Eltwise rsqrt.

Eltwise rsqrt operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.scatter (tt::ttnn::ScatterOp)

Scatter op.

Embeds the values of the 'update' tensor into 'input' at the given index and puts the value in the 'output' tensor.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.sigmoid (tt::ttnn::SigmoidOp)

Eltwise sigmoid.

Eltwise sigmoid operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.sign (tt::ttnn::SignOp)

Eltwise sign operation.

Returns the sign of the operand element-wise and produces a result tensor.

Example: %a: [[3, -2, 0], [1, -4, 4]] "ttnn.sign"(%a, %out) -> %out: [[1, -1, 0], [1, -1, 1]]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.sin (tt::ttnn::SinOp)

Eltwise sine.

Eltwise sine operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.slice (tt::ttnn::SliceOp)

Slice op.

Extract a portion of a tensor based on the specified start (begins), stop (ends), and step indices for each dimension.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.softmax (tt::ttnn::SoftmaxOp)

Softmax op.

Softmax operation.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.sqrt (tt::ttnn::SqrtOp)

Eltwise sqrt.

Eltwise sqrt operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.subtract (tt::ttnn::SubtractOp)

Eltwise subtract.

Eltwise subtract operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.sum (tt::ttnn::SumOp)

Sum reduction op.

Sum reduction op.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.tan (tt::ttnn::TanOp)

Eltwise tan op.

Eltwise tan operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.tanh (tt::ttnn::TanhOp)

Eltwise tanh op.

Eltwise tanh operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.to_dtype (tt::ttnn::ToDTypeOp)

ToDType op.

This op converts the data type of the input tensor based on the given data type on the host.

Args:

• :attr:input: the ttnn.Tensor
• :attr:dtype: ttnn data type.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.to_device (tt::ttnn::ToDeviceOp)

ToDevice op.

This op sends the input tensor to the given device with the given memory config.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.to_layout (tt::ttnn::ToLayoutOp)

ToLayout op.

This op wraps all layout information gathered from ttir.toLayout. It is used/updated by the optimizer to perform optimizations, and later broken down into specific memory/layout operations (toDevice, toMemoryConfig etc.). Currently in the TTNN backend, we use this op solely for tilize/untilize, therefore marking all other attrs as optional. Once ttnn::to_layout supports other attrs, we can remove the optional tag.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.to_memory_config (tt::ttnn::ToMemoryConfigOp)

ToMemoryConfig op.

This op converts the memory config of the input tensor based on the given memory config. It handles:

• Dram to L1
• L1 to Dram
• Interleaved to sharded
• Sharded to interleaved
• Sharded to sharded (reshard)

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.transpose (tt::ttnn::TransposeOp)

Transpose op.

Transpose tensor along two given dimensions.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.typecast (tt::ttnn::TypecastOp)

Typecast op.

This op converts the data type of the input tensor based on the given data type. It handles:

• conversions of data types.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), OpModel, TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.update_cache (tt::ttnn::UpdateCacheOp)

Update static cache tensor.

Updates the cache tensor in-place with values from input at update_index and batch_offset.

Interfaces: MemoryEffectOpInterface (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{MemoryEffects::Write on ::mlir::SideEffects::DefaultResource}

Attributes:

Operands:

ttnn.upsample (tt::ttnn::UpsampleOp)

Upsample 2D op.

Upsample 2D operation. Input tensor is assumed to be in NHWC format.

Attributes:

• scale_factor (si32 | array): The scale factor for upsampling in H and W dimensions respectively.
• mode (str): The upsampling algorithm. Currently only "nearest" and "bilinear" are supported. Default is "nearest".

Example: // %a: tensor<10x64x32xbf16> %0 = "ttnn.upsample"(%a) <{scale_factor = array<i32: 2, 4>}> : (tensor<10x64x32x3xbf16>) -> tensor<10x128x128x3xbf16>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.where (tt::ttnn::WhereOp)

Eltwise where.

Eltwise where operation.

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttnn.zeros (tt::ttnn::ZerosOp)

Creates a tensor filled with zeros.

Tensor operation to create a tensor filled with zeros.

Given a ShapeAttr shape, produces a tensor with the same shape, filled with zeros.

Example: %0 = "ttnn.zeros"() <{shape = array<i32:64, 28, 28>}> : () -> tensor<64x28x28xbf16> // %0: [[[0, 0, 0, ..., 0], [0, 0, 0, ..., 0], ..., [0, 0, 0, ..., 0]]]

Traits: AlwaysSpeculatableImplTrait, HasMemoryConfigTrait, TT_CreationOpTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), TTNN_WorkaroundInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results: