tt-mlir documentation

ttir.abs (tt::ttir::AbsOp)

Eltwise absolute op.

Eltwise absolute operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TTIR_Idempotence, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.add (tt::ttir::AddOp)

Eltwise add.

Eltwise add operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable, TTIR_GenericRegionOpInterface

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.all_gather (tt::ttir::AllGatherOp)

All gather operation.

All gather op.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.all_reduce (tt::ttir::AllReduceOp)

AllReduce operation.

AllReduce op.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.alloc (tt::ttir::AllocOp)

Alloc op.

Tensor Alloc operation

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Attributes:

Results:

ttir.arange (tt::ttir::ArangeOp)

Arange operation.

Tensor arange operation.

Produces a tensor with values from start to end (exclusive) with a step size of step, along the dimension specified by arange_dimension.

Examples: %0 = "ttir.arange"() {start = 0 : i64, end = 5 : i64 step = 1 : i64, arange_dimension = 0 : i64} : () -> tensor<5xi64> // %0: [0, 1, 2, 3, 4]

%1 = "ttir.arange"() {start = 0 : i64, end = 10 : i64, step = 2 : i64, arange_dimension = 0 : i64} : () -> tensor<5xf32> // %1: [0.0, 2.0, 4.0, 6.0, 8.0]

%2 = "ttir.arange"() {start = 0 : i64, end = 5 : i64, step = 1 : i64, arange_dimension = 0 : i64} : () -> tensor<5x3xi64> // %2: [ [0, 0, 0], [1, 1, 1], [2, 2, 2], [3, 3, 3], [4, 4, 4] ]

%3 = "ttir.arange"() {start = 0 : i64, end = 3 : i64, step = 1 : i64, arange_dimension = 1 : i64} : () -> tensor<5x3xi64> // %3: [ [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2] ]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Attributes:

Results:

ttir.bitwise_and (tt::ttir::BitwiseAndOp)

Eltwise bitwise AND.

Performs element-wise bitwise AND of two tensors lhs and rhs and produces a result tensor.

Example: // %lhs: [[1, 2], [3, 4]] // %rhs: [[5, 6], [7, 8]] %result = "ttir.bitwise_and"(%lhs, %rhs) : (tensor<2x2xi32>, tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[1, 2], [3, 0]]

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TTIR_BinaryIdempotence, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.bitwise_not (tt::ttir::BitwiseNotOp)

Eltwise bitwise NOT.

Performs element-wise NOT of tensor operand and produces a result tensor.

Example: // Bitwise operation with with integer tensors // %operand: [[1, 2], [3, 4]] %result = "ttir.bitwise_not"(%operand) : (tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[-2, -3], [-4, -5]]

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TTIR_Involution, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.bitwise_or (tt::ttir::BitwiseOrOp)

Eltwise bitwise OR.

Performs element-wise bitwise OR of two tensors lhs and rhs and produces a result tensor.

Example: // %lhs: [[1, 2], [3, 4]] // %rhs: [[5, 6], [7, 8]] %result = "ttir.bitwise_or"(%lhs, %rhs) : (tensor<2x2xi32>, tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[5, 6], [7, 12]]

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TTIR_BinaryIdempotence, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.bitwise_xor (tt::ttir::BitwiseXorOp)

Eltwise bitwise XOR.

Performs element-wise bitwise XOR of two tensors lhs and rhs and produces a result tensor.

Example: // %lhs: [[1, 2], [3, 4]] // %rhs: [[5, 6], [7, 8]] %result = "ttir.bitwise_xor"(%lhs, %rhs) : (tensor<2x2xi32>, tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[4, 4], [4, 12]]

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.broadcast (tt::ttir::BroadcastOp)

Broadcast operation.

Broadcasts each dimension of the input tensor based on the broadcast dimensions.

Example: // %arg0: tensor<1x1x32xf32> %0 = tensor.empty() : tensor<1x16x32xf32> %1 = "ttir.broadcast"(%arg0, %0) <{broadcast_dimensions = array<i32: 1, 16, 1>}> : (tensor<1x1x32xf32>, tensor<1x16x32xf32>) -> tensor<1x16x32xf32>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.cbrt (tt::ttir::CbrtOp)

Eltwise cubic root op.

Eltwise cubic root operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.ceil (tt::ttir::CeilOp)

Eltwise ceil op.

Eltwise ceil operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TTIR_Idempotence, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.clamp (tt::ttir::ClampOp)

Clamp op.

Clamp tensor values to a specified range.

Example: min: 2.000000+00 input: [[0, 1, 2, 3, 4, 5, 6, 7]] max: 5.000000+00

"ttir.clamp"(%arg0) <{max = 2.000000e+00 : f32, min = 5.000000e+00 : f32}> -> %out = [[2, 2, 2, 3, 4, 5, 5, 5]]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.concat (tt::ttir::ConcatOp)

Concat op.

Concat tensors along a given dimension.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.constant (tt::ttir::ConstantOp)

Constant op.

Produces tensor filled with given constant value.

Examples: %0 = "ttir.constant"() {value = dense<0> : tensor<2x3xi32>} : () -> tensor<2x3xi32> // %0: [[0, 0, 0], [0, 0, 0]] %1 = "ttir.constant"() {value = dense<[0.2, 1.3]> : tensor<2xf32>} : () -> tensor<2xf32> // %1: [0.2, 1.3]

Traits: AlwaysSpeculatableImplTrait, ConstantLike

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Attributes:

Results:

ttir.conv2d (tt::ttir::Conv2dOp)

Conv2d operation.

Applies a 2D convolution over an input image composed of several input planes.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.conv_transpose2d (tt::ttir::ConvTranspose2dOp)

ConvTranspose2d operation.

Applies a 2D transposed convolution operator over an input image composed of several input planes.

Inputs:

• input AnyRankedTensor: NHWC format (batch_size x height x width x channels)
• weight AnyRankedTensor: OIHW format (output_channels x input_channels x height x width)
• bias Optional: (1 x 1 x 1 x output_channels)
• output AnyRankedTensor: NHWC format (batch_size x height x width x channels)

Attributes:

• stride (i32 | array): Controls the stride for the cross-correlation.
• padding (i32 | array): Controls the amount of implicit zero padding on both sides for dilation * (kernel_size - 1) - padding number of points.
• output_padding (i32 | array): Controls the additional size added to one side of the output shape.
• dilation (i32 | array): Controls the spacing between the kernel points
• groups i32: Controls the connections between inputs and outputs. Must be divisible by input and output channels.

Example: %input = tensor.empty() : () -> tensor<256x256x3x3xbf16> %weight = tensor.empty() : () -> tensor<256x256x3x3xbf16> %bias = tensor.empty() : () -> tensor<1x1x1x256xbf16> %output = tensor.empty() : () -> tensor<1x10x10x256xbf16> %0 = "ttir.conv_transpose2d"(%input, %weight, %bias, %output) <{ stride = = array<i32: 1, 1>, padding = 0: i32, output_padding = 0: i32, dilation = 1: i32, groups = 1: i32 > : (tensor<1x8x8x256xbf16>, tensor<256x256x3x3xbf16>, tensor<1x1x1x256xbf16>, tensor<1x10x10x256xbf16>) -> tensor<1x10x10x256xbf16>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.convolution (tt::ttir::ConvolutionOp)

Generalized convolution op.

Applies a convolution of the rhs with the lhs.

This operation captures convolutions of all dimensionality as well as deconvolution/conv transpose.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.cos (tt::ttir::CosOp)

Eltwise cosine op.

Eltwise cosine operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.dealloc (tt::ttir::DeallocOp)

Dealloc op.

Tensor Dealloc operation

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Operands:

ttir.div (tt::ttir::DivOp)

Eltwise divide.

Eltwise divide operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable, TTIR_GenericRegionOpInterface

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.dot_general (tt::ttir::DotGeneralOp)

Dot general operation.

Flexible tensor operation that generalizes matrix multiplication by allowing user to specify which dimensions of two tensors to contract. Matrix multiplication is a special case of this operation, where the contraction happens along the last axis of the first tensor and the second-to-last axis of the second tensor. From StableHLO DotGeneral Op https://openxla.org/stablehlo/spec#dot_general

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.embedding_backward (tt::ttir::EmbeddingBackwardOp)

Embedding backward op.

Embedding backward operation. Generates the gradient of the embedding operation with respect to the input.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.embedding (tt::ttir::EmbeddingOp)

Embedding op.

Embedding operation.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.eq (tt::ttir::EqualOp)

Eltwise equal to.

Eltwise equal to operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.exp (tt::ttir::ExpOp)

Eltwise exponential op.

Eltwise exponential operation. Calculates e^x for all elements x in input tensor.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable, TTIR_GenericRegionOpInterface

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.expm1 (tt::ttir::Expm1Op)

Eltwise unary op.

Performs element-wise exponential minus one operation on operand tensor and stores the result in the output tensor.

Example: %a: [[0, 1], [0, 0]] "ttir.exmp1"(%a, %out) -> %out: [[0, 1.71828], [0, 0]]

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.fill_cache (tt::ttir::FillCacheOp)

Fill static cache tensor.

Fills the cache tensor in-place with values from input at batch_offset.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.fill (tt::ttir::FillOp)

Fill operation.

Produces tensor filled with given fill value.

Examples: %0 = tensor.empty() : () -> tensor<2x3xi32> %1 = "ttir.fill"(%0) {value = dense<0> : tensor<2x3xi32>} : () -> tensor<2x3xi32> %2 = tensor.empty() : () -> tensor<2xf32> %3 = "ttir.fill"(%2) {value = dense<[0.2, 1.3]> : tensor<2xf32>} : () -> tensor<2xf32>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.floor (tt::ttir::FloorOp)

Eltwise floor op.

Eltwise floor operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TTIR_Idempotence, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.gather (tt::ttir::GatherOp)

Gather operation.

Gathers slices from operand tensor from offsets specified in start_indices and produces a result tensor. From StableHLO Gather Op: https://openxla.org/stablehlo/spec#gather

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.gelu (tt::ttir::GeluOp)

Eltwise GELU op.

Eltwise GELU operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.generic (tt::ttir::GenericOp)

Generically dispatch work to a grid of cores.

This generic op carries a region that represents the work each core does. The region is expected to have the same signature as the op itself with respect to input and output operands. The op is expected to be lowered to a backend specific form by a consuming backend. This op is heavily inspired by the linalg.generic op so it can be useful to refer to linalg.generic documentation for more details.

%5 = "ttir.generic"(%1, %2, %3, %4) <{
  grid = #tt.grid<1x1>,                        // The grid range of cores to dispatch work to.
  indexing_maps = [#map, #map, #map],          // Affine maps for indexing into the input/output tensors. See linalg.generic
  iterator_types = [#parallel, #parallel],     // Iterator types for the input/output tensors. See linalg.generic
  operandSegmentSizes = array<i32: 2, 1, 1>,   // Sizes of the operand segments, i.e. 2 inputs, 1 cb and 1 output.
  operand_cb_mapping = array<i64: -1, 0, -1>,  // Mapping of input & output operands to cbs. -1 means no mapping.
                                               // Mapped operands correspond to buffers in streaming mode.
                                               // Non-mapped operands correspond to buffers in alias mode.
({
^bb0(%arg2: tensor<64x128xf32, #tt.buffer<memref<64x128xf32, #l1_>, alias>>,
     %arg3: tensor<64x128xf32, #tt.buffer<memref<64x128xf32, #l1_>, stream>>,
     %arg4: tensor<64x128xf32, #tt.buffer<memref<64x128xf32, #l1_>, alias>>):
    // Region body, would contain some computation that represents the work each core does.
}) : (tensor<64x128xf32, #layout1>, tensor<64x128xf32, #layout1>, tensor<64x128xf32, #layout1>, tensor<64x128xf32, #layout1>) -> tensor<64x128xf32, #layout1>

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.get_dimension_size (tt::ttir::GetDimensionSizeOp)

GetDimensionSize op.

Produces the size of the given dimension of the operand.

Example: %operand: [[3, 2, 7], [1, 4, 4]] "ttir.get_dimension_size"(%operand, value = dense<0>, %out) -> %out: [[3]]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.ge (tt::ttir::GreaterEqualOp)

Eltwise greater than or equal to.

Eltwise greater than or equal to operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.gt (tt::ttir::GreaterThanOp)

Eltwise greater than.

Eltwise greater than operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.index (tt::ttir::IndexOp)

Index op.

Extract a sub-tensor (slice) from the input tensor along a specified dimension. The begin, end, and step attributes define the start, stop, and step indices for the selected dimension (dim) of the tensor.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.isfinite (tt::ttir::IsFiniteOp)

Eltwise isfinite op.

Eltwise isfinite operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.kernel (tt::ttir::KernelOp)

Kernel call.

A generic kernel call operation. This operation is used to pattern match by some consuming backend.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.leaky_relu (tt::ttir::LeakyReluOp)

Eltwise leaky relu operation.

The Leaky ReLU (Rectified Linear Unit) operation computes an element-wise activation function over its input tensor. It is defined as:

y = x if x > 0 y = parameter * x if x <= 0

where parameter is a small, user-defined constant that determines the slope for negative inputs.

Attributes:

• parameter (float): The slope for negative values.

Inputs:

• input (Tensor): The input tensor to be activated.

Outputs:

• output (Tensor): The tensor after applying the Leaky ReLU activation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.le (tt::ttir::LessEqualOp)

Eltwise less than or equal to.

Eltwise less than or equal to operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.lt (tt::ttir::LessThanOp)

Eltwise less than.

Eltwise less than operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.linear (tt::ttir::LinearOp)

Linear transformation of inputs.

Produces the matmul of tensors a and b with optional addition with bias.

Example: %a = tensor.empty() : () -> tensor<10x64x32xbf16> %b = tensor.empty() : () -> tensor<32x128xbf16> %bias = tensor.empty() : () -> tensor<128xbf16> %output = tensor.empty() : () -> tensor<10x64x128xbf16> %0 = "ttir.linear"(%a, %b, %bias, %output) : (tensor<10x64x32xbf16>, tensor<32x128xbf16>, tensor<128xbf16>, tensor<10x64x128xbf16>) -> tensor<10x64x128xbf16>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.log1p (tt::ttir::Log1pOp)

Eltwise log1p operation.

Performs element-wise logarithm plus one operation on operand tensor and puts the result in the output tensor.

Example: %a: [0.0, -0.999, 7.0, 6.38905621, 15.0] "ttir.logp1"(%a, %out) -> %out: [0.0, -6.90776825, 2.07944155, 2.0, 2.77258873]

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.log (tt::ttir::LogOp)

Eltwise logarithm op.

Eltwise logarithm operation. Calculates log(x) for all elements x in input tensor.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.logical_and (tt::ttir::LogicalAndOp)

Eltwise logical and.

Eltwise logical and operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TTIR_BinaryIdempotence, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.logical_not (tt::ttir::LogicalNotOp)

Eltwise logical not op.

Eltwise logical not operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.logical_or (tt::ttir::LogicalOrOp)

Eltwise logical or.

Eltwise logical or operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TTIR_BinaryIdempotence, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.logical_xor (tt::ttir::LogicalXorOp)

Eltwise logical xor.

Eltwise logical xor operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.matmul (tt::ttir::MatmulOp)

Matrix multiply operation.

Matrix multiply operation.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.max (tt::ttir::MaxOp)

Max reduction op.

Max reduction op.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_GenericRegionOpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.max_pool2d (tt::ttir::MaxPool2dOp)

Applies a 2D max pooling over an input signal composed of several input planes.

Applies a 2D max pooling over an input signal composed of several input planes.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.maximum (tt::ttir::MaximumOp)

Eltwise maximum.

Calculates maximum of input tensors' values element-wise and stores result in output tensor.

Example: %lhs: [[3, 2, 7], [1, 4, 4]] %rhs: [[1, 4, 2], [1, 2, 3]] "ttir.maximum"(%lhs, %rhs, %out) -> %out: [[3, 4, 7], [1, 4, 4]]

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable, TTIR_GenericRegionOpInterface

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.mean (tt::ttir::MeanOp)

Mean reduction op.

Mean reduction op.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_GenericRegionOpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.mesh_shard (tt::ttir::MeshShardOp)

Mesh shard operation.

MeshShard op shards the inputs (FullToShard) or concatnates the outputs (ShardToFull) for ccl ops.

shard_direction attribute determines whether to shard or concat.

shard_type attribute determines how to shard or concat. manual: no sharding replicate: all devices have identical data maximal: only one device contains full data devices: shard_shape determines sharded dimensions

For example, on 2x4 mesh hardware, following op shards arg0 to 8 slices, row divided by 2 and col divided by 4.

%1 = "ttir.mesh_shard"(%arg0, %0) < {... shard_direction = #tt.shard_direction<full_to_shard>, shard_shape = #tt.grid<2x4>, shard_type = #tt.shard_type}> : (tensor<8192x784xf32>, ...) -> tensor<4096x196xf32>

On the other hand, this op concatnates %4 to single tensor by concatnating one of the top row tensor with one of the bottom row tensor.

%6 = "ttir.mesh_shard"(%4, %5) < {..., shard_direction = #tt.shard_direction<shard_to_full>, shard_shape = #tt.grid<2x1>, shard_type = #tt.shard_type}> : (tensor<4096x16384xf32>, ...) -> tensor<8192x16384xf32>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.minimum (tt::ttir::MinimumOp)

Eltwise minimum OP.

Calculates minimum of input tensors' values element-wise and stores result in output tensor.

Example: %lhs: [[3, 2, 7], [1, 4, 4]] %rhs: [[1, 4, 2], [1, 2, 3]] "ttir.minimum"(%lhs, %rhs, %out) -> %out: [[1, 2, 2], [1, 2, 3]]

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TTIR_BinaryIdempotence, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.multiply (tt::ttir::MultiplyOp)

Eltwise multiply.

Eltwise multiply operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable, TTIR_GenericRegionOpInterface

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.neg (tt::ttir::NegOp)

Eltwise negate op.

Eltwise negate operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TTIR_Involution, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.ne (tt::ttir::NotEqualOp)

Eltwise not equal to.

Eltwise not equal to operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.ones (tt::ttir::OnesOp)

Creates a tensor filled with ones.

Tensor operation to create a tensor filled with ones.

Given a shape, produces a tensor with the shape, filled with ones.

Example: %0 = "ttir.ones"() <{shape = array<i32:64, 28, 28>}> : () -> tensor<64x28x28xbf16> // %0: [[[1, 1, 1, ..., 1], [1, 1, 1, ..., 1], ..., [1, 1, 1, ..., 1]]]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface)

Effects: MemoryEffects::Effect{}

Attributes:

Results:

ttir.permute (tt::ttir::PermuteOp)

Permute operation.

Permute input tensor dimensions.

Attributes:

• permutation array: The permutation of the input tensor dimensions.

Example: %a = tensor.empty() : () -> tensor<2x3x4xi32> %output = tensor.empty() : () -> tensor<3x4x2xi32> %0 = "ttir.permute"(%a, %output) {permutation = array<i64: 1, 2, 0>} : (tensor<2x3x4xi32>, tensor<3x4x2xi32>) -> tensor<3x4x2xi32>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.pooling (tt::ttir::PoolingOp)

General pooling op

General pooling op

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.reciprocal (tt::ttir::ReciprocalOp)

Eltwise reciprocal.

Eltwise reciprocal operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TTIR_Involution, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.relu (tt::ttir::ReluOp)

Eltwise ReLU.

Eltwise ReLU operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TTIR_Idempotence, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.remainder (tt::ttir::RemainderOp)

Eltwise remainder.

Performs element-wise remainder of dividend lhs and divisor rhs tensors and produces a result tensor.

Example:

// %lhs: [17, -17, 17, -17] // %rhs: [3, 3, -3, -3] %result = "ttir.remainder"(%lhs, %rhs) : (tensor<4xi64>, tensor<4xi64>) -> tensor<4xi64> // %result: [2, -2, 2, -2]

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.reshape (tt::ttir::ReshapeOp)

Reshape op.

Reshape tensor.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.reverse (tt::ttir::ReverseOp)

Reverse operation.

Reverses the order of elements in the operand along the specified dimensions and produces a result tensor.

Examples: // %operand = [[1, 2], [3, 4], [5, 6]] %result = "ttir.reverse"(%operand) { dimensions = array<i64: 1> } : (tensor<3x2xi32>) -> tensor<3x2xi32> // %result: [[2, 1], [4, 3], [6, 5]]

// %operand = [[1, 2], [3, 4], [5, 6]] %result = "ttir.reverse"(%operand) { dimensions = array<i64: 1, 0> } : (tensor<3x2xi64>) -> tensor<3x2xi64> // %result: [[6, 5], [4, 3], [2, 1]]

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.rsqrt (tt::ttir::RsqrtOp)

Eltwise reciprocal square root.

Eltwise reciprocal square root operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.scatter (tt::ttir::ScatterOp)

Scatter operation

Produces a 'result' tensor which are equal to input tensor except that several slices specified by scatter_indices are updated with the values updates.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.select (tt::ttir::SelectOp)

Select op.

Extracts a sub-tensor (slice) from the input tensor along a specified dimension in few steps defined by the begin, length, and stride attributes. The begin specifies the start index for the selected dimension of the tensor. The length specifies the number of elements to extract from the input tensor along the selected dimension. The stride specifies the step size for the start index. The default value is 0. 0 means no stride.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.sigmoid (tt::ttir::SigmoidOp)

Eltwise sigmoid.

Eltwise sigmoid operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.sign (tt::ttir::SignOp)

Eltwise sign operation.

Returns the sign of the operand element-wise and produces a result tensor.

Example: %a: [[3, -2, 0], [1, -4, 4]] "ttir.sign"(%a, %out) -> %out: [[1, -1, 0], [1, -1, 1]]

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TTIR_Idempotence, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.sin (tt::ttir::SinOp)

Eltwise sine.

Eltwise sine operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.slice (tt::ttir::SliceOp)

Slice op.

Extract a sub-tensor (slice) from the input tensor across one or more dimensions. The begins, ends, and step attributes specify the start, stop, and step indices for each dimension of the tensor.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.softmax (tt::ttir::SoftmaxOp)

Softmax operation.

Softmax operation.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.sqrt (tt::ttir::SqrtOp)

Eltwise square root.

Eltwise square root operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.squeeze (tt::ttir::SqueezeOp)

Squeeze op.

Squeeze tensor.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.subtract (tt::ttir::SubtractOp)

Eltwise subtract.

Eltwise subtract operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, ThreeOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.sum (tt::ttir::SumOp)

Sum reduction op.

Sum reduction op.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_GenericRegionOpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.tan (tt::ttir::TanOp)

Eltwise tan op.

Eltwise tan operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.tanh (tt::ttir::TanhOp)

Eltwise tanh op.

Eltwise tanh operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.to_layout (tt::ttir::ToLayoutOp)

Layout op.

ToLayout operation, transition tensors from one layout to another. Some examples include:

• Transitioning between different memory spaces, e.g. DRAM to L1.
• Transitioning between different data types, e.g. f32 to f16.
• Transitioning between different tile sizes, e.g. 1x16 to 32x32
• Transitioning between different tensor sharding
• Some combination of the above

#layout = #tt.metal_layout<8192x128x1, undef, <1x1>, memref<64x128xf32, #system>>
#layout1 = #tt.metal_layout<8192x128x1, undef, <1x1>, memref<64x128xf32, #l1_>>
%1 = "ttir.to_layout"(%arg0, %0) : (tensor<64x128xf32, #layout>, tensor<64x128xf32, #layout1>) -> tensor<64x128xf32, #layout1>

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.transpose (tt::ttir::TransposeOp)

Transpose op.

Transpose tensor along two given dimensions.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.typecast (tt::ttir::TypecastOp)

Eltwise cast op.

Eltwise cast operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, TwoOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.unsqueeze (tt::ttir::UnsqueezeOp)

Unsqueeze op.

Unsqueeze tensor.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.update_cache (tt::ttir::UpdateCacheOp)

Update static cache tensor.

Updates the cache tensor in-place with values from input at update_index and batch_offset.

Traits: AlwaysSpeculatableImplTrait

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface

Effects: MemoryEffects::Effect{}

Attributes:

Operands:

Results:

ttir.where (tt::ttir::WhereOp)

Eltwise where op.

Eltwise where operation.

Traits: AlwaysSpeculatableImplTrait, AttrSizedOperandSegments, FourOperands

Interfaces: ConditionallySpeculatable, DestinationStyleOpInterface, NoMemoryEffect (MemoryEffectOpInterface), TTIROpInterface, TTIR_Broadcastable

Effects: MemoryEffects::Effect{}

Operands:

Results:

ttir.yield (tt::ttir::YieldOp)

Yield op.

Yield operation, this is required by MLIR to mark the end of a dispatch region.

Traits: AlwaysSpeculatableImplTrait, ReturnLike, Terminator

Interfaces: ConditionallySpeculatable, NoMemoryEffect (MemoryEffectOpInterface), RegionBranchTerminatorOpInterface

Effects: MemoryEffects::Effect{}

Operands: