ttnn.abs
(tt::ttnn::AbsOp)
Eltwise absolute.
Eltwise absolute operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.add
(tt::ttnn::AddOp)
Eltwise add.
Eltwise add operation.
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, OpModel
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.all_gather
(tt::ttnn::AllGatherOp)
All gather op.
Tensor All Gather operation
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
all_gather_dim | ::mlir::IntegerAttr | 32-bit signed integer attribute |
cluster_axis | ::mlir::IntegerAttr | 32-bit unsigned integer attribute |
num_links | ::mlir::IntegerAttr | 32-bit unsigned integer attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
device | TTNN device |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.all_reduce
(tt::ttnn::AllReduceOp)
All reduce op.
Tensor All Reduce operation
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
reduce_type | ::mlir::tt::ReduceTypeAttr | TT Reduce Type{{% markdown %}}Enum cases: * sum (`Sum`) * mean (`Mean`) * max (`Max`) * min (`Min`) * std (`Std`) * var (`Var`){{% /markdown %}} |
cluster_axis | ::mlir::IntegerAttr | 32-bit unsigned integer attribute |
num_links | ::mlir::IntegerAttr | 32-bit unsigned integer attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
device | TTNN device |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.alloc
(tt::ttnn::AllocOp)
Alloc op.
Tensor Alloc operation
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
address | ::mlir::IntegerAttr | 64-bit signless integer attribute |
size | ::mlir::IntegerAttr | 64-bit signless integer attribute |
buffer_type | ::mlir::tt::ttnn::BufferTypeAttr | TTNN Buffer Type{{% markdown %}}Enum cases: * dram (`DRAM`) * l1 (`L1`) * system_memory (`SystemMemory`) * l1_small (`L1Small`) * trace (`Trace`){{% /markdown %}} |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.arange
(tt::ttnn::ArangeOp)
Arange operation.
Tensor arange operation.
Produces a (1, 1, 1, N)-shaped tensor with values from start
to end
(exclusive) with a step size of step
.
Examples: %0 = "ttnn.arange"() {start = 0 : i64, end = 5 : i64 step = 1 : i64} : () -> tensor<1x1x1x5xi64> // %0: [[[[0, 1, 2, 3, 4]]]]
%1 = "ttnn.arange"() {start = 0 : i64, end = 10 : i64, step = 2 : i64} : () -> tensor<1x1x1x5xf32> // %1: [[[[0.0, 2.0, 4.0, 6.0, 8.0]]]]
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
start | ::mlir::IntegerAttr | 64-bit signless integer attribute |
end | ::mlir::IntegerAttr | 64-bit signless integer attribute |
step | ::mlir::IntegerAttr | 64-bit signless integer attribute |
dtype | ::mlir::tt::DataTypeAttr | TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}} |
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
device | TTNN device |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.argmax
(tt::ttnn::ArgMaxOp)
Argmax reduction op.
Determine the indices of the maximum values along a specified dimension of a tensor or over all elements in a tensor.
Parameters:
input
: The input tensor.dim
: Specifies the dimension along which the argmax is applied.use_multicore
: Whether to use multiple cores or not.
IR usage: // Input tensor of shape (128, 28, 28, 64) %input = ... : tensor<128x28x28x64xbf16>
%empty = "ttnn.empty"(%0) <{dtype = #tt.supportedDataTypes
Example: input: [[1, 5, 3], [2, 4, 6]]
// Computing along dim 0 output: [1, 0, 1]
// Computing along dim 1 output: [1, 2]
// Computing for entire tensor output: 5
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
dim | ::mlir::IntegerAttr | 32-bit signless integer attribute |
use_multicore | ::mlir::BoolAttr | bool attribute |
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.atan
(tt::ttnn::AtanOp)
Eltwise arctangent op.
Performs an elementwise arctangent (atan
) operation on the input tensor.
This operation computes the inverse tangent of each element, returning
values in the range [-π/2, π/2]. Supports floating-point tensor types.
Example:
%input = tensor<4xf32> {1.0, 0.5, 0.0, -1.0}
%result = "ttir.atan"(%input) : (tensor<4xf32>) -> tensor<4xf32>
Given the input [1.0, 0.5, 0.0, -1.0]
, the result would be approximately:
[0.785, 0.464, 0.0, -0.785]
(values in radians).
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.bitwise_and
(tt::ttnn::BitwiseAndOp)
Eltwise bitwise AND.
Performs element-wise bitwise AND of two tensors lhs
and rhs
and produces a result
tensor.
Example: // %lhs: [[1, 2], [3, 4]] // %rhs: [[5, 6], [7, 8]] %result = "ttnn.bitwise_and"(%lhs, %rhs) : (tensor<2x2xi32>, tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[1, 2], [3, 0]]
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.bitwise_not
(tt::ttnn::BitwiseNotOp)
Eltwise bitwise NOT.
Performs element-wise NOT of tensor operand
and produces a result
tensor.
Example: // Bitwise operation with with integer tensors // %operand: [[1, 2], [3, 4]] %result = "ttnn.bitwise_not"(%operand) : (tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[-2, -3], [-4, -5]]
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.bitwise_or
(tt::ttnn::BitwiseOrOp)
Eltwise bitwise OR.
Performs element-wise bitwise OR of two tensors lhs
and rhs
and produces a result
tensor.
Example: // %lhs: [[1, 2], [3, 4]] // %rhs: [[5, 6], [7, 8]] %result = "ttnn.bitwise_or"(%lhs, %rhs) : (tensor<2x2xi32>, tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[5, 6], [7, 12]]
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.bitwise_xor
(tt::ttnn::BitwiseXorOp)
Eltwise bitwise XOR.
Performs element-wise bitwise XOR of two tensors lhs
and rhs
and produces a result
tensor.
Example: // %lhs: [[1, 2], [3, 4]] // %rhs: [[5, 6], [7, 8]] %result = "ttnn.bitwise_xor"(%lhs, %rhs) : (tensor<2x2xi32>, tensor<2x2xi32>) -> tensor<2x2xi32> // %result: [[4, 4], [4, 12]]
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.cbrt
(tt::ttnn::CbrtOp)
Eltwise cubic root.
Eltwise cubic root operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.ceil
(tt::ttnn::CeilOp)
Eltwise ceil.
Eltwise ceil operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.clamp_scalar
(tt::ttnn::ClampScalarOp)
Clamp op.
Clamp tensor values to a specified range.
Example: min: 2.000000+00 input: [[0, 1, 2, 3, 4, 5, 6, 7]] max: 5.000000+00
"ttnn.clamp_scalar"(%arg0) <{max = 2.000000e+00 : f32, min = 5.000000e+00 : f32}> -> %out = [[2, 2, 2, 3, 4, 5, 5, 5]]
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
min | ::mlir::FloatAttr | 32-bit float attribute |
max | ::mlir::FloatAttr | 32-bit float attribute |
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | variadic of ranked tensor of any type values |
ttnn.clamp_tensor
(tt::ttnn::ClampTensorOp)
Clamp op.
Clamp tensor values to a specified range using min/max as tensor.
Example: min: [[2, 2, 2, 3, 3, 3, 0, 0]] input: [[0, 1, 2, 3, 4, 5, 6, 7]] max: [[5, 5, 5, 9, 9, 9, 6, 6]]
"ttnn.clamp_tensor"(%input, %min, %max) %out: [[2, 2, 2, 3, 4, 5, 6, 6]]
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
min | ranked tensor of any type values |
max | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | variadic of ranked tensor of any type values |
ttnn.collective_permute
(tt::ttnn::CollectivePermuteOp)
Collective permute op.
Collective permute op. This operation ingests a multi-device tensor spread across multi-devices and will shuffle the data according to source_target_pairs [['src', 'dest']].
Example: For a 1x2 mesh, the following will take the device shard living in device 0 and move it to device 1. The device shard living in device 1 will move to device 0. %source_target_pairs: [[0, 1], [1, 0]]
In the case of missing 'dest', the device shard living on that device will contain values of 0. For example, device shard living in device 0 will contain 0 values. %source_target_pairs: [[0, 1]]
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
source_target_pairs | ::mlir::DenseIntElementsAttr | 64-bit signless integer elements attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
device | TTNN device |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.concat
(tt::ttnn::ConcatOp)
Concat op.
Concat tensors along a given dimension.
Traits: AlwaysSpeculatableImplTrait
, HasMemoryConfigTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
dim | ::mlir::IntegerAttr | 32-bit signed integer attribute |
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.constant
(tt::ttnn::ConstantOp)
Constant op.
Produces tensor filled with given constant value.
Examples: %0 = "ttnn.constant"() {value = dense<[[3, 4, 2], [1, 7, 8]]> : tensor<2x3xui16>} : () -> tensor<2x3xui16> // %0: [[3, 4, 2], [1, 7, 8]] %1 = "ttnn.constant"() {value = dense<[0.2, 1.3]> : tensor<2xf32>} : () -> tensor<2xf32> // %1: [0.2, 1.3]
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
value | ::mlir::ElementsAttr | constant vector/tensor attribute |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.construct_tensor
(tt::ttnn::ConstructTensorOp)
Construct empty tensor on host op.
Invoke tensor ctor to create an empty tensor on host.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
shape | ::mlir::tt::ttnn::ShapeAttr | TTNN Shape attribute{{% markdown %}} TTNN shape attribute {{% /markdown %}} |
dtype | ::mlir::tt::DataTypeAttr | TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}} |
layout | ::mlir::tt::ttnn::LayoutAttr | TTNN Layout{{% markdown %}}Enum cases: * row_major (`RowMajor`) * tile (`Tile`) * invalid (`Invalid`){{% /markdown %}} |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.conv2d
(tt::ttnn::Conv2dOp)
Conv2d operation.
Applies a 2D convolution over an input image composed of several input planes.
Inputs:
input
(AnyRankedTensor): expected in the following flattened format (1, 1, N * H_in * W_in, C) where:- N is the batch size
- H_in is the height of the input planes
- W_in is the width of the input planes
- C is the number of channels
weight
(AnyRankedTensor): expected in the following format (O, C/G, K_H, K_W).bias
(Optional): expected in the following format (1, 1, 1, O) where: - C is the number of input channels
- O is the number of output channels
- G is the number of groups
- K_H is the height of the kernel
- K_W is the width of the kernel
Attributes:
in_channels
(i32): The number of input channels.out_channels
(i32): The number of output channels.batch_size
(i32): The batch size.input_height
(i32): The input height.input_width
(i32): The input width.kernel_size
(array<2xi32>): [K_H, K_W] where K_H is the kernel height and K_W is the kernel width.stride
(array<2xi32>): [sH, sW] where sH is stride for height and sW is stride for width.padding
(array<2xi32>): [pH, pW] where pH is padding for height (top/bottom) and pW is padding for width (left/right).dilation
(array<2xi32>): [dH, dW] where dH is dilation for height and dW is dilation for width.groups
(i32): Number of blocked connections from input channels to output channels. Input and output channels must both be divisible by groups.
Outputs:
result
(AnyRankedTensor): returned in the following flattened format (1, 1, N * H_out * W_out, O) where:H_out = (H_in + 2 * pH - dH * (K_H - 1) - 1) / sH + 1
W_out = (W_in + 2 * pW - dW * (K_W - 1) - 1) / sW + 1
Example: %input = ttir.empty() : () -> tensor<1x1x1024x64xbf16> %weight = ttir.empty() : () -> tensor<64x64x3x3xbf16> %bias = ttir.empty() : () -> tensor<1x1x1x64xbf16> %device = "ttnn.get_device"() <{mesh_shape = #ttnn<mesh_shape 1x1>}> : () -> !ttnn.device %0 = "ttnn.conv2d"(%input, %weight, %bias, %device) <{ in_channels = 64: i32, out_channels = 64: i32, batch_size = 1: i32, input_height = 32: i32, input_width = 32: i32, kernel_size = array<i32: 3, 3>, stride = array<i32: 1, 1>, padding = array<i32: 0, 0>, dilation = array<i32: 1, 1>, groups = 1: i32 }> : (tensor<1x1x1024x64xbf16>, tensor<64x64x3x3xbf16>, tensor<1x1x1x64xbf16>, !ttnn.device) -> tensor<1x1x900x64xbf16>
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, OpModel
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
in_channels | ::mlir::IntegerAttr | 32-bit signless integer attribute |
out_channels | ::mlir::IntegerAttr | 32-bit signless integer attribute |
batch_size | ::mlir::IntegerAttr | 32-bit signless integer attribute |
input_height | ::mlir::IntegerAttr | 32-bit signless integer attribute |
input_width | ::mlir::IntegerAttr | 32-bit signless integer attribute |
kernel_size | ::mlir::DenseI32ArrayAttr | i32 dense array attribute |
stride | ::mlir::DenseI32ArrayAttr | i32 dense array attribute |
padding | ::mlir::DenseI32ArrayAttr | i32 dense array attribute |
dilation | ::mlir::DenseI32ArrayAttr | i32 dense array attribute |
groups | ::mlir::IntegerAttr | 32-bit signless integer attribute |
conv2d_config | ::mlir::tt::ttnn::Conv2dConfigAttr | TTNN Conv2dConfig attribute{{% markdown %}} TTNN conv2d config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
weight | ranked tensor of any type values |
bias | ranked tensor of any type values |
device | TTNN device |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.conv_transpose2d
(tt::ttnn::ConvTranspose2dOp)
ConvTranspose2d operation.
Applies a 2D transposed convolution operator over an input image composed of several input planes.
Inputs:
-
input
AnyRankedTensor: expected in the following format (N, H_in, W_in, C) where:- N is the batch size
- H_in is the height of the input planes
- W_in is the width of the input planes
- C is the number of channels
-
weight
AnyRankedTensor: expected in the following format (C, O/G, K_H, K_W). -
bias
Optional: expected in the following format (1, 1, 1, O) where: - C is the number of input channels
- O is the number of output channels
- G is the number of groups
- K_H is the height of the kernel
- K_W is the width of the kernel
-
output
AnyRankedTensor: expected in the following format (N, H_out, W_out, O) where:- H_out = (H_in - 1) * stride[0] - 2 * padding[0] + dilation[0] * (K_H - 1) + output_padding[0] + 1
- W_out = (W_in - 1) * stride[1] - 2 * padding[1] + dilation[1] * (K_W - 1) + output_padding[1] + 1
Attributes:
in_channels
i32: The number of input channels.out_channels
i32: The number of output channels.batch_size
i32: The batch size.input_height
i32: The input height.input_width
i32: The input width.kernel_size
array<2xi32>: The kernel size.stride
array<2xi32>: Controls the stride for the cross-correlation.padding
array<2xi32>: Controls the amount of implicit zero padding on both sides for dilation * (kernel_size - 1) - padding number of points.output_padding
array<2xi32>: Controls the additional size added to one side of the output shape.dilation
array<2xi32>: Controls the spacing between the kernel pointsgroups
i32: Controls the connections between inputs and outputs. Must be divisible by input and output channels.
Example: // %input: tensor<3x8x8x256xbf16> // %weight: tensor<256x256x3x3xbf16> // %bias: tensor<1x1x1x256xbf16> // %output: tensor<3x10x10x256xbf16> %0 = "ttnn.conv_transpose2d"(%input, %weight, %bias, %output, %device) <{ batch_size = 3: i32, dilation = array<i32: 1, 1>, groups = 1: i32, in_channels = 256: i32, input_height = 8: i32, input_width = 8: i32, kernel_size = array<i32: 3, 3>, out_channels = 256: i32, output_padding = array<i32: 0, 0>, padding = array<i32: 0, 0>, stride = array<i32: 1, 1> > : (tensor<3x8x8x256xbf16>, tensor<256x256x3x3xbf16>, tensor<1x1x1x256xbf16>, tensor<3x10x10x256xbf16>) -> tensor<3x10x10x256xbf16>
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
in_channels | ::mlir::IntegerAttr | 32-bit signless integer attribute |
out_channels | ::mlir::IntegerAttr | 32-bit signless integer attribute |
batch_size | ::mlir::IntegerAttr | 32-bit signless integer attribute |
input_height | ::mlir::IntegerAttr | 32-bit signless integer attribute |
input_width | ::mlir::IntegerAttr | 32-bit signless integer attribute |
kernel_size | ::mlir::DenseI32ArrayAttr | i32 dense array attribute |
stride | ::mlir::DenseI32ArrayAttr | i32 dense array attribute |
padding | ::mlir::DenseI32ArrayAttr | i32 dense array attribute |
output_padding | ::mlir::DenseI32ArrayAttr | i32 dense array attribute |
dilation | ::mlir::DenseI32ArrayAttr | i32 dense array attribute |
groups | ::mlir::IntegerAttr | 32-bit signless integer attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
weight | ranked tensor of any type values |
bias | ranked tensor of any type values |
device | TTNN device |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.cos
(tt::ttnn::CosOp)
Eltwise cosine.
Eltwise cosine operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.deallocate
(tt::ttnn::DeallocateOp)
Deallocate op.
Tensor Deallocate operation
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
force | ::mlir::BoolAttr | bool attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
ttnn.divide
(tt::ttnn::DivideOp)
Eltwise divide.
Eltwise divide operation.
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.embedding_bw
(tt::ttnn::EmbeddingBackwardOp)
Embedding backward op.
Embedding backward operation. Generates the gradient of the embedding operation with respect to the input.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
dtype | ::mlir::tt::DataTypeAttr | TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}} |
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
weight | ranked tensor of any type values |
in_gradient | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.embedding
(tt::ttnn::EmbeddingOp)
Embedding op.
Embedding operation.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
weight | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.empty
(tt::ttnn::EmptyOp)
Empty op.
Tensor empty operation
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
shape | ::mlir::tt::ttnn::ShapeAttr | TTNN Shape attribute{{% markdown %}} TTNN shape attribute {{% /markdown %}} |
dtype | ::mlir::tt::DataTypeAttr | TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}} |
layout | ::mlir::tt::ttnn::LayoutAttr | TTNN Layout{{% markdown %}}Enum cases: * row_major (`RowMajor`) * tile (`Tile`) * invalid (`Invalid`){{% /markdown %}} |
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
device | TTNN device |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.eq
(tt::ttnn::EqualOp)
Eltwise equal to.
Eltwise equal to operation.
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.exp
(tt::ttnn::ExpOp)
Eltwise exponential.
Eltwise exponential operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.expm1
(tt::ttnn::Expm1Op)
Eltwise unary op.
Performs element-wise exponential minus one operation on operand
tensor
and stores the result in the output tensor.
Example: %a: [[0, 1], [0, 0]] "ttnn.exmp1"(%a, %out) -> %out: [[0, 1.71828], [0, 0]]
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.fill_cache
(tt::ttnn::FillCacheOp)
Fill static cache tensor.
Fills the cache
tensor in-place with values from input
at batch_offset
.
Interfaces: MemoryEffectOpInterface (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{MemoryEffects::Write on ::mlir::SideEffects::DefaultResource}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
batch_offset | ::mlir::IntegerAttr | 32-bit signless integer attribute |
Operands:
Operand | Description |
---|---|
cache | ranked tensor of any type values |
input | ranked tensor of any type values |
ttnn.floor
(tt::ttnn::FloorOp)
Eltwise floor op.
Eltwise floor operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.from_device
(tt::ttnn::FromDeviceOp)
FromDevice op.
This op retrieves the input tensor from the given device.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.full
(tt::ttnn::FullOp)
Full op.
Tensor full operation
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
fillValue | ::mlir::FloatAttr | 32-bit float attribute |
Operands:
Operand | Description |
---|---|
device | TTNN device |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.gelu
(tt::ttnn::GeluOp)
Eltwise GELU.
Eltwise GELU operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.get_device
(tt::ttnn::GetDeviceOp)
Get Device op.
This op returns the current runtime device.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, InferTypeOpInterface
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
mesh_shape | ::mlir::tt::ttnn::MeshShapeAttr | TTNN Mesh Shape{{% markdown %}} TTNN mesh shape {{% /markdown %}} |
Results:
Result | Description |
---|---|
device | TTNN device |
ttnn.ge
(tt::ttnn::GreaterEqualOp)
Eltwise greater than or equal to.
Eltwise greater than or equal to operation.
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.gt
(tt::ttnn::GreaterThanOp)
Eltwise greater than.
Eltwise greater than operation.
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.isfinite
(tt::ttnn::IsFiniteOp)
Eltwise isfinite op.
Eltwise isfinite operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.leaky_relu
(tt::ttnn::LeakyReluOp)
Eltwise leaky relu operation.
The Leaky ReLU (Rectified Linear Unit) operation computes an element-wise activation function over its input tensor. It is defined as:
y = x if x > 0 y = parameter * x if x <= 0
where parameter
is a small, user-defined constant that determines the slope for
negative inputs.
Attributes:
parameter
(float): The slope for negative values.
Inputs:
input
(Tensor): The input tensor to be activated.
Outputs:
output
(Tensor): The tensor after applying the Leaky ReLU activation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
parameter | ::mlir::FloatAttr | 32-bit float attribute |
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.le
(tt::ttnn::LessEqualOp)
Eltwise less than or equal to.
Eltwise less than or equal to operation.
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.lt
(tt::ttnn::LessThanOp)
Eltwise less than.
Eltwise less than operation.
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.linear
(tt::ttnn::LinearOp)
Linear transformation of inputs.
Produces the matmul of tensors a
and b
with optional addition with bias
.
Example: // %a = [[1., 2.]], [2., 1.]] // %b = [[0., 1.], [1., 0.]] // %bias = [[1.]] "ttnn.linear"(%a, %b, %bias, %result) : (tensor<2x2xf16>, tensor<2x2xf16>, tensor<1xf16>, tensor<2x2xf16>) -> tensor<2x2xf16> // %result = [[3., 2.], [2., 3.]]
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
transpose_a | ::mlir::BoolAttr | bool attribute |
transpose_b | ::mlir::BoolAttr | bool attribute |
Operands:
Operand | Description |
---|---|
a | ranked tensor of any type values |
b | ranked tensor of any type values |
bias | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.log1p
(tt::ttnn::Log1pOp)
Eltwise log1p operation.
Performs element-wise logarithm plus one operation on operand
tensor and
puts the result in the output tensor.
Example: %a: [0.0, -0.999, 7.0, 6.38905621, 15.0] "ttnn.logp1"(%a, %out) -> %out: [0.0, -6.90776825, 2.07944155, 2.0, 2.77258873]
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.log
(tt::ttnn::LogOp)
Eltwise logarithm.
Eltwise logarithm operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.logical_and
(tt::ttnn::LogicalAndOp)
Eltwise logical and.
Eltwise logical and operation.
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.logical_not
(tt::ttnn::LogicalNotOp)
Eltwise logical not op.
Eltwise logical not operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.logical_or
(tt::ttnn::LogicalOrOp)
Eltwise logical or.
Eltwise logical or operation.
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.logical_xor
(tt::ttnn::LogicalXorOp)
Eltwise logical xor.
Eltwise logical xor operation.
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.matmul
(tt::ttnn::MatmulOp)
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, OpModel
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
transpose_a | ::mlir::BoolAttr | bool attribute |
transpose_b | ::mlir::BoolAttr | bool attribute |
Operands:
Operand | Description |
---|---|
a | ranked tensor of any type values |
b | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.max
(tt::ttnn::MaxOp)
Max reduction op.
Max reduction op.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
keep_dim | ::mlir::BoolAttr | bool attribute |
dim_arg | ::mlir::ArrayAttr | 32-bit integer array attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.max_pool2d
(tt::ttnn::MaxPool2dOp)
Applies a 2D max pooling over an input signal composed of several input planes.
Applies a 2D max pooling over an input signal composed of several input planes.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, OpModel
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
batch_size | ::mlir::IntegerAttr | 32-bit signed integer attribute |
input_height | ::mlir::IntegerAttr | 32-bit signed integer attribute |
input_width | ::mlir::IntegerAttr | 32-bit signed integer attribute |
channels | ::mlir::IntegerAttr | 32-bit signed integer attribute |
kernel_size | ::mlir::DenseI32ArrayAttr | i32 dense array attribute |
stride | ::mlir::DenseI32ArrayAttr | i32 dense array attribute |
padding | ::mlir::DenseI32ArrayAttr | i32 dense array attribute |
dilation | ::mlir::DenseI32ArrayAttr | i32 dense array attribute |
ceil_mode | ::mlir::BoolAttr | bool attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.maximum
(tt::ttnn::MaximumOp)
Eltwise maximum OP.
Calculates maximum of input tensors' values element-wise and stores result in output tensor.
Example: %lhs: [[3, 2, 7], [1, 4, 4]] %rhs: [[1, 4, 2], [1, 2, 3]] "ttnn.maximum"(%lhs, %rhs, %out) -> %out: [[3, 4, 7], [1, 4, 4]]
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.mean
(tt::ttnn::MeanOp)
Mean reduction op.
Mean reduction op.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, OpModel
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
keep_dim | ::mlir::BoolAttr | bool attribute |
dim_arg | ::mlir::ArrayAttr | 32-bit integer array attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.mesh_shard
(tt::ttnn::MeshShardOp)
Mesh shard op.
Tensor Mesh Shard operation
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
shard_direction | ::mlir::tt::MeshShardDirectionAttr | TT MeshShardDirection{{% markdown %}}Enum cases: * full_to_shard (`FullToShard`) * shard_to_full (`ShardToFull`){{% /markdown %}} |
shard_type | ::mlir::tt::MeshShardTypeAttr | MeshShard shard_type attribute in TT dialect{{% markdown %}} Define sharded tensor data of mesh_shard op. - Identity: input and output tensors are pre-sharded (same data) and no sharding is required. - Replicate: all of the devices has full tensor (same data). - Maximal: one or part of the devcices has full tensor (same data). - Devices: all or part of the devices has sharded (partial) tensor (different data). {{% /markdown %}} |
shard_shape | ::mlir::DenseI64ArrayAttr | i64 dense array attribute |
shard_dims | ::mlir::DenseI64ArrayAttr | i64 dense array attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
device | TTNN device |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.min
(tt::ttnn::MinOp)
Min reduction op.
This op computes the minimum of all elements of the tensor or along specified dimension.
Example: input: [[1, 5, 3], [4, 2, 6]]
// Computing along dim 0 output: [1, 2, 3]
// Computing along dim 1 output: [1, 2]
// Computing for entire tensor output: 1
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
keep_dim | ::mlir::BoolAttr | bool attribute |
dim_arg | ::mlir::ArrayAttr | 32-bit integer array attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.minimum
(tt::ttnn::MinimumOp)
Eltwise minimum OP.
Calculates minimum of input tensors' values element-wise and stores result in output tensor.
Example: %lhs: [[3, 2, 7], [1, 4, 4]] %rhs: [[1, 4, 2], [1, 2, 3]] "ttnn.minimum"(%lhs, %rhs, %out) -> %out: [[1, 2, 2], [1, 2, 3]]
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.moreh_cumsum
(tt::ttnn::MorehCumSumOp)
Moreh cummulative sum op.
Computes the cumulative sum of elements of a tensor along specified dimension.
Example: input: [[1, 2, 3], [4, 5, 6]]
// Cumulative sum along dim=0: output: [[1, 2, 3], [5, 7, 9]]
// Cumulative sum along dim=1: output: [[1, 3, 6], [4, 9, 15]]
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
dim | ::mlir::IntegerAttr | 64-bit signless integer attribute |
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.multiply
(tt::ttnn::MultiplyOp)
Eltwise multiply.
Eltwise multiply operation.
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, OpModel
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.neg
(tt::ttnn::NegOp)
Eltwise negate.
Eltwise negate operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.ne
(tt::ttnn::NotEqualOp)
Eltwise not equal to.
Eltwise not equal to operation.
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.ones
(tt::ttnn::OnesOp)
Creates a tensor filled with ones.
Tensor operation to create a tensor filled with ones.
Given a ShapeAttr shape
, produces a tensor with the same shape, filled with ones.
Example: %0 = "ttnn.ones"() <{shape = array<i32:64, 28, 28>}> : () -> tensor<64x28x28xbf16> // %0: [[[1, 1, 1, ..., 1], [1, 1, 1, ..., 1], ..., [1, 1, 1, ..., 1]]]
Traits: AlwaysSpeculatableImplTrait
, HasMemoryConfigTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
shape | ::mlir::tt::ttnn::ShapeAttr | TTNN Shape attribute{{% markdown %}} TTNN shape attribute {{% /markdown %}} |
dtype | ::mlir::tt::DataTypeAttr | TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}} |
layout | ::mlir::tt::ttnn::LayoutAttr | TTNN Layout{{% markdown %}}Enum cases: * row_major (`RowMajor`) * tile (`Tile`) * invalid (`Invalid`){{% /markdown %}} |
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
device | TTNN device |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.pad
(tt::ttnn::PadOp)
Pad op.
Pad input tensor by padding the input_shape to output_shape using the provided value.
The padding
attribute must be a sequence of integers that is twice the size as the rank of the input.
Each pair of integers in the padding attribute represents the amount of padding to add to the low and high of that dimension.
I.e: an input tensor of shape <1x30x30x64xf32> with padding attribute <0, 0, 1, 1, 1, 1, 0, 0> will return a tensor of shape <1x32x32x64xf32>,
and so will a padding attribute of <0, 0, 0, 2, 0, 2, 0, 0>.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
padding | ::mlir::DenseI32ArrayAttr | i32 dense array attribute |
value | ::mlir::FloatAttr | 32-bit float attribute |
use_multicore | ::mlir::BoolAttr | bool attribute |
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.permute
(tt::ttnn::PermuteOp)
Permute operation.
Permute input tensor dimensions.
Attributes:
permutation
array: The permutation of the input tensor dimensions.
Example: %a = ttir.empty() : () -> tensor<2x3x4xi32> %0 = "ttir.permute"(%a) {permutation = array<i64: 1, 2, 0>} : (tensor<2x3x4xi32>) -> tensor<3x4x2xi32>
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
permutation | ::mlir::DenseI64ArrayAttr | i64 dense array attribute |
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
pad_value | ::mlir::FloatAttr | 32-bit float attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.pow
(tt::ttnn::PowerOp)
Eltwise power OP.
Performs element-wise exponentiation of lhs tensor by rhs tensor and produces a result tensor. Tensors must be of same shape.
Example:
%result = "ttnn.pow"(%lhs, %rhs) : (tensor<6xf64>, tensor<6xf64>) -> tensor<6xf64>
%lhs: [-2.0, -0.0, -36.0, 5.0, 3.0, 10000.0]
%rhs: [2.0, 2.0, 1.1, 2.0, -1.0, 10.0]
%result: [4.0, 0.0, -nan, 25.0, 0.333333343, inf]
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.prod
(tt::ttnn::ProdOp)
Product reduction op.
This op computes the product of all elements of the tensor (full product) or along a specific dimension.
Example: input: [[1, 2, 3], [4, 5, 6]]
// Computing along dim 0 output: [4, 10, 18]
// Computing along dim 1 output: [6, 120]
// Computing full product output: 720
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
all_dimensions | ::mlir::BoolAttr | bool attribute |
keep_dim | ::mlir::BoolAttr | bool attribute |
dim_arg | ::mlir::IntegerAttr | 64-bit signless integer attribute |
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.reciprocal
(tt::ttnn::ReciprocalOp)
Eltwise reciprocal.
Eltwise reciprocal operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.reduce_scatter
(tt::ttnn::ReduceScatterOp)
Reduce scatter op.
Tensor Reduce Scatter operation
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
reduce_type | ::mlir::tt::ReduceTypeAttr | TT Reduce Type{{% markdown %}}Enum cases: * sum (`Sum`) * mean (`Mean`) * max (`Max`) * min (`Min`) * std (`Std`) * var (`Var`){{% /markdown %}} |
scatter_dim | ::mlir::IntegerAttr | 32-bit signed integer attribute |
cluster_axis | ::mlir::IntegerAttr | 32-bit unsigned integer attribute |
num_links | ::mlir::IntegerAttr | 32-bit unsigned integer attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
device | TTNN device |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.relu
(tt::ttnn::ReluOp)
Eltwise ReLU.
Eltwise ReLU operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, OpModel
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.remainder
(tt::ttnn::RemainderOp)
Eltwise remainder.
Performs element-wise remainder of dividend lhs and divisor rhs tensors and produces a result tensor.
Example:
// %lhs: [17, -17, 17, -17] // %rhs: [3, 3, -3, -3] %result = "ttnn.remainder"(%lhs, %rhs) : (tensor<4xi64>, tensor<4xi64>) -> tensor<4xi64> // %result: [2, -2, 2, -2]
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.repeat_interleave
(tt::ttnn::RepeatInterleaveOp)
Repeat interleave op.
Repeats elements of a tensor along a specified dimension. It allows for flexible repetition patterns, where each element can be repeated a different number of times. This is particularly useful for tasks that require duplicating elements in a non-uniform manner.
Parameters:
input
: The input tensor.repeats
: Specifies the number of repetitions for each element, each element is repeated that number of times.dim
: The dimension along which to repeat values.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
repeats | ::mlir::IntegerAttr | 32-bit unsigned integer attribute |
dim | ::mlir::IntegerAttr | 32-bit signed integer attribute |
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.repeat
(tt::ttnn::RepeatOp)
Repeat op.
Returns a new tensor filled with repetition of input tensor according to number of times specified in repeat_dims.
Parameters:
input_tensor
(ttnn.Tensor): the input tensor.repeat_dims
(number): The number of repetitions for each element.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
repeat_dims | ::mlir::tt::ttnn::ShapeAttr | TTNN Shape attribute{{% markdown %}} TTNN shape attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.reshape
(tt::ttnn::ReshapeOp)
Reshape op.
Reshape tensor.
Traits: AlwaysSpeculatableImplTrait
, HasMemoryConfigTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, OpModel
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
shape | ::mlir::ArrayAttr | 32-bit integer array attribute |
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.rsqrt
(tt::ttnn::RsqrtOp)
Eltwise rsqrt.
Eltwise rsqrt operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.scatter
(tt::ttnn::ScatterOp)
Scatter op.
Embeds the values of the 'update' tensor into 'input' at the given index and puts the value in the 'output' tensor.
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.sigmoid
(tt::ttnn::SigmoidOp)
Eltwise sigmoid.
Eltwise sigmoid operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.sign
(tt::ttnn::SignOp)
Eltwise sign operation.
Returns the sign of the operand
element-wise and produces a result
tensor.
Example: %a: [[3, -2, 0], [1, -4, 4]] "ttnn.sign"(%a, %out) -> %out: [[1, -1, 0], [1, -1, 1]]
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.sin
(tt::ttnn::SinOp)
Eltwise sine.
Eltwise sine operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.slice
(tt::ttnn::SliceOp)
Slice op.
Extract a portion of a tensor based on the specified start (begins
), stop (ends
), and step
indices for each dimension.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
begins | ::mlir::ArrayAttr | 32-bit integer array attribute |
ends | ::mlir::ArrayAttr | 32-bit integer array attribute |
step | ::mlir::ArrayAttr | 32-bit integer array attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.softmax
(tt::ttnn::SoftmaxOp)
Softmax op.
Softmax operation.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, OpModel
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
dimension | ::mlir::IntegerAttr | 32-bit signed integer attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.sqrt
(tt::ttnn::SqrtOp)
Eltwise sqrt.
Eltwise sqrt operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, OpModel
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.subtract
(tt::ttnn::SubtractOp)
Eltwise subtract.
Eltwise subtract operation.
Traits: AlwaysSpeculatableImplTrait
, TwoOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.sum
(tt::ttnn::SumOp)
Sum reduction op.
Sum reduction op.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
keep_dim | ::mlir::BoolAttr | bool attribute |
dim_arg | ::mlir::ArrayAttr | 32-bit integer array attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.tan
(tt::ttnn::TanOp)
Eltwise tan op.
Eltwise tan operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.tanh
(tt::ttnn::TanhOp)
Eltwise tanh op.
Eltwise tanh operation.
Traits: AlwaysSpeculatableImplTrait
, OneOperand
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.to_dtype
(tt::ttnn::ToDTypeOp)
ToDType op.
This op converts the data type of the input tensor based on the given data type on the host.
Args:
- :attr:
input
: the ttnn.Tensor - :attr:
dtype
:ttnn
data type.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
dtype | ::mlir::tt::DataTypeAttr | TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}} |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.to_device
(tt::ttnn::ToDeviceOp)
ToDevice op.
This op sends the input tensor to the given device with the given memory config.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
device | TTNN device |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.to_layout
(tt::ttnn::ToLayoutOp)
ToLayout op.
This op wraps all layout information gathered from ttir.toLayout. It is used/updated by the optimizer to perform optimizations, and later broken down into specific memory/layout operations (toDevice, toMemoryConfig etc.). Currently in the TTNN backend, we use this op solely for tilize/untilize, therefore marking all other attrs as optional. Once ttnn::to_layout supports other attrs, we can remove the optional tag.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, OpModel
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
layout | ::mlir::tt::ttnn::LayoutAttr | TTNN Layout{{% markdown %}}Enum cases: * row_major (`RowMajor`) * tile (`Tile`) * invalid (`Invalid`){{% /markdown %}} |
dtype | ::mlir::tt::DataTypeAttr | TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}} |
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
device | TTNN device |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.to_memory_config
(tt::ttnn::ToMemoryConfigOp)
ToMemoryConfig op.
This op converts the memory config of the input tensor based on the given memory config. It handles:
- Dram to L1
- L1 to Dram
- Interleaved to sharded
- Sharded to interleaved
- Sharded to sharded (reshard)
Traits: AlwaysSpeculatableImplTrait
, HasMemoryConfigTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.transpose
(tt::ttnn::TransposeOp)
Transpose op.
Transpose tensor along two given dimensions.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, OpModel
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
dim0 | ::mlir::IntegerAttr | 32-bit signed integer attribute |
dim1 | ::mlir::IntegerAttr | 32-bit signed integer attribute |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.typecast
(tt::ttnn::TypecastOp)
Typecast op.
This op converts the data type of the input tensor based on the given data type. It handles:
- conversions of data types.
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, OpModel
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
dtype | ::mlir::tt::DataTypeAttr | TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}} |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.update_cache
(tt::ttnn::UpdateCacheOp)
Update static cache tensor.
Updates the cache
tensor in-place with values from input
at update_index
and batch_offset
.
Interfaces: MemoryEffectOpInterface (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{MemoryEffects::Write on ::mlir::SideEffects::DefaultResource}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
batch_offset | ::mlir::IntegerAttr | 32-bit signless integer attribute |
Operands:
Operand | Description |
---|---|
cache | ranked tensor of any type values |
input | ranked tensor of any type values |
update_index | ranked tensor of any type values |
ttnn.upsample
(tt::ttnn::UpsampleOp)
Upsample 2D op.
Upsample 2D operation. Input tensor is assumed to be in NHWC format.
Attributes:
scale_factor
(si32 | array): The scale factor for upsampling in H and W dimensions respectively. mode
(str): The upsampling algorithm. Currently only "nearest" and "bilinear" are supported. Default is "nearest".
Example: // %a: tensor<10x64x32xbf16> %0 = "ttnn.upsample"(%a) <{scale_factor = array<i32: 2, 4>}> : (tensor<10x64x32x3xbf16>) -> tensor<10x128x128x3xbf16>
Traits: AlwaysSpeculatableImplTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
scale_factor | ::mlir::Attribute | 32-bit signed integer attribute or i32 dense array attribute |
mode | ::mlir::StringAttr | string attribute |
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
input | ranked tensor of any type values |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |
ttnn.where
(tt::ttnn::WhereOp)
Eltwise where.
Eltwise where operation.
Traits: AlwaysSpeculatableImplTrait
, ThreeOperands
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Operands:
Operand | Description |
---|---|
inputs | variadic of ranked tensor of any type values |
Results:
Result | Description |
---|---|
results | variadic of ranked tensor of any type values |
ttnn.zeros
(tt::ttnn::ZerosOp)
Creates a tensor filled with zeros.
Tensor operation to create a tensor filled with zeros.
Given a ShapeAttr shape
, produces a tensor with the same shape, filled with zeros.
Example: %0 = "ttnn.zeros"() <{shape = array<i32:64, 28, 28>}> : () -> tensor<64x28x28xbf16> // %0: [[[0, 0, 0, ..., 0], [0, 0, 0, ..., 0], ..., [0, 0, 0, ..., 0]]]
Traits: AlwaysSpeculatableImplTrait
, HasMemoryConfigTrait
Interfaces: ConditionallySpeculatable
, NoMemoryEffect (MemoryEffectOpInterface)
, TTNN_WorkaroundInterface
Effects: MemoryEffects::Effect{}
Attributes:
Attribute | MLIR Type | Description |
---|---|---|
shape | ::mlir::tt::ttnn::ShapeAttr | TTNN Shape attribute{{% markdown %}} TTNN shape attribute {{% /markdown %}} |
dtype | ::mlir::tt::DataTypeAttr | TT DataTypes{{% markdown %}}Enum cases: * f32 (`Float32`) * f16 (`Float16`) * bf16 (`BFloat16`) * bfp_f8 (`BFP_Float8`) * bfp_bf8 (`BFP_BFloat8`) * bfp_f4 (`BFP_Float4`) * bfp_bf4 (`BFP_BFloat4`) * bfp_f2 (`BFP_Float2`) * bfp_bf2 (`BFP_BFloat2`) * u32 (`UInt32`) * u16 (`UInt16`) * u8 (`UInt8`) * si32 (`Int32`){{% /markdown %}} |
layout | ::mlir::tt::ttnn::LayoutAttr | TTNN Layout{{% markdown %}}Enum cases: * row_major (`RowMajor`) * tile (`Tile`) * invalid (`Invalid`){{% /markdown %}} |
memory_config | ::mlir::tt::ttnn::MemoryConfigAttr | TTNN MemoryConfig attribute{{% markdown %}} TTNN memory config attribute {{% /markdown %}} |
Operands:
Operand | Description |
---|---|
device | TTNN device |
Results:
Result | Description |
---|---|
result | ranked tensor of any type values |