tt-mlir documentation

ArchAttr

TT Arch

Syntax:

#ttcore.arch<
  ::mlir::tt::ttcore::Arch   # value
>

Parameters:

ArgumentAllocationAttr

Argument allocation attribute in TT dialect

Syntax:

#ttcore.arg_alloc<
  uint64_t,   # address
  uint64_t,   # size
  MemorySpace   # memorySpace
>

Holds the metadata for the allocation of an function argument i.e. for graph inputs.

Parameters:

ArgumentTypeAttr

Argument Type

Syntax:

#ttcore.argument_type<
  ::mlir::tt::ttcore::ArgumentType   # value
>

Parameters:

CPUDescAttr

TT cpu_desc attribute

Syntax:

#ttcore.cpu_desc<
  CPURole,   # role
  StringAttr   # target_triple
>

TT cpu_desc attribute

Parameters:

CPURoleAttr

TT CPU Role

Syntax:

#ttcore.cpu_role<
  ::mlir::tt::ttcore::CPURole   # value
>

Parameters:

ChipChannelAttr

TT chip_channel attribute

Syntax:

#ttcore.chip_channel<
  unsigned,   # deviceId0
  ::llvm::ArrayRef<int64_t>,   # ethernetCoreCoord0
  unsigned,   # deviceId1
  ::llvm::ArrayRef<int64_t>   # ethernetCoreCoord1
>

TT chip_channel attribute

Parameters:

ChipCoordAttr

TT chip_coord attribute

Syntax:

#ttcore.chip_coord<
  unsigned,   # rack
  unsigned,   # shelf
  unsigned,   # y
  unsigned   # x
>

TT chip_coord attribute

Parameters:

ChipDescAttr

TT chip_desc attribute

Syntax:

#ttcore.chip_desc<
  ArchAttr,   # arch
  ::llvm::ArrayRef<int64_t>,   # grid
  ::llvm::ArrayRef<int64_t>,   # coordTranslationOffsets
  unsigned,   # l1Size
  unsigned,   # numDramChannels
  unsigned,   # dramChannelSize
  unsigned,   # nocL1AddressAlignBytes
  unsigned,   # pcieAddressAlignBytes
  unsigned,   # nocDRAMAddressAlignBytes
  unsigned,   # l1UnreservedBase
  unsigned,   # eriscL1UnreservedBase
  unsigned,   # dramUnreservedBase
  unsigned,   # dramUnreservedEnd
  ChipPhysicalHelperCoresAttr,   # chipPhysicalHelperCores
  ::llvm::ArrayRef<DataTypeAttr>,   # supportedDataTypes
  ::llvm::ArrayRef<TileSizeAttr>,   # supportedTileSizes
  unsigned,   # dstRegisterSizeTiles
  unsigned,   # numCBs
  unsigned,   # numComputeThreads
  unsigned   # numDatamovementThreads
>

TT chip_desc attribute

Parameters:

ChipPhysicalHelperCoresAttr

TT chip_physical_helper_cores attribute

Syntax:

#ttcore.chip_physical_helper_cores<
  ::llvm::ArrayRef<CoreCoordAttr>,   # dram
  ::llvm::ArrayRef<CoreCoordAttr>,   # eth
  ::llvm::ArrayRef<CoreCoordAttr>   # eth_inactive
>

TT chip_physical_helper_cores attribute containing arrays of physical helper cores by core type in order of logical cores.

Parameters:

CoreCoordAttr

TT core_coord attribute

Syntax:

#ttcore.core_coord<
  int64_t,   # y
  int64_t   # x
>

TT core_coord attribute containing a single physical core coordinate.

Parameters:

DataTypeAttr

TT DataTypes

Syntax:

#ttcore.supportedDataTypes<
  ::mlir::tt::ttcore::DataType   # value
>

Parameters:

DeviceAttr

Device attribute in TT dialect.

Syntax:

#ttcore.device<
  ::mlir::tt::ttcore::GridAttr,   # workerGrid
  AffineMap,   # l1Map
  AffineMap,   # dramMap
  ::llvm::ArrayRef<int64_t>,   # meshShape
  ::llvm::ArrayRef<unsigned>   # chipIds
>

Describes the physical layout of a device in the system and is made up of a few components:

• A grid attribute that describes the device's compute grid shape. It not only describes the shape of the compute grid, but also carries an affine map that describes how the logical grid maps to the physical grid.
• Two affine maps that describe how a tensor layout's linear attribute maps to the L1 and DRAM memory spaces.
• A mesh shape that describes the virtual layout of the chips with respect to each other. Note that in a multi-chip system, this grid encapsulates the entire system's grid shape, e.g. 8x16 grid could be made up of a 1x2 mesh of chips side-by-side. The mesh attribute configures how the above grid/map attributes are created such that they implement this mesh topology.
• An array of chip ids that this device is made up of. This array's length must match the volume of the mesh shape and should be interpreted in row-major order.

Parameters:

GridAttr

TT grid attribute

Syntax:

#ttcore.grid<
  ::llvm::ArrayRef<int64_t>,   # shape
  AffineMap   # mapping
>

TT grid attribute

Parameters:

IteratorTypeAttr

TT IteratorType

Syntax:

#ttcore.iterator_type<
  ::mlir::tt::ttcore::IteratorType   # value
>

Parameters:

MemorySpaceAttr

TT MemorySpace

Syntax:

#ttcore.memory_space<
  ::mlir::tt::ttcore::MemorySpace   # value
>

Parameters:

MeshAttr

Mesh reference attribute in TT dialect.

Syntax:

#ttcore.mesh<
  StringAttr,   # name
  ::llvm::ArrayRef<int64_t>   # shape
>

Describes a mesh config including name and shape.

Parameters:

MeshShardDirectionAttr

TT MeshShardDirection

Syntax:

#ttcore.shard_direction<
  ::mlir::tt::ttcore::MeshShardDirection   # value
>

Parameters:

MeshShardTypeAttr

MeshShard shard_type attribute in TT dialect

Syntax:

#ttcore.shard_type<
  ::mlir::tt::ttcore::MeshShardType   # value
>

Define sharded tensor data of mesh_shard op.

• Identity: input and output tensors are pre-sharded (same data) and no sharding is required.
• Replicate: all of the devices has full tensor (same data).
• Maximal: one or part of the devcices has full tensor (same data).
• Devices: all or part of the devices has sharded (partial) tensor (different data).

Parameters:

MeshesAttr

TT system meshes attribute.

Syntax:

#ttcore.meshes<
  ::llvm::ArrayRef<MeshAttr>   # meshes
>

TT system meshes attribute includes one or more mesh configs used for networks.

Parameters:

MetalLayoutAttr

Tensor layout attribute with explicit physical shape

Syntax:

#ttcore.metal_layout<
  ::llvm::ArrayRef<int64_t>,   # logical_shape
  ::llvm::ArrayRef<int64_t>,   # dim_alignments
  DenseIntElementsAttr,   # collapsed_intervals
  OOBVal,   # oob_val
  MemorySpace   # memory_space
>

The tensor layout attribute captures how tensor data is sharded across a grid of devices/cores and is laid out in memory. Note that the presence of this attribute implies that the tensor shape includes sharding (i.e. the first half of the tensor shape represents the grid shape).

Some high level goals:

• Logical shapes: Store the original tensor shape and rank intact and agnostic to underlying storage layout. Keeping the logical shapes not only makes some graph transformations vastly simpler, in particular convs, but it makes the lowered IR much easier to read and reason about. The original tensor shapes leave breadcrumbs that make it much easier to map back to the input representation.
• Collapsed dims: We may collapse dimensions during transformation, but it is important we capture this information such that it is not lost during tensor transformation. The collapsed_intervals field stores the collapses performed during conversion from logical_shape to physical tensor shape.
• Padding: store the desired alignments s.t. padding can be simply encoded; dim_alignments field represents alignment along each logical dim during collapse.
• Memref translation: ensure we have all necessary info s.t. we can trivally lower a tensor into a memref without any intermediate passes.

For a logical tensor of shape [H, W] distributed across a grid [GY, GX], the tensor shape would be:

• Without tiling: [GY, GX, H/GY, W/GX]
• With tiling: [GY, GX, H/GY/TH, W/GX/TW, TH, TW] where TH,TW are tile dimensions

This makes the representation 1:1 with memrefs and eliminates the need for shape conversion passes.

Examples:

// Logical 8x300 tensor distributed across 1x2 grid:
// tensor<1x2x8x150xf32, #tt.metal_layout<logical_shape=8x300, ...>>

// Logical 1024x1024 tensor distributed across 2x2 grid with 32x32 tiles:
// tensor<2x2x16x16x!ttcore.tile<32x32xf32>, #tt.metal_layout<logical_shape=1024x1024, ...>>

Parameters:

OOBValAttr

TT OOBVal

Syntax:

#ttcore.oob_val<
  ::mlir::tt::ttcore::OOBVal   # value
>

Parameters:

ReduceTypeAttr

TT Reduce Type

Syntax:

#ttcore.reduce_type<
  ::mlir::tt::ttcore::ReduceType   # value
>

Parameters:

ShardLayoutAttr

Shard layout attribute in TT dialect

Syntax:

#ttcore.shard<
  ::llvm::ArrayRef<int64_t>,   # stride
  uint32_t   # buffers
>

Describes shard layout of a memref buffer.

• Stride: Stride of each dim in bytes.
• Buffers: Number of back buffers used for double buffering, I/O latency hiding, etc

The shard layout attribute is a description of how each shard of a memref is laid out in memory. Memref's with this layout type implicitly mean their data is distributed across a grid of cores.

Parameters:

SystemDescAttr

TT system_desc attribute

Syntax:

#ttcore.system_desc<
  ::llvm::ArrayRef<CPUDescAttr>,   # cpuDescs
  ::llvm::ArrayRef<ChipDescAttr>,   # chipDescs
  ::llvm::ArrayRef<unsigned>,   # chipDescIndices
  ::llvm::ArrayRef<ChipCapabilityAttr>,   # chipCapabilities
  ::llvm::ArrayRef<ChipCoordAttr>,   # chipCoords
  ::llvm::ArrayRef<ChipChannelAttr>   # chipChannels
>

TT system_desc attribute

Parameters:

TensorMeshShardingAttr

Tensor mesh sharding attribute in TT dialect.

Syntax:

#ttcore.mesh_sharding<
  StringAttr,   # name
  ::llvm::ArrayRef<TensorMeshShardingAxisAttr>   # tensor_mesh_sharding_axis
>

Describes a tensor's multi-device status.

• Single device tensor has no TensorMeshShardingAttr. tensor<784x16384xf32>

• Multi-device tensors have TensorMeshShardingAttr. (i) multi-device tensor without tensor mesh shard axis indicates all devices in "mesh" have full size tensors e.g., 784x16384 for tensor<784x16384xf32, #ttcore.mesh_sharding<"mesh">>

  (ii) multi-device tensor with tensor mesh shard axis indicate all devices in "mesh" have sharded tensor defined by the TensorMeshShardingAxisAttr. e.g., 192x16384 for tensor<784x16384xf32, #ttcore.mesh_sharding<"mesh", [ 4(1), 1]>>. Here, 4(1) indicates shard_shape(shard_dim), so 784 should be sharded by 4 at "mesh"'s second hardware dimension. 1 indicates no sharding, so 16384 is not being sharded.

Parameters:

TensorMeshShardingAxisAttr

Tensor mesh sharding axis info attribute in TT dialect.

Syntax:

#ttcore.tensor_sharding<
  int64_t,   # shard_shape
  ::llvm::ArrayRef<int64_t>   # axes
>

Details per tensor dimension sharding and axes info.

• shard_shape: shard shape at a tensor dimension.
• (optional) axes: mesh shard dimensions. Axes may be empty if it is not being sharded.

Parameters:

TileSizeAttr

TT tile_size attribute

Syntax:

#ttcore.tile_size<
  int64_t,   # y
  int64_t   # x
>

TT tile_size attribute containing a supported Tensix tile shape.

Parameters:

ViewLayoutAttr

View layout attribute in TT dialect

Syntax:

#ttcore.view<
  AffineMap   # affineMap
>

Describes a view layout of a memref buffer.

• AffineMap: Provides affine map indexing into the associated data view.

Only the view_layout or stream_layout ops should return memref's with this attribute. The view layout attribute is necessary for two reasons:

• It provides a way to reblock the data view into a different shape (via affine map). Usually this would be some subblock of the original backing memory to chunk the data into smaller pieces.
• The type itself is a signal to datamovement passes that the memref is a view and should be treated as such.

Parameters: