ttnn.gather

ttnn.gather(input: ttnn.Tensor, dim: int, index: ttnn.Tensor, *, sparse_grad: bool = False, memory_config: ttnn.MemoryConfig = None, out: ttnn.Tensor = None, sub_core_grids: ttnn.CoreRangeSet = None) → ttnn.Tensor

The gather operation extracts values from the input tensor based on indices provided in the index tensor along a specified dimension.

The input tensor and the index tensor must have the same number of dimensions. For all dimensions except the specified one (dim), the size of the index tensor must not exceed the size of the input tensor. The output tensor will have the same shape as the index tensor. Note that the input and index tensors do not broadcast against each other.

Index values must lie in the range [0, input.shape[dim]). The value produced at any output position whose index falls outside that range is unspecified.

Parameters:
  • input (ttnn.Tensor)The source tensor from which values are gathered.

  • dim (int)The dimension along which values are gathered.

  • index (ttnn.Tensor)A tensor containing the indices of elements to gather, with the same number of dimensions as the input tensor.

Keyword Arguments:
  • sparse_grad (bool, optional)Not supported: it must be False, and True raises an error. Defaults to False.

  • memory_config (ttnn.MemoryConfig, optional)Specifies the memory configuration for the output tensor. Defaults to None.

  • out (ttnn.Tensor, optional)A preallocated tensor to store the gathered values. Defaults to None.

  • sub_core_grids (ttnn.CoreRangeSet, optional)Custom core range set for operation execution. Allows specification of which cores should be used for the operation. Defaults to None.

Returns:

ttnn.Tensorthe gathered values, with the same shape as index.

Additional Information:
  • Currently, the sparse_grad argument is not supported.

Note

Supported dtypes and layouts for input tensor values:

Dtypes

Layouts

BFLOAT16, FLOAT32

TILE, ROW_MAJOR

UINT16, UINT32, INT32

TILE

Supported dtypes and layouts for index tensor values:

Dtypes

Layouts

UINT16, UINT32

TILE, ROW_MAJOR

The input and index tensors must use the same layout.

Memory Support:
  • Interleaved: DRAM and L1

  • Sharded: HEIGHT_SHARDED, WIDTH_SHARDED and BLOCK_SHARDED in L1; TILE input and index tensors can also be HEIGHT_SHARDED in DRAM

Example

# Create a tensor and an index tensor
ttnn_input = ttnn.rand((4, 4), dtype=ttnn.bfloat16, layout=ttnn.Layout.TILE, device=device)
ttnn_index = ttnn.from_torch(
    torch.tensor([[0, 1], [1, 2], [2, 3], [3, 0]], dtype=torch.int32),
    dtype=ttnn.uint16,
    layout=ttnn.Layout.TILE,
    device=device,
)

# Gather elements along dimension 1 using the index tensor
gathered_tensor = ttnn.gather(ttnn_input, 1, index=ttnn_index)
logger.info(f"Gathered Tensor Shape: {gathered_tensor.shape}")  # Shape([4, 2])