ttnn.gather
- ttnn.gather(input: ttnn.Tensor, dim: int, index: ttnn.Tensor, *, sparse_grad: bool = False, memory_config: ttnn.MemoryConfig = None, out: ttnn.Tensor = None, sub_core_grids: ttnn.CoreRangeSet = None) ttnn.Tensor
-
The gather operation extracts values from the input tensor based on indices provided in the index tensor along a specified dimension.
The input tensor and the index tensor must have the same number of dimensions. For all dimensions except the specified one (dim), the size of the index tensor must not exceed the size of the input tensor. The output tensor will have the same shape as the index tensor. Note that the input and index tensors do not broadcast against each other.
Index values must lie in the range [0, input.shape[dim]). The value produced at any output position whose index falls outside that range is unspecified.
- Parameters:
-
input (ttnn.Tensor)The source tensor from which values are gathered.
dim (int)The dimension along which values are gathered.
index (ttnn.Tensor)A tensor containing the indices of elements to gather, with the same number of dimensions as the input tensor.
- Keyword Arguments:
-
sparse_grad (bool, optional)Not supported: it must be False, and True raises an error. Defaults to False.
memory_config (ttnn.MemoryConfig, optional)Specifies the memory configuration for the output tensor. Defaults to None.
out (ttnn.Tensor, optional)A preallocated tensor to store the gathered values. Defaults to None.
sub_core_grids (ttnn.CoreRangeSet, optional)Custom core range set for operation execution. Allows specification of which cores should be used for the operation. Defaults to None.
- Returns:
-
ttnn.Tensorthe gathered values, with the same shape as index.
- Additional Information:
-
Currently, the sparse_grad argument is not supported.
Note
Supported dtypes and layouts for input tensor values:
Dtypes
Layouts
BFLOAT16, FLOAT32
TILE, ROW_MAJOR
UINT16, UINT32, INT32
TILE
Supported dtypes and layouts for index tensor values:
Dtypes
Layouts
UINT16, UINT32
TILE, ROW_MAJOR
The input and index tensors must use the same layout.
- Memory Support:
-
Interleaved: DRAM and L1
Sharded: HEIGHT_SHARDED, WIDTH_SHARDED and BLOCK_SHARDED in L1; TILE input and index tensors can also be HEIGHT_SHARDED in DRAM
Example
# Create a tensor and an index tensor ttnn_input = ttnn.rand((4, 4), dtype=ttnn.bfloat16, layout=ttnn.Layout.TILE, device=device) ttnn_index = ttnn.from_torch( torch.tensor([[0, 1], [1, 2], [2, 3], [3, 0]], dtype=torch.int32), dtype=ttnn.uint16, layout=ttnn.Layout.TILE, device=device, ) # Gather elements along dimension 1 using the index tensor gathered_tensor = ttnn.gather(ttnn_input, 1, index=ttnn_index) logger.info(f"Gathered Tensor Shape: {gathered_tensor.shape}") # Shape([4, 2])