ttnn.experimental.rotary_embedding

ttnn.experimental.rotary_embedding(input_tensor: ttnn.Tensor, cod_cache: ttnn.Tensor, sin_cache: ttnn.Tensor, token_index: int | None = None, *, memory_config: ttnn.MemoryConfig | None = None, compute_kernel_config: ttnn.DeviceComputeKernelConfig | None = None) → ttnn.Tensor

Applies the rotary embedding to the input_tensor tensor using the cos_cache and sin_cache tensors.

When token_idx is passed, this assumes input is transposed to [seq_len, 1, B, head_dim], and seq_len is 1.

Parameters:

• input_tensor (ttnn.Tensor) – the input tensor.

• cod_cache (ttnn.Tensor) – the Cosine Cache tensor.

• sin_cache (ttnn.Tensor) – the Sine Cache tensor.

• token_index (int, optional) – Defaults to None.

Keyword Arguments:

• memory_config (ttnn.MemoryConfig, optional) – Memory configuration for the operation. Defaults to None.

• compute_kernel_config (ttnn.DeviceComputeKernelConfig, optional) – Defaults to None.

Returns:

ttnn.Tensor – the output tensor.