ttnn.fill_cache

ttnn.fill_cache() → None

Fills the cache tensor in place with the values from input at the specified batch_idx.

:param * cache_tensor: The cache tensor to be written to. :type * cache_tensor: ttnn.Tensor :param * input_tensor: The token tensor to be written to the cache. :type * input_tensor: ttnn.Tensor :param * batch_idx: The index into the cache tensor. :type * batch_idx: int

Example

>>> tensor1 = ttnn.from_torch(torch.tensor((1, 2), dtype=torch.bfloat16), device=device)
>>> tensor2 = ttnn.from_torch(torch.tensor((1, 2), dtype=torch.bfloat16), device=device)
>>> output = ttnn.update_cache(tensor1, tensor2, batch_idx)