ttnn.kv_cache.update_cache_for_token_

ttnn.kv_cache.update_cache_for_token_ = Operation(python_fully_qualified_name='ttnn.kv_cache.update_cache_for_token_', function=<ttnn._ttnn.operations.kv_cache.update_cache_for_token__t object>, preprocess_golden_function_inputs=<function default_preprocess_golden_function_inputs>, golden_function=None, postprocess_golden_function_outputs=<function default_postprocess_golden_function_outputs>, is_cpp_operation=True, is_experimental=False)

Updates the cache tensor in-place with values from input at update_index and batch_offset.

Parameters:

• cache (ttnn.Tensor) – the cache tensor to be written to.

• token (ttnn.Tensor) – the token tensor to be written to the cache.

• update_index (int) – the index into the cache tensor.

• batch_offset (int) – the batch_offset into the cache tensor.

Returns:

ttnn.Tensor – the output tensor.