APIs

Device

ttnn.open_device	int, l1_small_size: int, trace_region_size: int, dispatch_core_config: ttnn.device.DispatchCoreConfig, worker_l1_size: int)
ttnn.close_device	Close the device and remove it from the device cache.
ttnn.manage_device	Context manager for opening and closing a device.
ttnn.synchronize_device	ttnn._ttnn.multi_device.MeshDevice, cq_id: ttnn._ttnn.types.QueueId | None = None, sub_device_ids: collections.abc.Sequence[ttnn._ttnn.device.SubDeviceId] = []) -> None
ttnn.SetDefaultDevice	ttnn._ttnn.multi_device.MeshDevice | None = None) -> None
ttnn.GetDefaultDevice	GetDefaultDevice() -> ttnn._ttnn.multi_device.MeshDevice
ttnn.pad_to_tile_shape	collections.abc.Sequence[int]) -> list[int]

Memory Config

ttnn.create_sharded_memory_config

Creates a MemoryConfig object with a sharding spec, required for sharded ops.

Operations

Core

ttnn.as_tensor	Converts the torch.Tensor tensor into a ttnn.Tensor.
ttnn.copy_device_to_host_tensor	copy_device_to_host_tensor(device_tensor: ttnn._ttnn.tensor.Tensor, host_tensor: ttnn._ttnn.tensor.Tensor, blocking: bool = True, cq_id: ttnn._ttnn.types.QueueId | None = None) -> None
ttnn.copy_host_to_device_tensor	copy_host_to_device_tensor(host_tensor: ttnn._ttnn.tensor.Tensor, device_tensor: ttnn._ttnn.tensor.Tensor, cq_id: ttnn._ttnn.types.QueueId | None = None) -> None
ttnn.deallocate	Releases the resources for ttnn.Tensor tensor explicitly.
ttnn.dump_tensor	Dump tensor to a file.
ttnn.from_device	Copies the ttnn.Tensor tensor to the host.
ttnn.from_torch	Converts the torch.Tensor tensor into a ttnn.Tensor.
ttnn.get_device_tensors	ttnn._ttnn.tensor.Tensor) -> list[ttnn._ttnn.tensor.Tensor]
ttnn.load_tensor	Load tensor from a file.
ttnn.reallocate	reallocate(tensor: ttnn._ttnn.tensor.Tensor, memory_config: ttnn._ttnn.tensor.MemoryConfig | None = None) -> ttnn._ttnn.tensor.Tensor
ttnn.split_work_to_cores	ttnn._ttnn.tensor.CoreCoord, units_to_divide: int, row_wise: bool = False) -> tuple[int, ttnn._ttnn.tensor.CoreRangeSet, ttnn._ttnn.tensor.CoreRangeSet, ttnn._ttnn.tensor.CoreRangeSet, int, int] split_work_to_cores(core_grid: ttnn._ttnn.tensor.CoreRangeSet, units_to_divide: int, row_wise: bool = False) -> tuple[int, ttnn._ttnn.tensor.CoreRangeSet, ttnn._ttnn.tensor.CoreRangeSet, ttnn._ttnn.tensor.CoreRangeSet, int, int]
ttnn.to_device	Copies the ttnn.Tensor tensor to the tt_lib.device.MeshDevice.
ttnn.to_dtype	Converts a tensor to the desired dtype
ttnn.to_layout	Organizes the ttnn.Tensor tensor into either ttnn.ROW_MAJOR_LAYOUT or ttnn.TILE_LAYOUT.
ttnn.to_memory_config	Converts a tensor to the desired memory configuration.
ttnn.to_torch	Converts the ttnn.Tensor tensor into a torch.Tensor.
ttnn.typecast	Applies typecast to input_tensor.

Tensor Creation

ttnn.arange	Creates a tensor with values ranging from start (inclusive) to end (exclusive) with a specified step size.
ttnn.bernoulli	Generates a tensor to draw binary random numbers (0 or 1) from a Bernoulli distribution.
ttnn.complex_tensor	Create a complex tensor from real and imaginary part tensors.
ttnn.empty	Creates a device tensor with uninitialized values of the specified shape, data type, layout, and memory configuration.
ttnn.empty_like	Creates a new tensor with the same shape as the given reference, but without initializing its values.
ttnn.from_buffer	Creates a device tensor with values from a buffer of the specified, data type, layout, and memory configuration.
ttnn.full	Creates a tensor of the specified shape and fills it with the specified scalar value.
ttnn.full_like	Creates a tensor of the same shape as the input tensor and fills it with the specified scalar value.
ttnn.index_fill	index_fill(input: Tensor, dim: uint32, index: Tensor, value: int or float, memory_config: MemoryConfig) -> Tensor Create or fill a tensor with the given value, with the specified memory_config.
ttnn.ones	Creates a tensor with the specified shape and fills it with the value of 1.0.
ttnn.ones_like	Creates a tensor of the same shape as the input tensor and fills it with the value of 1.0.
ttnn.rand	Generates a tensor with the given shape, filled with random values from a uniform distribution.
ttnn.uniform	Update in-place the input tensor with values drawn from the continuous uniform distribution 1 / (to - from).
ttnn.zeros	Creates a tensor with the specified shape and fills it with the value of 0.0.
ttnn.zeros_like	Creates a tensor of the same shape as the input tensor and fills it with the value of 0.0.

Matrix Multiplication

ttnn.matmul	Returns the matrix product of two tensors.
ttnn.linear	Returns the linear transformation of the inputs.
ttnn.addmm	Returns a matrix products of tensors mat1_tensor and mat2_tensor.
ttnn.sparse_matmul	Returns the matrix product of two tensors.

ttnn.MatmulMultiCoreReuseProgramConfig	Configuration class for multi-core reusable matmul operations.
ttnn.MatmulMultiCoreReuseMultiCastProgramConfig	The "2D" matmul program config is used for block sharded tensors, and general interleaved tensors.
ttnn.MatmulMultiCoreReuseMultiCast1DProgramConfig	Configuration class for 1D multicast matmul operations with advanced features.
ttnn.MatmulMultiCoreReuseMultiCastDRAMShardedProgramConfig	This program config is a specialized config for very narrow tensors stored in DRAM.

Pointwise Unary

ttnn.abs	Applies abs to input_tensor element-wise.
ttnn.acos	Applies acos to input_tensor element-wise.
ttnn.acosh	Applies acosh to input_tensor element-wise.
ttnn.alt_complex_rotate90	Applies alt_complex_rotate90 to input_tensor element-wise.
ttnn.angle	Performs complex operations for angle of input_tensor.
ttnn.asin	Applies asin to input_tensor element-wise.
ttnn.asinh	Applies asinh to input_tensor element-wise.
ttnn.atan	Applies atan to input_tensor element-wise.
ttnn.atanh	Performs atanh function on input_tensor.
ttnn.bitcast	Bitcast reinterprets the bit pattern without conversion (unlike typecast which converts values).
ttnn.bitwise_left_shift	Perform bitwise_left_shift operation on input_tensor_a by input_tensor_b and returns the tensor with the same layout as input_tensor_a.
ttnn.bitwise_not	Applies bitwise_not to input_tensor element-wise.
ttnn.bitwise_right_shift	Perform bitwise_right_shift operation on input_tensor_a by input_tensor_b and returns the tensor with the same layout as input_tensor_a.
ttnn.cbrt	Applies cbrt to input_tensor element-wise.
ttnn.ceil	Applies ceil to input_tensor element-wise.
ttnn.celu	Applies celu to input_tensor element-wise with alpha.
ttnn.clamp	Applies clamp to input_tensor element-wise.
ttnn.clip	Performs clip function on input_tensor, min, max.
ttnn.clone	Clones the input tensor, creating a copy with the specified memory configuration and converting its data type to dtype.
ttnn.conj	Returns complex conjugate value of complex tensor input_tensor.
ttnn.cos	Applies cos to input_tensor element-wise.
ttnn.cosh	Performs cosh function on input_tensor.
ttnn.deg2rad	Applies deg2rad to input_tensor element-wise.
ttnn.digamma	Performs digamma function on input_tensor.
ttnn.eqz	Applies eqz to input_tensor element-wise.
ttnn.erf	Applies erf to input_tensor element-wise.
ttnn.erfc	Applies erfc to input_tensor element-wise.
ttnn.erfinv	Applies erfinv to input_tensor element-wise.
ttnn.exp	Applies exp to input_tensor element-wise.
ttnn.exp2	Applies exp2 to input_tensor element-wise.
ttnn.experimental.dropout	Applies dropout to input_tensor element-wise.
ttnn.elu	Performs elu function on input_tensor, alpha.
ttnn.expm1	Applies expm1 to input_tensor element-wise.
ttnn.fill	Applies fill to input_tensor element-wise with fill_value.
ttnn.floor	Applies floor to input_tensor element-wise.
ttnn.frac	Applies frac to input_tensor element-wise.
ttnn.geglu	Applies geglu to input_tensor element-wise.
ttnn.gelu	Applies gelu to input_tensor element-wise.
ttnn.gez	Applies gez to input_tensor element-wise.
ttnn.glu	Applies glu to input_tensor element-wise.
ttnn.gtz	Applies gtz to input_tensor element-wise.
ttnn.hardmish	Applies hardmish to input_tensor element-wise.
ttnn.hardshrink	Performs hardshrink function on input_tensor, lambd.
ttnn.hardsigmoid	Applies hardsigmoid to input_tensor element-wise.
ttnn.hardswish	Applies hardswish to input_tensor element-wise.
ttnn.hardtanh	Performs hardtanh function on input_tensor, min_val, max_val.
ttnn.heaviside	Applies heaviside to input_tensor element-wise with value.
ttnn.i0	Applies i0 to input_tensor element-wise.
ttnn.i1	Applies i1 to input_tensor element-wise.
ttnn.identity	Returns a copy of the input_tensor; useful for profiling the SFPU.
ttnn.imag	Performs complex operations for imag of input_tensor.
ttnn.is_imag	Returns boolean tensor if value of input_tensor is imag.
ttnn.is_real	Returns boolean tensor if value of input_tensor is real.
ttnn.isfinite	Applies isfinite to input_tensor element-wise.
ttnn.isinf	Applies isinf to input_tensor element-wise.
ttnn.isnan	Applies isnan to input_tensor element-wise.
ttnn.isneginf	Applies isneginf to input_tensor element-wise.
ttnn.isposinf	Applies isposinf to input_tensor element-wise.
ttnn.leaky_relu	Applies leaky_relu to input_tensor element-wise with negative_slope.
ttnn.lez	Applies lez to input_tensor element-wise.
ttnn.lgamma	Performs lgamma function on input_tensor.
ttnn.log	Applies log to input_tensor element-wise.
ttnn.log10	Applies log10 to input_tensor element-wise.
ttnn.log1p	Applies log1p to input_tensor element-wise.
ttnn.log2	Applies log2 to input_tensor element-wise.
ttnn.log_sigmoid	Applies log_sigmoid to input_tensor element-wise.
ttnn.logical_left_shift	Perform logical_left_shift operation on input_tensor_a by input_tensor_b and returns the tensor with the same layout as input_tensor_a.
ttnn.logical_not	Applies logical_not to input_tensor element-wise.
ttnn.logical_not_	Performs logical_not inplace function on input_tensor.
ttnn.logical_right_shift	Perform logical_right_shift operation on input_tensor_a by input_tensor_b and returns the tensor with the same layout as input_tensor_a.
ttnn.logit	Performs logit function on input_tensor, eps.
ttnn.ltz	Applies ltz to input_tensor element-wise.
ttnn.mish	Applies mish to input_tensor element-wise.
ttnn.multigammaln	Performs multigammaln function on input_tensor.
ttnn.neg	Applies neg to input_tensor element-wise.
ttnn.nez	Applies nez to input_tensor element-wise.
ttnn.normalize_global	Performs normalize_global function on input_tensor.
ttnn.normalize_hw	Performs normalize_hw function on input_tensor.
ttnn.polar	Perform an polar to Cartesian transformation on input_tensor, input_tensor.real(r), input_tensor.imag(theta) into x + i*y generating a complex tensor.
ttnn.polygamma	Performs polygamma function on input_tensor, decimals.
ttnn.prelu	Perform an eltwise-prelu operation.
ttnn.rad2deg	Applies rad2deg to input_tensor element-wise.
ttnn.rdiv	Performs the element-wise division of a scalar value by a tensor input and rounds the result using rounding_mode.
ttnn.real	Performs complex operations for real of input_tensor.
ttnn.reciprocal	Applies reciprocal to input_tensor element-wise.
ttnn.reglu	Applies reglu to input_tensor element-wise.
ttnn.relu	Applies relu to input_tensor element-wise.
ttnn.relu6	Applies relu6 to input_tensor element-wise.
ttnn.relu_max	Applies relu_max to input_tensor element-wise with upper_limit.
ttnn.relu_min	Applies relu_min to input_tensor element-wise with lower_limit.
ttnn.remainder	Performs an eltwise-modulus operation.
ttnn.round	Applies round to input_tensor element-wise with decimals.
ttnn.rsqrt	Applies rsqrt to input_tensor element-wise.
ttnn.selu	Performs selu function on input_tensor, scale, alpha.
ttnn.sigmoid	Applies sigmoid to input_tensor element-wise.
ttnn.sigmoid_accurate	Applies sigmoid_accurate to input_tensor element-wise.
ttnn.sign	Applies sign to input_tensor element-wise.
ttnn.signbit	Applies signbit to input_tensor element-wise.
ttnn.silu	Applies silu to input_tensor element-wise.
ttnn.sin	Applies sin to input_tensor element-wise.
ttnn.sinh	Performs sinh function on input_tensor.
ttnn.softplus	Applies softplus to input_tensor element-wise.
ttnn.softshrink	Performs softshrink function on input_tensor, lambd.
ttnn.softsign	Applies softsign to input_tensor element-wise.
ttnn.sqrt	Applies sqrt to input_tensor element-wise.
ttnn.square	Applies square to input_tensor element-wise.
ttnn.std_hw	Computes the standard deviation across the height (H) and width (W) dimensions for each batch and channel.
ttnn.swiglu	Applies swiglu to input_tensor element-wise.
ttnn.swish	Applies swish to input_tensor element-wise.
ttnn.tan	Applies tan to input_tensor element-wise.
ttnn.tanh	Applies tanh to input_tensor element-wise.
ttnn.tanhshrink	Applies tanhshrink to input_tensor element-wise.
ttnn.threshold	Performs threshold function on input_tensor, threshold, value.
ttnn.tril	Performs tril function on input_tensor, diagonal.
ttnn.triu	Performs triu function on input_tensor, diagonal.
ttnn.trunc	Applies trunc to input_tensor element-wise.
ttnn.unary_chain	Applies unary_chain to input_tensor element-wise.
ttnn.var_hw	Computes the variance across the height (H) and width (W) dimensions for each batch and channel.

Pointwise Binary

ttnn.add	Adds input_tensor_a to input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.add_	Adds input_tensor_a to input_tensor_b and returns the tensor with the same layout as input_tensor_a in-place
ttnn.addalpha	Computes addalpha for input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.atan2	Computes atan2 input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.bias_gelu	Computes bias_gelu of input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.bias_gelu_	Performs bias_gelu in-place operation on input_a and input_b and returns the tensor with the same layout as input_tensor
ttnn.bitwise_and	Perform bitwise_and operation on input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.bitwise_or	Perform bitwise_or operation on input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.bitwise_xor	Perform bitwise_xor operation on input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.div	Divides input_tensor_a by input_tensor_b and returns a tensor with the same layout as input_tensor_a
ttnn.div_no_nan	Computes div_no_nan for input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.divide	Divides input_tensor_a by input_tensor_b and returns a tensor with the same layout as input_tensor_a
ttnn.divide_	Performs division in-place operation on input_a and input_b and returns the tensor with the same layout as input_tensor
ttnn.eq	Compares if input_tensor_a is equal to input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.eq_	Performs Equal to in-place operation on input_a and input_b and returns the tensor with the same layout as input_tensor
ttnn.floor_div	Computes floor division for input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.fmod	Performs an eltwise-fmod operation.
ttnn.gcd	Computes Greatest common divisor of input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a.
ttnn.ge	Compares if input_tensor_a is greater than or equal to input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.ge_	Performs Greater than or equal to in-place operation on input_a and input_b and returns the tensor with the same layout as input_tensor
ttnn.gt	Compares if input_tensor_a is greater than input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.gt_	Performs Greater than in-place operation on input_a and input_b and returns the tensor with the same layout as input_tensor
ttnn.hypot	Computes hypot input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.isclose	Computes isclose for input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.lcm	Computes Least common multiple of input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a.
ttnn.ldexp	Computes ldexp of input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.ldexp_	Performs ldexp in-place operation on input_a and input_b and returns the tensor with the same layout as input_tensor
ttnn.le	Compares if input_tensor_a is less than or equal to input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.le_	Performs Less than or equal to in-place operation on input_a and input_b and returns the tensor with the same layout as input_tensor
ttnn.logaddexp	Computes logaddexp of input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.logaddexp2	Computes logaddexp2 of input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.logaddexp2_	Performs logaddexp2 in-place operation on input_a and input_b and returns the tensor with the same layout as input_tensor
ttnn.logaddexp_	Performs logaddexp in-place operation on input_a and input_b and returns the tensor with the same layout as input_tensor
ttnn.logical_and	Computes logical AND of input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.logical_and_	Computes inplace logical AND of input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.logical_or	Computes logical OR of input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.logical_or_	Computes inplace logical OR of input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.logical_xor	Compute logical_xor input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.logical_xor_	Computes inplace logical XOR of input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.lt	Compares if input_tensor_a is less than input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.lt_	Performs Less than in-place operation on input_a and input_b and returns the tensor with the same layout as input_tensor
ttnn.maximum	Computes maximum for input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.minimum	Computes minimum for input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.multiply	Multiplies input_tensor_a by input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.multiply_	Multiplies input_tensor_a by input_tensor_b and returns the tensor with the same layout as input_tensor_a in-place
ttnn.ne	Compares if input_tensor_a is not equal to input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.ne_	Performs Not equal to in-place operation on input_a and input_b and returns the tensor with the same layout as input_tensor
ttnn.nextafter	Computes nextafter input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.outer	Computes outer for input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.polyval	Computes polyval of all elements of input_tensor_a with coefficients coeffs and returns the tensor with the same layout as input_tensor_a
ttnn.pow	Perform element-wise pow operation on input_tensor with exponent.
ttnn.remainder	Performs an eltwise-modulus operation.
ttnn.rpow	Applies rpow to input_tensor element-wise with exponent.
ttnn.rsub	Subtracts input_tensor_a from input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.rsub_	Subtracts input_a from input_b in-place and returns the tensor with the same layout as input_tensor
ttnn.scatter	Scatters the source tensor's values along a given dimension according to the index tensor.
ttnn.scatter_add	Scatters the source tensor's values along a given dimension according to the index tensor, adding source values associated with according repeated indices.
ttnn.squared_difference	Computes squared difference of input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.squared_difference_	Performs squared_difference in-place operation on input_a and input_b and returns the tensor with the same layout as input_tensor
ttnn.subalpha	Computes subalpha for input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a
ttnn.subtract	Subtracts input_tensor_b from input_tensor_a and returns the tensor with the same layout as input_tensor_a
ttnn.subtract_	Subtracts input_tensor_b from input_tensor_a and returns the tensor with the same layout as input_tensor_a in-place
ttnn.xlogy	Computes xlogy input_tensor_a and input_tensor_b and returns the tensor with the same layout as input_tensor_a

Pointwise Ternary

ttnn.addcdiv	Computes Addcdiv on input_tensor_a, input_tensor_b and input_tensor_c and returns the tensor with the same layout as input_tensor_a
ttnn.addcmul	Computes addcmul: output = input_a + value * input_b * input_c
ttnn.lerp	Computes Lerp on input, end and weight and returns the tensor with the same layout as input
ttnn.mac	Computes Mac on input_tensor_a, input_tensor_b and input_tensor_c and returns the tensor with the same layout as input_tensor_a
ttnn.where	Selects elements from true_value or false_value depending on the corresponding value in condition. For each element, if the corresponding entry in condition is 1, the output element is taken from true_value; otherwise, it is taken from false_value.

Quantization

ttnn.dequantize	De-quantize Operation
ttnn.quantize	Quantize Operation
ttnn.requantize	Re-quantize Operation

Losses

ttnn.l1_loss	Returns mean absolute error loss function for input_reference and input_prediction
ttnn.mse_loss	Returns mean squared error loss function for input_reference and input_prediction

Reduction

ttnn.argmax	Returns the indices of the maximum value of elements in the input_tensor.
ttnn.cumprod	Returns cumulative product of input along dimension dim For a given input of size N, the output will also contain N elements and be such that:
ttnn.cumsum	Returns cumulative sum of input along dimension dim For a given input of size N, the output will also contain N elements and be such that:
ttnn.ema	ttnn.ema(input: ttnn.Tensor, alpha: float, out: Optional[ttnn.Tensor] = None, memory_config: Optional[ttnn.MemoryConfig] = None) -> ttnn.Tensor
ttnn.manual_seed	Sets a seed to pseudo random number generators (PRNGs) on the specified device.
ttnn.max	Computes the max of the input tensor input_a along the specified dimension dim.
ttnn.mean	Computes the mean of the input tensor input_a along the specified dimension dim.
ttnn.min	Computes the min of the input tensor input_a along the specified dimension dim.
ttnn.moe	Returns the weight of the zero-th MoE expert.
ttnn.prod	Computes the product of all elements on specified dim of the input_tensor tensor.
ttnn.sampling	Samples from the input_values_tensor based on provided top-k and top-p constraints.
ttnn.std	Computes the std of the input tensor input_a along the specified dimension dim.
ttnn.sum	Computes the sum of the input tensor input_a along the specified dimension dim.
ttnn.topk	Returns the k largest or k smallest elements of the input_tensor along a given dimension dim.
ttnn.var	Computes the var of the input tensor input_a along the specified dimension dim.

Data Movement

ttnn.assign	Returns a new tensor which is a new copy of input tensor.
ttnn.bcast	Perform a binary elementwise operation math_op between tensors input_a and input_b, where values from tensor input_b are broadcast.
ttnn.chunk	Splits a tensor into multiple chunks along a specified dimension.
ttnn.concat	param input_tensor: the input tensors.
ttnn.copy	Copies the elements from input_a into input_b.
ttnn.expand	Returns a new tensor where singleton dimensions are expanded to a larger side.
ttnn.fill_implicit_tile_padding	Fills the implicit padding of a tiled input tensor with the specified value.
ttnn.fill_ones_rm	Same as fill_rm, but val_hi is set to 1 and val_lo is 0.
ttnn.fill_rm	Generates an NCHW row-major tensor and fill it with high values up to hOnes, wOnes in each HW tile with the rest padded with high values.
ttnn.fold	Fold TT Tensor.
ttnn.gather	The gather operation extracts values from the input tensor based on indices provided in the index tensor along a specified dimension.
ttnn.indexed_fill	Replaces batch of input in input_b denoted by batch_ids into input_a.
ttnn.interleaved_to_sharded	Converts a tensor from interleaved to sharded memory layout
ttnn.interleaved_to_sharded_partial	Converts a partial tensor from interleaved to sharded memory layout
ttnn.moe_expert_token_remap	Remap MoE CCL Metadata from global experts to local device experts
ttnn.moe_routing_remap	Remap MoE routing weights to local device routing weights.
ttnn.move	Moves the elements of the input tensor arg0 to a location in memory with specified memory layout.
ttnn.nonzero	Returns the number of elements (N) that are non-zero as well as a tensor of the same shape as input where the first N elements are the indices of non-zero elements.
ttnn.pad	Returns a padded tensor, with a specified value at the specified location.
ttnn.permute	Permutes the dimensions of the input tensor according to the specified permutation.
ttnn.repeat	Returns a new tensor filled with repetition of input input_tensor according to number of times specified in shape.
ttnn.repeat_interleave	Repeats elements of a tensor in the given dim.
ttnn.reshape	Note: for a 0 cost view, the following conditions must be met:
ttnn.reshape_on_device	Returns a tensor with the new shape of [W, Z, Y, X].
ttnn.reshard	Converts a tensor from one sharded layout to another sharded layout
ttnn.roll	Performs circular shifting of elements along the specified dimension(s).
ttnn.sharded_to_interleaved	Converts a tensor from sharded to interleaved memory layout
ttnn.sharded_to_interleaved_partial	Converts a partial tensor from sharded_to_interleaved memory layout
ttnn.slice	Returns a sliced tensor.
ttnn.sort	Sorts the elements of the input tensor along the specified dimension in ascending order by default.
ttnn.split	Returns a tensor that is in num_splits ways on dim.
ttnn.squeeze	Returns a tensor with the specified dimensions squeezed.
ttnn.stack	Stacks tensors along a new dimension.
ttnn.tilize	Changes data layout of input tensor to TILE.
ttnn.tilize_with_val_padding	Changes data layout of input tensor to TILE.
ttnn.tilize_with_zero_padding	Changes data layout of input tensor to TILE.
ttnn.transpose	Returns a tensor that is transposed along dims dim1 and dim2
ttnn.unsqueeze	Returns a tensor unsqueezed at the specified dimension
ttnn.unsqueeze_to_4D	unsqueeze_to_4D(tensor: ttnn._ttnn.tensor.Tensor) -> ttnn._ttnn.tensor.Tensor
ttnn.untilize	Changes data layout of input tensor to ROW_MAJOR.
ttnn.untilize_with_unpadding	Changes data layout of input tensor to ROW_MAJOR and unpads/removes elements from the tensor.
ttnn.view	This is a 0 cost view operation that returns the same tensor that was passed to it but with a new shape

Normalization

ttnn.batch_norm	Applies batch norm over each channel on input_tensor.
ttnn.group_norm	Computes group_norm over input_tensor.
ttnn.layer_norm	Computes layer norm over input_tensor.
ttnn.layer_norm_post_all_gather	This operation is used in conjunction with ttnn.layer_norm_pre_all_gather() to compute layer norm on a distributed setup, where layer norm is defined as:
ttnn.layer_norm_pre_all_gather	This operation is used in conjunction with ttnn.layer_norm_post_all_gather() to compute layer norm on a distributed setup, where layer norm is defined as:
ttnn.rms_norm	Computes RMS norm over input_tensor.
ttnn.rms_norm_post_all_gather	This operation is used in conjunction with ttnn.rms_norm_pre_all_gather() to compute RMS norm on a distributed setup, where RMS norm is defined as:
ttnn.rms_norm_pre_all_gather	This operation is used in conjunction with ttnn.rms_norm_post_all_gather() to compute RMS norm on a distributed setup, where RMS norm is defined as:
ttnn.scale_causal_mask_hw_dims_softmax_in_place	Specialized in-place operation for causal masked softmax with height-width dimension constraints.
ttnn.scale_mask_softmax	Computes a fused scale-mask-softmax operation along the last dimension of the input tensor.
ttnn.scale_mask_softmax_in_place	Computes a fused scale-mask-softmax operation along the last dimension in-place.
ttnn.softmax	Computes the softmax function over the specified dimension of the input tensor.
ttnn.softmax_in_place	Computes the softmax function along the last dimension of the input tensor in-place.

Normalization Program Configs

ttnn.SoftmaxDefaultProgramConfig	Default program configuration for Softmax operations.
ttnn.SoftmaxProgramConfig	Base program configuration variant for Softmax operations.
ttnn.SoftmaxShardedMultiCoreProgramConfig	Multi-core sharded program configuration for Softmax operations.

Transformer

ttnn.transformer.attention_softmax	Divides tensor by the square root of head_size, adds attention_mask (optionally) and computes softmax.
ttnn.transformer.attention_softmax_	In-Place divides tensor by the square root of head_size, adds attention_mask (optionally) and computes softmax.
ttnn.transformer.chunked_flash_mla_prefill	Chunked causal scaled dot product attention for processing long sequences in chunks.
ttnn.transformer.chunked_scaled_dot_product_attention	Chunked causal scaled dot product attention for processing long sequences in chunks.
ttnn.transformer.concatenate_heads	Takes in a tensor of shape [batch_size, num_heads, sequence_size, head_size], concatenates heads back along the width dimension and returns the tensor of shape [batch_size, sequence_size, num_heads * head_size]
ttnn.transformer.flash_mla_prefill	Causal MLA attention."
ttnn.transformer.flash_multi_latent_attention_decode	A version of scaled dot product attention specifically for decode.
ttnn.transformer.joint_scaled_dot_product_attention	JointAttention operation that efficiently performs non-causal attention over two sets of query, key, and value tensors.
ttnn.transformer.paged_flash_multi_latent_attention_decode	A version of scaled dot product attention specifically for decode.
ttnn.transformer.paged_scaled_dot_product_attention_decode	A version of scaled dot product attention specifically for decode.
ttnn.transformer.ring_distributed_scaled_dot_product_attention	Ring-distributed causal scaled dot product attention for multi-device execution.
ttnn.transformer.ring_joint_scaled_dot_product_attention	RingJointAttention operation that efficiently performs non-causal attention over two sets of query, key, and value tensors, where the first set is sharded across devices in the sequence dimension.
ttnn.transformer.scaled_dot_product_attention	Causal scaled dot product attention.
ttnn.transformer.scaled_dot_product_attention_decode	A version of scaled dot product attention specifically for decode.
ttnn.transformer.split_query_key_value_and_split_heads	Splits input_tensor of shape [batch_size, sequence_size, 3 * hidden_size] into 3 tensors (Query, Key, Value) of shape [batch_size, sequence_size, hidden_size].
ttnn.transformer.windowed_scaled_dot_product_attention	Windowed scaled dot product attention.

CCL

ttnn.all_broadcast	All-broadcast operation across devices.
ttnn.all_gather	All-gather operation across devices along a selected dimension and optional cluster axis.
ttnn.all_reduce	All-reduce operation across devices with Sum reduction.
ttnn.all_to_all_combine	All to all combine operation for combining the output tokens from the experts, based on the expert metadata and expert mapping tensors.
ttnn.all_to_all_dispatch	All to all dispatch operation for dispatching the input tokens to devices with the selected experts, based on the expert indices and expert mapping tensors.
ttnn.broadcast	Performs a broadcast operation from a sender device to all other mesh devices across a cluster axis.
ttnn.mesh_partition	Partitions the input tensor across the mesh such that each device has the i/num_devices-th partition of the input tensor along the specified dimension.
ttnn.point_to_point	Point-to-point send and receive operation. Send a tensor from one device to another.
ttnn.reduce_scatter	Reduce-scatter operation across devices along a selected dimension and optional cluster axis.
ttnn.reduce_to_root	Reduce-to-root operation. Performs sdpa tree reduction across 4 devices and stores the output on the root device only.

Embedding

ttnn.embedding

Retrieves word embeddings using input_tensor.

Convolution

ttnn.conv1d	Applies a 1D convolution over an input signal composed of several input planes.
ttnn.conv2d	Applies a 2D convolution over an input signal composed of several input planes.
ttnn.conv_transpose2d	Applies a 2D transposed convolution operator over an input image composed of several input planes.
ttnn.experimental.conv3d	Applies a 3D convolution over an input signal composed of several input planes.
ttnn.prepare_conv_bias	TTNN Conv2D applies preprocessing to the bias tensors before performing the convolution operation, to convert the bias into a format suitable for the operation.
ttnn.prepare_conv_transpose2d_bias	TTNN ConvTranspose2D applies preprocessing to the bias tensors before performing the convolution operation, to convert the bias into a format suitable for the operation.
ttnn.prepare_conv_transpose2d_weights	TTNN ConvTranspose2D applies preprocessing to the weights tensors before performing the conv_tranpose2D operation, to convert the weights into a format suitable for the operation.
ttnn.prepare_conv_weights	TTNN Conv2D applies preprocessing to the weights tensors before performing the convolution operation, to convert the weights into a format suitable for the operation.

ttnn.Conv2dConfig	Conv2DConfig is a structure that contains all the Tenstorrent device specific & implementation specific flags for the ttnn.conv1d(), ttnn.conv2d() and ttnn.conv_transpose2d() ops
ttnn.Conv2dSliceConfig	alias of Op2DSliceConfig

Pooling

ttnn.adaptive_avg_pool2d	Applies experimental adaptive average pooling to the input tensor.
ttnn.adaptive_max_pool2d	Applies experimental adaptive max pooling to the input tensor.
ttnn.avg_pool2d	Applies an average pool convolution to the input tensor.
ttnn.global_avg_pool2d	Applies global_avg_pool2d to input_tensor by performing a 2D adaptive average pooling over an input signal composed of several input planes.
ttnn.max_pool2d	Applies a max pool convolution to the input tensor.

Prefetcher

ttnn.dram_prefetcher

Asynchronously pre-fetch tensors from DRAM into the neighbouring L1 cores.

Vision

ttnn.grid_sample	Performs grid sampling on the input tensor using the provided sampling grid.
ttnn.upsample	Upsamples a given multi-channel 2D (spatial) data.

Generic

ttnn.generic_op

Executes a custom operation with user-defined kernels on the device.

KV Cache

ttnn.kv_cache.fill_cache_for_user_	Populates the cache tensor in-place with values sourced from input at batch_index.
ttnn.kv_cache.update_cache_for_token_	Updates the cache tensor in-place with values from input at update_index and batch_offset.
ttnn.fill_cache	Fills the cache tensor in place with the values from input at the specified batch_idx.
ttnn.update_cache	Updates the cache tensor in place with the values from input at the specified update_idx.

Backward operations

ttnn.abs_bw	Performs backward operations for abs on input_tensor with given grad_tensor
ttnn.acos_bw	Performs backward operations for inverse cosine (acos) on input_tensor with given grad_tensor.
ttnn.acosh_bw	Performs backward operations for inverse hyperbolic cosine (acosh) on input_tensor with given grad_tensor.
ttnn.add_bw	Performs backward operations for add of input_tensor_a and input_tensor_b or scalar with given grad_tensor.
ttnn.addalpha_bw	Performs backward operations for addalpha on input_tensor_b , input_tensor_a and alpha with given grad_tensor.
ttnn.addcdiv_bw	Performs backward operations for addcdiv of input_tensor_a, input_tensor_b and input_tensor_c with given grad_tensor.
ttnn.addcmul_bw	Performs backward operations for addcmul of input_tensor_a, input_tensor_b and input_tensor_c with given grad_tensor.
ttnn.angle_bw	Performs backward operations for complex angle function on input_tensor with given grad_tensor.
ttnn.asin_bw	Performs backward operations for inverse sine (asin) on input_tensor with given grad_tensor
ttnn.asinh_bw	Performs backward operations for inverse hyperbolic sine (asinh) on input_tensor with given grad_tensor
ttnn.assign_bw	Performs backward operations for assign of input_tensor_a, input_tensor_b with given grad_tensor.
ttnn.atan2_bw	Performs backward operations for atan2 of input_tensor_a and input_tensor_b with given grad_tensor.
ttnn.atan_bw	Performs backward operations for inverse tangent (atan) on input_tensor with given grad_tensor.
ttnn.atanh_bw	Performs backward operations for inverse hyperbolic tangent (atanh) on input_tensor with given grad_tensor
ttnn.bias_gelu_bw	Performs backward operations for bias_gelu on input_tensor_a and input_tensor_b or input_tensor and bias, with given grad_tensor using given approximate mode.
ttnn.ceil_bw	Performs backward operations for ceil on input_tensor with given grad_tensor
ttnn.celu_bw	Performs backward operations for celu on input_tensor, alpha, with given grad_tensor.
ttnn.clamp_bw	Performs backward operations for clamp on input_tensor, min, max with given grad_tensor.
ttnn.clip_bw	Performs backward operations for clip on input_tensor, min, max with given grad_tensor.
ttnn.concat_bw	Performs backward operations for concat on input_tensor_a and input_tensor_b with given grad_tensor.
ttnn.conj_bw	Performs backward operations for complex conj function on input_tensor with given grad_tensor.
ttnn.cos_bw	Performs backward operations for cosine on input_tensor with given grad_tensor.
ttnn.cosh_bw	Performs backward operations for hyperbolic cosine (cosh) on input_tensor with given grad_tensor.
ttnn.deg2rad_bw	Performs backward operations for degree to radian conversion (deg2rad) on input_tensor with given grad_tensor
ttnn.digamma_bw	Performs backward operations for digamma on input_tensor with given grad_tensor
ttnn.div_bw	Performs backward operations for divide on input_tensor, alpha or input_tensor_a, input_tensor_b, rounding_mode, with given grad_tensor.
ttnn.div_no_nan_bw	Performs backward operations for div_no_nan on input_tensor, scalar with given grad_tensor.
ttnn.elu_bw	Performs backward operations for elu on input_tensor, alpha, with given grad_tensor.
ttnn.embedding_bw	Returns the input gradients of the output gradients tensor with respect to the input indices.
ttnn.erf_bw	Performs backward operations for erf on input_tensor with given grad_tensor
ttnn.erfc_bw	Performs backward operations for erfc on input_tensor with given grad_tensor
ttnn.erfinv_bw	Performs backward operations for erfinv on input_tensor with given grad_tensor
ttnn.exp2_bw	Performs backward operations for exp2 on input_tensor with given grad_tensor
ttnn.exp_bw	Performs backward operations for exponential function on input_tensor with given grad_tensor.
ttnn.experimental.gelu_bw	Applies the backward pass of the GELU function using ttnn experimental kernels.
ttnn.expm1_bw	Performs backward operations for expm1 on input_tensor with given grad_tensor
ttnn.fill_bw	Performs backward operations for fill on input_tensor with given grad_tensor.
ttnn.fill_zero_bw	Performs backward operations for fill zero on input_tensor with given grad_tensor.
ttnn.floor_bw	Performs backward operations for floor on input_tensor with given grad_tensor
ttnn.fmod_bw	Performs backward operations for fmod of input_tensor_a, scalar or input_tensor_b with given grad_tensor.
ttnn.frac_bw	Performs backward operations for frac on input_tensor with given grad_tensor.
ttnn.gelu_bw	Performs backward operations for gelu on input_tensor, with given grad_tensor using given approximate mode.
ttnn.hardshrink_bw	Performs backward operations for hardshrink on input_tensor, lambd, with given grad_tensor.
ttnn.hardsigmoid_bw	Performs backward operations for hardsigmoid on input_tensor with given grad_tensor.
ttnn.hardswish_bw	Performs backward operations for hardswish on input_tensor with given grad_tensor
ttnn.hardtanh_bw	Performs backward operations for hardtanh activation function on input_tensor, min, max with given grad_tensor.
ttnn.hypot_bw	Performs backward operations for hypot of input_tensor_a and input_tensor_b with given grad_tensor.
ttnn.i0_bw	Performs backward operations for i0 on input_tensor with given grad_tensor.
ttnn.imag_bw	Performs backward operations for complex imaginary function on input_tensor with given grad_tensor.
ttnn.ldexp_bw	Performs backward operations for ldexp of input_tensor_a and input_tensor_b with given grad_tensor.
ttnn.leaky_relu_bw	Performs backward operations for leaky_relu on input_tensor, negative_slope, with given grad_tensor.
ttnn.lerp_bw	Performs backward operations for lerp of input_tensor_a, input_tensor_b and input_tensor_c or scalar with given grad_tensor.
ttnn.lgamma_bw	Performs backward operations for lgamma on input_tensor with given grad_tensor.
ttnn.log10_bw	Performs backward operations for log10 on input_tensor with given grad_tensor
ttnn.log1p_bw	Performs backward operations for log1p on input_tensor with given grad_tensor
ttnn.log2_bw	Performs backward operations for log2 on input_tensor with given grad_tensor.
ttnn.log_bw	Performs backward operations for logarithm on input_tensor with given grad_tensor
ttnn.log_sigmoid_bw	Performs backward operations for log sigmoid on input_tensor with given grad_tensor.
ttnn.logaddexp2_bw	Performs backward operations for logaddexp2 of input_tensor_a and input_tensor_b with given grad_tensor.
ttnn.logaddexp_bw	Performs backward operations for logaddexp of input_tensor_a and input_tensor_b with given grad_tensor.
ttnn.logit_bw	Performs backward operations for logit on input_tensor with given grad_tensor.
ttnn.logiteps_bw	Performs backward operations for logiteps on input_tensor, eps, with given grad_tensor.
ttnn.max_bw	Performs backward operations for maximum of input_tensor_a and input_tensor_b with given grad_tensor.
ttnn.min_bw	Performs backward operations for minimum of input_tensor_a and input_tensor_b with given grad_tensor.
ttnn.mul_bw	Performs backward operations for multiply on input_tensor_a, input_tensor_b, with given grad_tensor.
ttnn.multigammaln_bw	Performs backward operations for multivariate logarithmic gamma function (also referred to as mvlgamma) on input_tensor with given grad_tensor.
ttnn.neg_bw	Performs backward operations for neg on input_tensor with given grad_tensor.
ttnn.polar_bw	Performs backward operations for complex polar function on input_tensor with given grad_tensor.
ttnn.polygamma_bw	Performs backward operations for polygamma on input_tensor, scalar with given grad_tensor.
ttnn.pow_bw	Performs backward operations for power on input_tensor, exponent with given grad_tensor.
ttnn.prod_bw	Performs backward operations for prod on input_tensor with given grad_tensor along a particular dim.
ttnn.rad2deg_bw	Performs backward operations for radian to degree conversion (rad2deg) on input_tensor with given grad_tensor.
ttnn.rdiv_bw	Performs backward operations for Unary rdiv on input_tensor, scalar with given grad_tensor using given rounding_mode.
ttnn.real_bw	Performs backward operations for complex real function on input_tensor with given grad_tensor.
ttnn.reciprocal_bw	Performs backward operations for reciprocal on input_tensor with given grad_tensor
ttnn.relu6_bw	Performs backward operations for relu6 on input_tensor with given grad_tensor
ttnn.relu_bw	Performs backward operations for relu on input_tensor with given grad_tensor.
ttnn.remainder_bw	Performs backward operations for remainder of input_tensor_a, scalar or input_tensor_b with given grad_tensor.
ttnn.repeat_bw	Performs backward operations for repeat on input_tensor, with given grad_tensor using given shape.
ttnn.round_bw	Performs backward operations for round on input_tensor with given grad_tensor.
ttnn.rpow_bw	Performs backward operations for rpow on input_tensor, exponent with given grad_tensor.
ttnn.rsqrt_bw	Performs backward operations for reciprocal of square-root on input_tensor with given grad_tensor.
ttnn.rsub_bw	Performs backward operations for subraction of input_tensor_a from input_tensor_b with given grad_tensor (reversed order of subtraction operator).
ttnn.selu_bw	Performs backward operations for selu on input_tensor with given grad_tensor
ttnn.sigmoid_bw	Performs backward operations for sigmoid on input_tensor with given grad_tensor.
ttnn.sign_bw	Performs backward operations for sign on input_tensor with given grad_tensor.
ttnn.silu_bw	Performs backward operations for silu on input_tensor with given grad_tensor
ttnn.sin_bw	Performs backward operations for sin on input_tensor with given grad_tensor
ttnn.sinh_bw	Performs backward operations for hyperbolic sine (sinh) on input_tensor with given grad_tensor
ttnn.softplus_bw	Performs backward operations for softplus on input_tensor, beta, threshold with given grad_tensor.
ttnn.softshrink_bw	Performs backward operations for softshrink on input_tensor, lambd, with given grad_tensor.
ttnn.softsign_bw	Performs backward operations for softsign on input_tensor with given grad_tensor
ttnn.sqrt_bw	Performs backward operations for square-root on input_tensor with given grad_tensor.
ttnn.square_bw	Performs backward operations for square on input_tensor with given grad_tensor
ttnn.squared_difference_bw	Performs backward operations for squared_difference of input_tensor_a and input_tensor_b with given grad_tensor.
ttnn.sub_bw	Performs backward operations for subtract of input_tensor_a and input_tensor_b or scalar with given grad_tensor.
ttnn.subalpha_bw	Performs backward operations for subalpha of input_tensor_a and input_tensor_b with given grad_tensor.
ttnn.tan_bw	Performs backward operations for tan on input_tensor with given grad_tensor.
ttnn.tanh_bw	Performs backward operations for hyperbolic tangent (tanh) function on input_tensor with given grad_tensor.
ttnn.tanhshrink_bw	Performs backward operations for tanhshrink on input_tensor with given grad_tensor
ttnn.threshold_bw	Performs backward operations for threshold on input_tensor, threshold, value with given grad_tensor.
ttnn.trunc_bw	Performs backward operations for truncation on input_tensor with given grad_tensor.
ttnn.where_bw	Performs backward operations for where of input_tensor_a, input_tensor_b and input_tensor_c with given grad_tensor.
ttnn.xlogy_bw	Performs backward operations for xlogy of input_tensor_a and input_tensor_b with given grad_tensor.

Model Conversion

ttnn.model_preprocessing.preprocess_model	Preprocess modules and parameters of a given model.
ttnn.model_preprocessing.preprocess_model_parameters	Preprocess parameters of a given model.

Reports

ttnn.set_printoptions

str, sci_mode: Optional[str|bool], precision: Optional[int]

Operation Hooks

ttnn.register_pre_operation_hook	register_pre_operation_hook is a context manager that registers a pre-operation hook.
ttnn.register_post_operation_hook	register_post_operation_hook is a context manager that registers a post-operation hook.