Welcome to TT-NN documentation!

TTNN

• What is ttnn?
  • Key features of ttnn
• Getting Started
  • 1. Install and Build
  • 2. Explore our model demos
  • 3. TT-NN Tutorial: Multi-Head Attention (Simple)
  • 4. TT-NN Tutorial: Multi-Head Attention (Optimized)
  • Where to go from here
• Install
  • Installation Steps
    • Step 1. Driver & Firmware
    • Step 2. System-level dependencies
    • Step 3. Install and start using TT-NN and TT-Metalium!
      • Option 1: From source
      • Option 2: From wheel
      • Option 3: From Docker Release Image
    • Step 5. (Optional) Software dependencies for codebase contributions
• Using ttnn
  • Basic Examples
    • 1. Converting from and to torch tensor
    • 2. Running an operation on the device
    • 3. Using __getitem__ to slice the tensor
    • 4. Enabling program cache
    • 5. Debugging intermediate tensors
    • 6. Tracing the graph of operations
    • 7. Using tt_lib operation in ttnn
    • 8. Enabling Logging
    • 9. Supported Python Operators
    • 10. Changing the string representation of the tensor
    • 11. Visualize using Web Browser
    • 12. Register pre- and/or post-operation hooks
    • 13. Query all operations
    • 14. Falling back to torch
    • 15. Capturing graph of C++ functions, buffer allocations, etc
• Tensor
  • Shape
  • Layout
  • Data Type
  • Storage
  • Tensor Sharding
  • Memory Config
  • APIs
    • Shape
      • Shape.rank
      • Shape.to_rank()
      • Shape.with_tile_padding()
• APIs
  • Device
    • ttnn.open_device
      • open_device()
    • ttnn.close_device
      • close_device()
    • ttnn.manage_device
      • manage_device()
    • ttnn.synchronize_device
      • synchronize_device()
    • ttnn.SetDefaultDevice
      • SetDefaultDevice()
    • ttnn.GetDefaultDevice
      • GetDefaultDevice()
    • ttnn.format_input_tensor
      • format_input_tensor()
    • ttnn.format_output_tensor
      • format_output_tensor()
    • ttnn.pad_to_tile_shape
      • pad_to_tile_shape()
  • Memory Config
    • ttnn.create_sharded_memory_config
      • create_sharded_memory_config()
  • Operations
    • Core
      • ttnn.as_tensor
        • as_tensor()
      • ttnn.from_torch
        • from_torch()
      • ttnn.to_torch
        • to_torch()
      • ttnn.to_device
        • to_device()
      • ttnn.from_device
        • from_device()
      • ttnn.to_layout
        • to_layout()
      • ttnn.dump_tensor
        • dump_tensor()
      • ttnn.load_tensor
        • load_tensor()
      • ttnn.deallocate
        • deallocate()
      • ttnn.reallocate
        • reallocate()
      • ttnn.to_memory_config
        • to_memory_config()
    • Tensor Creation
      • ttnn.arange
        • arange()
      • ttnn.empty
        • empty()
      • ttnn.empty_like
        • empty_like()
      • ttnn.zeros
        • zeros()
      • ttnn.zeros_like
        • zeros_like()
      • ttnn.ones
        • ones()
      • ttnn.ones_like
        • ones_like()
      • ttnn.full
        • full()
      • ttnn.full_like
        • full_like()
    • Matrix Multiplication
      • ttnn.matmul
        • matmul()
      • ttnn.linear
        • linear()
    • Pointwise Unary
      • ttnn.abs
        • abs()
      • ttnn.acos
        • acos()
      • ttnn.acosh
        • acosh()
      • ttnn.asin
        • asin()
      • ttnn.asinh
        • asinh()
      • ttnn.atan
        • atan()
      • ttnn.atanh
        • atanh()
      • ttnn.bitwise_not
        • bitwise_not()
      • ttnn.bitwise_left_shift
        • bitwise_left_shift()
      • ttnn.bitwise_right_shift
        • bitwise_right_shift()
      • ttnn.cbrt
        • cbrt()
      • ttnn.ceil
        • ceil()
      • ttnn.celu
        • celu()
      • ttnn.clamp
        • clamp()
      • ttnn.clip
        • clip()
      • ttnn.clone
        • clone()
      • ttnn.cos
        • cos()
      • ttnn.cosh
        • cosh()
      • ttnn.deg2rad
        • deg2rad()
      • ttnn.digamma
        • digamma()
      • ttnn.experimental.dropout
        • dropout()
      • ttnn.elu
        • elu()
      • ttnn.eqz
        • eqz()
      • ttnn.erf
        • erf()
      • ttnn.erfc
        • erfc()
      • ttnn.erfinv
        • erfinv()
      • ttnn.exp
        • exp()
      • ttnn.exp2
        • exp2()
      • ttnn.expm1
        • expm1()
      • ttnn.fill
        • fill()
      • ttnn.floor
        • floor()
      • ttnn.frac
        • frac()
      • ttnn.geglu
        • geglu()
      • ttnn.gelu
        • gelu()
      • ttnn.glu
        • glu()
      • ttnn.gez
        • gez()
      • ttnn.gtz
        • gtz()
      • ttnn.hardshrink
        • hardshrink()
      • ttnn.hardsigmoid
        • hardsigmoid()
      • ttnn.hardswish
        • hardswish()
      • ttnn.hardtanh
        • hardtanh()
      • ttnn.heaviside
        • heaviside()
      • ttnn.i0
        • i0()
      • ttnn.identity
        • identity()
      • ttnn.isfinite
        • isfinite()
      • ttnn.isinf
        • isinf()
      • ttnn.isnan
        • isnan()
      • ttnn.isneginf
        • isneginf()
      • ttnn.isposinf
        • isposinf()
      • ttnn.leaky_relu
        • leaky_relu()
      • ttnn.lez
        • lez()
      • ttnn.lgamma
        • lgamma()
      • ttnn.log
        • log()
      • ttnn.log10
        • log10()
      • ttnn.log1p
        • log1p()
      • ttnn.log2
        • log2()
      • ttnn.log_sigmoid
        • log_sigmoid()
      • ttnn.logical_not
        • logical_not()
      • ttnn.logical_not_
        • logical_not_()
      • ttnn.logit
        • logit()
      • ttnn.ltz
        • ltz()
      • ttnn.mish
        • mish()
      • ttnn.multigammaln
        • multigammaln()
      • ttnn.neg
        • neg()
      • ttnn.nez
        • nez()
      • ttnn.normalize_global
        • normalize_global()
      • ttnn.normalize_hw
        • normalize_hw()
      • ttnn.polygamma
        • polygamma()
      • ttnn.prelu
        • prelu()
      • ttnn.rad2deg
        • rad2deg()
      • ttnn.rdiv
        • rdiv()
      • ttnn.reciprocal
        • reciprocal()
      • ttnn.reglu
        • reglu()
      • ttnn.relu
        • relu()
      • ttnn.relu_max
        • relu_max()
      • ttnn.relu_min
        • relu_min()
      • ttnn.relu6
        • relu6()
      • ttnn.remainder
        • remainder()
      • ttnn.round
        • round()
      • ttnn.rsqrt
        • rsqrt()
      • ttnn.selu
        • selu()
      • ttnn.sigmoid
        • sigmoid()
      • ttnn.sigmoid_accurate
        • sigmoid_accurate()
      • ttnn.sign
        • sign()
      • ttnn.signbit
        • signbit()
      • ttnn.silu
        • silu()
      • ttnn.sin
        • sin()
      • ttnn.sinh
        • sinh()
      • ttnn.softmax
        • softmax()
      • ttnn.softplus
        • softplus()
      • ttnn.softshrink
        • softshrink()
      • ttnn.softsign
        • softsign()
      • ttnn.sqrt
        • sqrt()
      • ttnn.square
        • square()
      • ttnn.swiglu
        • swiglu()
      • ttnn.swish
        • swish()
      • ttnn.tan
        • tan()
      • ttnn.tanh
        • tanh()
      • ttnn.tanhshrink
        • tanhshrink()
      • ttnn.threshold
        • threshold()
      • ttnn.tril
        • tril()
      • ttnn.triu
        • triu()
      • ttnn.trunc
        • trunc()
      • ttnn.unary_chain
        • unary_chain()
      • ttnn.clamp_bw
        • clamp_bw()
      • ttnn.clip_bw
        • clip_bw()
      • ttnn.hardtanh_bw
        • hardtanh_bw()
      • ttnn.threshold_bw
        • threshold_bw()
      • ttnn.softplus_bw
        • softplus_bw()
      • ttnn.rdiv_bw
        • rdiv_bw()
      • ttnn.pow_bw
        • pow_bw()
      • ttnn.exp_bw
        • exp_bw()
      • ttnn.tanh_bw
        • tanh_bw()
      • ttnn.sqrt_bw
        • sqrt_bw()
      • ttnn.multigammaln_bw
        • multigammaln_bw()
      • ttnn.lgamma_bw
        • lgamma_bw()
      • ttnn.fill_bw
        • fill_bw()
      • ttnn.hardsigmoid_bw
        • hardsigmoid_bw()
      • ttnn.cos_bw
        • cos_bw()
      • ttnn.acosh_bw
        • acosh_bw()
      • ttnn.acos_bw
        • acos_bw()
      • ttnn.atan_bw
        • atan_bw()
      • ttnn.rad2deg_bw
        • rad2deg_bw()
      • ttnn.frac_bw
        • frac_bw()
      • ttnn.trunc_bw
        • trunc_bw()
      • ttnn.log_sigmoid_bw
        • log_sigmoid_bw()
      • ttnn.fill_zero_bw
        • fill_zero_bw()
      • ttnn.i0_bw
        • i0_bw()
      • ttnn.tan_bw
        • tan_bw()
      • ttnn.sigmoid_bw
        • sigmoid_bw()
      • ttnn.rsqrt_bw
        • rsqrt_bw()
      • ttnn.neg_bw
        • neg_bw()
      • ttnn.relu_bw
        • relu_bw()
      • ttnn.logit_bw
        • logit_bw()
      • ttnn.hardshrink_bw
        • hardshrink_bw()
      • ttnn.softshrink_bw
        • softshrink_bw()
      • ttnn.leaky_relu_bw
        • leaky_relu_bw()
      • ttnn.elu_bw
        • elu_bw()
      • ttnn.celu_bw
        • celu_bw()
      • ttnn.rpow_bw
        • rpow_bw()
      • ttnn.floor_bw
        • floor_bw()
      • ttnn.round_bw
        • round_bw()
      • ttnn.log_bw
        • log_bw()
      • ttnn.relu6_bw
        • relu6_bw()
      • ttnn.abs_bw
        • abs_bw()
      • ttnn.silu_bw
        • silu_bw()
      • ttnn.selu_bw
        • selu_bw()
      • ttnn.square_bw
        • square_bw()
      • ttnn.prod_bw
        • prod_bw()
      • ttnn.hardswish_bw
        • hardswish_bw()
      • ttnn.tanhshrink_bw
        • tanhshrink_bw()
      • ttnn.atanh_bw
        • atanh_bw()
      • ttnn.asin_bw
        • asin_bw()
      • ttnn.asinh_bw
        • asinh_bw()
      • ttnn.sin_bw
        • sin_bw()
      • ttnn.sinh_bw
        • sinh_bw()
      • ttnn.log10_bw
        • log10_bw()
      • ttnn.log1p_bw
        • log1p_bw()
      • ttnn.erfc_bw
        • erfc_bw()
      • ttnn.ceil_bw
        • ceil_bw()
      • ttnn.softsign_bw
        • softsign_bw()
      • ttnn.cosh_bw
        • cosh_bw()
      • ttnn.logiteps_bw
        • logiteps_bw()
      • ttnn.log2_bw
        • log2_bw()
      • ttnn.sign_bw
        • sign_bw()
      • ttnn.div_no_nan_bw
        • div_no_nan_bw()
      • ttnn.exp2_bw
        • exp2_bw()
      • ttnn.expm1_bw
        • expm1_bw()
      • ttnn.reciprocal_bw
        • reciprocal_bw()
      • ttnn.digamma_bw
        • digamma_bw()
      • ttnn.erfinv_bw
        • erfinv_bw()
      • ttnn.erf_bw
        • erf_bw()
      • ttnn.deg2rad_bw
        • deg2rad_bw()
      • ttnn.polygamma_bw
        • polygamma_bw()
      • ttnn.gelu_bw
        • gelu_bw()
      • ttnn.repeat_bw
        • repeat_bw()
      • ttnn.real
        • real()
      • ttnn.imag
        • imag()
      • ttnn.angle
        • angle()
      • ttnn.is_imag
        • is_imag()
      • ttnn.is_real
        • is_real()
      • ttnn.polar_bw
        • polar_bw()
      • ttnn.imag_bw
        • imag_bw()
      • ttnn.real_bw
        • real_bw()
      • ttnn.angle_bw
        • angle_bw()
      • ttnn.conj_bw
        • conj_bw()
      • ttnn.conj
        • conj()
      • ttnn.polar
        • polar()
    • Pointwise Binary
      • ttnn.add
        • add()
      • ttnn.addalpha
        • addalpha()
      • ttnn.subalpha
        • subalpha()
      • ttnn.multiply
        • multiply()
      • ttnn.subtract
        • subtract()
      • ttnn.div
        • div()
      • ttnn.div_no_nan
        • div_no_nan()
      • ttnn.floor_div
        • floor_div()
      • ttnn.remainder
        • remainder()
      • ttnn.fmod
        • fmod()
      • ttnn.gcd
        • gcd()
      • ttnn.lcm
        • lcm()
      • ttnn.logical_and_
        • logical_and_()
      • ttnn.logical_or_
        • logical_or_()
      • ttnn.logical_xor_
        • logical_xor_()
      • ttnn.rpow
        • rpow()
      • ttnn.rsub
        • rsub()
      • ttnn.ldexp
        • ldexp()
      • ttnn.logical_and
        • logical_and()
      • ttnn.logical_or
        • logical_or()
      • ttnn.logical_xor
        • logical_xor()
      • ttnn.bitwise_and
        • bitwise_and()
      • ttnn.bitwise_or
        • bitwise_or()
      • ttnn.bitwise_xor
        • bitwise_xor()
      • ttnn.logaddexp
        • logaddexp()
      • ttnn.logaddexp2
        • logaddexp2()
      • ttnn.hypot
        • hypot()
      • ttnn.xlogy
        • xlogy()
      • ttnn.squared_difference
        • squared_difference()
      • ttnn.gt
        • gt()
      • ttnn.gt_
        • gt_()
      • ttnn.lt_
        • lt_()
      • ttnn.ge_
        • ge_()
      • ttnn.le_
        • le_()
      • ttnn.eq_
        • eq_()
      • ttnn.ne_
        • ne_()
      • ttnn.ge
        • ge()
      • ttnn.lt
        • lt()
      • ttnn.le
        • le()
      • ttnn.eq
        • eq()
      • ttnn.ne
        • ne()
      • ttnn.isclose
        • isclose()
      • ttnn.nextafter
        • nextafter()
      • ttnn.maximum
        • maximum()
      • ttnn.minimum
        • minimum()
      • ttnn.outer
        • outer()
      • ttnn.pow
        • pow()
      • ttnn.polyval
        • polyval()
      • ttnn.scatter
        • scatter()
      • ttnn.atan2
        • atan2()
      • ttnn.add_bw
        • add_bw()
      • ttnn.assign_bw
        • assign_bw()
      • ttnn.atan2_bw
        • atan2_bw()
      • ttnn.bias_gelu_bw
        • bias_gelu_bw()
      • ttnn.div_bw
        • div_bw()
      • ttnn.embedding_bw
        • embedding_bw()
      • ttnn.fmod_bw
        • fmod_bw()
      • ttnn.remainder_bw
        • remainder_bw()
      • ttnn.addalpha_bw
        • addalpha_bw()
      • ttnn.subalpha_bw
        • subalpha_bw()
      • ttnn.xlogy_bw
        • xlogy_bw()
      • ttnn.hypot_bw
        • hypot_bw()
      • ttnn.ldexp_bw
        • ldexp_bw()
      • ttnn.logaddexp_bw
        • logaddexp_bw()
      • ttnn.logaddexp2_bw
        • logaddexp2_bw()
      • ttnn.mul_bw
        • mul_bw()
      • ttnn.sub_bw
        • sub_bw()
      • ttnn.squared_difference_bw
        • squared_difference_bw()
      • ttnn.concat_bw
        • concat_bw()
      • ttnn.rsub_bw
        • rsub_bw()
      • ttnn.min_bw
        • min_bw()
      • ttnn.max_bw
        • max_bw()
    • Pointwise Ternary
      • ttnn.addcdiv
        • addcdiv()
      • ttnn.addcmul
        • addcmul()
      • ttnn.mac
        • mac()
      • ttnn.where
        • where()
      • ttnn.lerp
        • lerp()
      • ttnn.addcmul_bw
        • addcmul_bw()
      • ttnn.addcdiv_bw
        • addcdiv_bw()
      • ttnn.where_bw
        • where_bw()
      • ttnn.lerp_bw
        • lerp_bw()
    • Losses
      • ttnn.l1_loss
        • l1_loss()
      • ttnn.mse_loss
        • mse_loss()
    • Reduction
      • ttnn.max
        • max()
      • ttnn.mean
        • mean()
      • ttnn.min
        • min()
      • ttnn.std
        • std()
      • ttnn.sum
        • sum()
      • ttnn.var
        • var()
      • ttnn.argmax
        • argmax()
      • ttnn.prod
        • prod()
      • ttnn.topk
        • topk()
    • Data Movement
      • ttnn.concat
        • concat()
      • ttnn.nonzero
        • nonzero()
      • ttnn.pad
        • pad()
      • ttnn.permute
        • permute()
      • ttnn.reshape
        • reshape()
      • ttnn.repeat
        • repeat()
      • ttnn.repeat_interleave
        • repeat_interleave()
      • ttnn.slice
        • slice()
      • ttnn.tilize
        • tilize()
      • ttnn.tilize_with_val_padding
        • tilize_with_val_padding()
      • ttnn.fill_rm
        • fill_rm()
      • ttnn.fill_ones_rm
        • fill_ones_rm()
      • ttnn.untilize
        • untilize()
      • ttnn.untilize_with_unpadding
        • untilize_with_unpadding()
      • ttnn.untilize_with_halo_v2
        • untilize_with_halo_v2()
      • ttnn.indexed_fill
        • indexed_fill()
    • Normalization
      • ttnn.group_norm
        • group_norm()
      • ttnn.layer_norm
        • layer_norm()
      • ttnn.rms_norm
        • rms_norm()
      • ttnn.batch_norm
        • batch_norm()
    • Moreh Operations
      • ttnn.moreh_sum
        • moreh_sum()
    • Transformer
      • ttnn.transformer.split_query_key_value_and_split_heads
        • split_query_key_value_and_split_heads()
      • ttnn.transformer.concatenate_heads
        • concatenate_heads()
      • ttnn.transformer.attention_softmax
        • attention_softmax()
      • ttnn.transformer.attention_softmax_
        • attention_softmax_()
      • ttnn.experimental.rotary_embedding
        • rotary_embedding()
      • ttnn.transformer.scaled_dot_product_attention
        • scaled_dot_product_attention()
      • ttnn.transformer.scaled_dot_product_attention_decode
        • scaled_dot_product_attention_decode()
    • CCL
      • ttnn.all_gather
        • all_gather()
      • ttnn.reduce_scatter
        • reduce_scatter()
      • ttnn.experimental.all_reduce
        • all_reduce()
    • Embedding
      • ttnn.embedding
        • embedding()
    • Pooling
      • ttnn.global_avg_pool2d
        • global_avg_pool2d()
      • ttnn.max_pool2d
        • max_pool2d()
    • Vision
      • ttnn.upsample
        • upsample()
      • ttnn.downsample
        • downsample()
    • KV Cache
      • ttnn.kv_cache.fill_cache_for_user_
        • fill_cache_for_user_()
      • ttnn.kv_cache.update_cache_for_token_
        • update_cache_for_token_()
  • Model Conversion
    • ttnn.model_preprocessing.preprocess_model
      • preprocess_model()
    • ttnn.model_preprocessing.preprocess_model_parameters
      • preprocess_model_parameters()
  • Reports
    • ttnn.set_printoptions
      • set_printoptions()
  • Operation Hooks
    • ttnn.register_pre_operation_hook
      • register_pre_operation_hook()
    • ttnn.register_post_operation_hook
      • register_post_operation_hook()
• Tutorials
  • Tensor and Add Operation
    • Tensor and Add Operation
      • Creating a tensor
      • Host Storage: Borrowed vs Owned
      • Data Type
      • Layout
      • Device storage
      • Open the device
      • Initialize tensors a and b with random values using torch
      • Add tensor a and b
      • Inspect the output tensor of the add in ttnn
      • Convert to torch and inspect the attributes of the torch tensor
      • Close the device
  • Matmul Operation
    • Matrix Multiplication
      • Enable program cache
    • Configuration
      • Initialize tensors a and b with random values using torch
      • Matrix multiply tensor a and b
      • Inspect the layout of matrix multiplication output
      • Inspect the result of the matrix multiplication
      • Matrix multiply tensor a and b by using more performant config
      • Close the device
  • Multi-Head Attention
    • Multi-Head Attention
      • Enable program cache
      • Write Multi-Head Attention using ttnn
      • Configuration
      • Initialize activations and weights using torch
      • Convert activations and weights to ttnn
      • Run the first iteration of Multi-Head Attention
      • Run a subsequent iteration of Multi-Head Attention
      • Write optimized version of Multi-Head Attention
      • Pre-process the parameters of the optimized model
      • Run the first iteration of the optimized Multi-Head Attention
      • Run a subsequent iteration of the optimized Multi-Head Attention
      • Check that the output of the optimized version matches the output of the original implementation
      • Close the device
  • ttnn Tracer
    • Tracing ttnn operations and torch modules/functions
      • Trace torch functions
      • Trace torch functions and ttnn operations
      • Trace torch functions, torch modules and ttnn operations
      • Trace models written using ttnn
  • ttnn Profiling
    • Profiling ttnn operations
  • Resnet Basic Block
    • Resnet Block
      • Torch Module (from torchvision)
      • Create torch module and preprocess it to get ttnn parameters
      • Display the parameters of the module
      • Display the traced torch graph
      • Implement ttnn version of the module. Pass in the parameters into the constructor.
      • Run ttnn module and display the traced graph
  • Graphing Torch DiT_XL_2 With TTNN
    • Build a graph of a pytorch based model
      • Clone the library from https://github.com/facebookresearch/DiT.git
      • Download DiT-XL/2 Models
      • Sample from Pre-trained DiT Models and build the graph
      • Display the graph
• Onboarding New Functionality
• Converting torch Model to ttnn
  • Step 1 - Rewriting the Model
  • Step 2 - Switching to ttnn Operations
  • Step 3 - Optimizing the Model
  • More examples
• Adding New ttnn Operation
  • FAQ
    • What is a ttnn operation?
    • What steps are needed to add ttnn operation in C++?
    • What steps are needed to add ttnn operation in Python?
  • Example of Adding a new Device Operation
    • C++ Implementation
      • Step 1: Implement device operation
      • Step 2: Implement the operation in C++
    • Python Implementation
      • Step 1: Add Python binding
      • Step 2: (Optional) Add golden function for the operation in Python
• Profiling ttnn Operations
  • Perf Report Headers
  • profile_this description
• Dependencies
  • TT-LIB
    • Overview
      • Operation Infrastructure
        • New Device Operation
        • New Device Operation with a member
        • New Device Operation with Optional Input Tensors
        • New Device Operation with Optional Output Tensors
      • Profiler
      • Fast Dispatch
        • cpu()
      • Program Caching
      • Logs
    • TT-LIB API through tt_lib
      • Primary Operations
        • softmax()
        • softmax_backward()
        • softmin()
        • softmin_backward()
        • logsoftmax()
        • logsoftmax_backward()
        • mean()
        • mean_backward()
        • group_norm()
        • group_norm_backward()
        • norm()
        • norm_backward()
      • Enums
        • BcastOpMath
        • BcastOpDim
    • Fallback Operations
      • full()
      • tensor_slice()
      • reshape()
      • chunk()
      • conv2d()
      • group_norm()
      • layer_norm()
      • pad()
      • interpolate()
      • repeat()
      • repeat_interleave()
      • concat()
      • silu()
      • softmax()
      • Conv2d
      • BatchNorm2d
      • GroupNorm
      • LayerNorm
      • MaxPool2d
      • AdaptiveAvgPool2d
      • ceil
      • floor
      • trunc
      • unary_fmod
      • binary_fmod
      • bitwise_not
      • unary_bitwise_or
      • unary_bitwise_and
      • unary_bitwise_xor
      • binary_bitwise_or
      • binary_bitwise_and
      • binary_bitwise_xor
      • unary_bitwise_left_shift
      • unary_bitwise_right_shift
      • binary_bitwise_left_shift
      • binary_bitwise_right_shift
      • torch_argmax
      • torch_argmin
    • Experimental Operations
      • Fused Operations from tt_lib Mini-Graph Library
        • Linear()
        • Layernorm()
        • AddAndNorm()
      • Complex Operations (Type 2)
  • Tensor
    • Overview
    • Tensor Storage
    • Tensor API
      • Tensor
        • Tensor.__init__()
        • Tensor.buffer()
        • Tensor.device()
        • Tensor.get_dtype()
        • Tensor.get_layout()
        • Tensor.pad()
        • Tensor.pad_to_tile()
        • Tensor.storage_type()
        • Tensor.to()
        • Tensor.unpad()
        • Tensor.unpad_from_tile()
    • MemoryConfig
      • MemoryConfig
        • MemoryConfig.__init__()
    • Examples of converting between PyTorch Tensor and TT Tensor
      • Converting a PyTorch Tensor to a TT Tensor
      • Converting a TT Tensor to a PyTorch Tensor
  • Examples of Tensor and TT-LIB Use
    • Run one OP from TT-LIB on TT Accelerator device
    • Run TT-LIB and PyTorch OPs
    • Tensors with odd size of last dim
• Building and Uplifting Demos

Models

• Getting Started
  • Prerequisites
  • Running an example model
  • Next steps
• Performance
  • Prerequisites
  • Running a perf file
  • Running all the perf files

Resources

• Support
  • Reporting bugs, feature proposals, or support requests
  • Troubleshooting and debugging tips
  • Community
• Contributing as a developer

Indices and tables

• Index

• Module Index

• Search Page