TT-Forge-ONNX Documentation

TT-Forge-ONNX is a graph compiler designed to optimize and transform computational graphs for deep learning models on Tenstorrent AI hardware. It is an integral component of the TT-Forge project, built on top of the TT-MLIR backend.

Introduction

• Introduction
• Getting Started
  • Setup Options
  • Configuring Hardware
  • Installing a Wheel and Running an Example
  • Other Set up Options
  • Where to Go Next
• Getting Started with Docker
  • Configuring Hardware
  • Setting up the Docker Container
  • Running Models in Docker
  • Where to Go Next
• Getting Started with Building from Source
  • Configuring Your Hardware
  • Prerequisites
  • Building the Environment
  • Building the Docs
  • Build Cleanup
  • Useful Build Environment Variables
• Architecture Overview
  • Table of contents
  • TT-Forge Overview
  • TT-TVM Overview
  • TT-Forge-ONNX Overview

Project Setup

• Testing
  • Unit tests
  • End to end tests
  • Single operator E2E tests
• Pytest
  • Testing with multiple input sets
  • Marking specific parameters
  • Skipping tests
  • Marking tests as expected to fail
  • Avoid adding decorators inside tests
• Performance Benchmarks
  • Directory layout
  • Running benchmarks locally
  • CLI options
  • Adding a new benchmark model
• Tools
  • Pre-commit
  • Documentation
  • Gather Unique Ops Configuration
  • Cross Correlate Models and Ops and Export Model Variants Unique Op Configuration
  • Operations Documentation Generator

Operations Reference

• Forge Operations Reference
  • Overview
  • Quick Navigation
  • Elementwise Operations
  • Convolution Operations
  • Pooling Operations
  • Normalization Operations
  • Tensor Manipulation
  • Reduction Operations
  • Linear Operations
  • Activation Functions
  • Memory Operations
  • Other Operations
  • Operation Details

Dev Notes

• How to run standalone MLIR, based on generated Forge-ONNX MLIR graphs
• Verification
  • General Overview
  • Verify Config Overview
• Forge Operation Docstring Standard
  • Standard Docstring Structure
  • Required Sections
  • Optional Sections
  • Example: Complete Docstring
  • Parsing Notes
  • Best Practices