Forge Operations Reference
Welcome to the Forge Operations Reference. This page provides a comprehensive guide to all supported operations in the Forge framework.
Overview
Forge operations are organized into logical categories based on their functionality. Each operation is documented with detailed information including function signatures, parameters, examples, and usage notes.
Quick Navigation
- Elementwise Operations - Mathematical operations applied element-wise
- Convolution Operations - Convolution and related transformations
- Pooling Operations - Pooling and downsampling operations
- Normalization Operations - Batch and layer normalization
- Tensor Manipulation - Reshaping, slicing, and tensor operations
- Reduction Operations - Aggregation and reduction operations
- Linear Operations - Matrix multiplication and linear transformations
- Activation Functions - Non-linear activation functions
- Memory Operations - Cache and memory management operations
- Other Operations - Miscellaneous operations
Elementwise Operations
Mathematical operations applied element-wise.
| Operation | Description | Link |
|---|---|---|
| Abs | Computes the elementwise absolute value of the input tensor. | forge.op.Abs |
| Add | Elementwise add of two tensors | forge.op.Add |
| Atan | Elementwise arctangent (atan) | forge.op.Atan |
| BitwiseAnd | Bitwise and operation. | forge.op.BitwiseAnd |
| Cast | Cast | forge.op.Cast |
| Clip | Clips tensor values between min and max | forge.op.Clip |
| Concatenate | Concatenate tensors along axis | forge.op.Concatenate |
| Cosine | Elementwise cosine | forge.op.Cosine |
| Divide | Elementwise divide of two tensors | forge.op.Divide |
| Equal | Elementwise equal of two tensors | forge.op.Equal |
| Erf | Error function (erf) | forge.op.Erf |
| Exp | Exponent operation. | forge.op.Exp |
| Greater | Elementwise greater of two tensors | forge.op.Greater |
| GreaterEqual | Elementwise greater or equal of two tensors | forge.op.GreaterEqual |
| Heaviside | Elementwise max of two tensors | forge.op.Heaviside |
| Identity | Identity operation. | forge.op.Identity |
| IndexCopy | Copies the elements of value into operandA at index along dim | forge.op.IndexCopy |
| Less | Elementwise less of two tensors | forge.op.Less |
| LessEqual | Elementwise less or equal of two tensors | forge.op.LessEqual |
| Log | Log operation: natural logarithm of the elements of operandA | forge.op.Log |
| LogicalAnd | Logical and operation. | forge.op.LogicalAnd |
| LogicalNot | Logical not operation. | forge.op.LogicalNot |
| Max | Elementwise max of two tensors | forge.op.Max |
| Min | Elementwise min of two tensors | forge.op.Min |
| Multiply | Elementwise multiply of two tensors | forge.op.Multiply |
| NotEqual | Elementwise equal of two tensors | forge.op.NotEqual |
| Pow | Pow operation: operandA to the power of exponent | forge.op.Pow |
| Power | OperandA to the power of OperandB | forge.op.Power |
| Reciprocal | Reciprocal operation. | forge.op.Reciprocal |
| Remainder | forge.op.Remainder | |
| Sine | Elementwise sine | forge.op.Sine |
| Sqrt | Square root. | forge.op.Sqrt |
| Stack | Stack tensors along new axis | forge.op.Stack |
| Subtract | Elementwise subtraction of two tensors | forge.op.Subtract |
| Where | forge.op.Where |
Convolution Operations
Convolution and related transformations.
| Operation | Description | Link |
|---|---|---|
| Conv2d | Conv2d transformation on input activations, with optional bias. | forge.op.Conv2d |
| Conv2dTranspose | Conv2dTranspose transformation on input activations, with optional bias. | forge.op.Conv2dTranspose |
Pooling Operations
Pooling and downsampling operations.
| Operation | Description | Link |
|---|---|---|
| AvgPool1d | Avgpool1d transformation on input activations | forge.op.AvgPool1d |
| AvgPool2d | Avgpool2d transformation on input activations | forge.op.AvgPool2d |
| MaxPool1d | MaxPool1d transformation on input activations | forge.op.MaxPool1d |
| MaxPool2d | Maxpool2d transformation on input activations | forge.op.MaxPool2d |
Normalization Operations
Batch and layer normalization.
| Operation | Description | Link |
|---|---|---|
| Batchnorm | Batch normalization. | forge.op.Batchnorm |
| Dropout | Dropout | forge.op.Dropout |
| Layernorm | Layer normalization. | forge.op.Layernorm |
| LogSoftmax | LogSoftmax operation. | forge.op.LogSoftmax |
| Softmax | Softmax operation. | forge.op.Softmax |
Tensor Manipulation
Reshaping, slicing, and tensor operations.
| Operation | Description | Link |
|---|---|---|
| AdvIndex | TM | forge.op.AdvIndex |
| Broadcast | TM | forge.op.Broadcast |
| ConstantPad | TM - Direct TTIR constant padding operation. | forge.op.ConstantPad |
| Downsample2d | Downsample 2D operation | forge.op.Downsample2d |
| Index | TM | forge.op.Index |
| Pad | TM | forge.op.Pad |
| PixelShuffle | Pixel shuffle operation. | forge.op.PixelShuffle |
| Repeat | Repeats this tensor along the specified dimensions. | forge.op.Repeat |
| RepeatInterleave | Repeat elements of a tensor. | forge.op.RepeatInterleave |
| Reshape | TM | forge.op.Reshape |
| Resize1d | Resize input activations, with default mode 'nearest' | forge.op.Resize1d |
| Resize2d | Resizes the spatial dimensions of a 2D input tensor using interpolation. | forge.op.Resize2d |
| Select | TM | forge.op.Select |
| Squeeze | TM | forge.op.Squeeze |
| Transpose | Tranpose X and Y (i.e. rows and columns) dimensions. | forge.op.Transpose |
| Unsqueeze | TM | forge.op.Unsqueeze |
| Upsample2d | Upsample 2D operation | forge.op.Upsample2d |
Reduction Operations
Aggregation and reduction operations.
| Operation | Description | Link |
|---|---|---|
| Argmax | Argmax | forge.op.Argmax |
| ReduceAvg | Reduce by averaging along the given dimension | forge.op.ReduceAvg |
| ReduceMax | Reduce by taking maximum along the given dimension | forge.op.ReduceMax |
| ReduceSum | Reduce by summing along the given dimension | forge.op.ReduceSum |
Linear Operations
Matrix multiplication and linear transformations.
| Operation | Description | Link |
|---|---|---|
| Matmul | Matrix multiplication transformation on input activations, with optional bias. y... | forge.op.Matmul |
Activation Functions
Non-linear activation functions.
| Operation | Description | Link |
|---|---|---|
| Gelu | GeLU | forge.op.Gelu |
| LeakyRelu | Leaky ReLU | forge.op.LeakyRelu |
| Relu | Applies the Rectified Linear Unit (ReLU) activation function elementwise. | forge.op.Relu |
| Sigmoid | Sigmoid | forge.op.Sigmoid |
| Tanh | Tanh operation. | forge.op.Tanh |
Memory Operations
Cache and memory management operations.
| Operation | Description | Link |
|---|---|---|
| FillCache | FillCache op writes the input into the cache tensor starting at the specified up... | forge.op.FillCache |
| UpdateCache | UpdateCache writes a single token (S=1) slice into the cache tensor on specified... | forge.op.UpdateCache |
Other Operations
Miscellaneous operations.
| Operation | Description | Link |
|---|---|---|
| Constant | Op representing user-defined constant | forge.op.Constant |
| CumSum | Cumulative sum operation. | forge.op.CumSum |
| Embedding | Embedding lookup | forge.op.Embedding |
Documentation Structure
Each operation documentation page includes:
- Overview: Brief description of what the operation does
- Function Signature: Python API signature with type hints
- Parameters: Detailed parameter descriptions with types and defaults
- Returns: Return value description
- Mathematical Definition: Mathematical formula (where applicable)
- Related Operations: Links to related operations
This documentation is automatically generated from operation definitions in forge/forge/op/*.py. For the most up-to-date information, refer to the source code.