Running Performance Benchmark Tests
You can use forge/test/benchmark/benchmark.py to run performance benchmark tests:
python forge/test/benchmark/benchmark.py [options]
Available Options:
| Option | Short | Type | Default | Description |
|---|---|---|---|---|
--model | -m | string | required | Model to benchmark (e.g. bert, mnist_linear). The test file name without .py extension |
--config | -c | string | None | Model configuration to benchmark (e.g. tiny, base, large) |
--training | -t | flag | False | Benchmark training mode |
--batch_size | -bs | integer | 1 | Batch size, number of samples to process at once |
--loop_count | -lp | integer | 1 | Number of times to run the benchmark |
--input_size | -isz | integer | None | Input size of the input sample (if model supports variable input size) |
--hidden_size | -hs | integer | None | Hidden layer size (if model supports variable hidden size) |
--output | -o | string | None | Output JSON file to write results to. Results will be appended if file exists |
--task | -ts | string | "na" | Task to benchmark (e.g. classification, segmentation) |
--data_format | -df | string | "float32" | Data format (e.g. float32, bfloat16) |
Example:
python forge/test/benchmark/benchmark.py -m mobilenetv2_basic -ts classification -bs 8 -df bfloat16 -lp 32 -o forge-fe-benchmark-e2e-mobilenetv2_basic.json
Alternatively, you can run specific model tests using pytest:
pytest [model_path]
Example:
pytest -svv forge/test/benchmark/benchmark/models/yolo_v8.py