Experiments

This page provides an overview of the experiments included in this repository, detailing their organization.

Available Experiments

The following table provides an overview of different model and method combinations within various frameworks explored in this project.

Framework	Model	Method	Details
PyTorch	MLP	Full-model	README
PyTorch	CNN	Full-model	README
PyTorch	Llama 3.2 1B	LoRA	README
PyTorch	Llama 3.2 1B	Adapters	README
PyTorch	Llama 3.2 3B	LoRA	README
PyTorch	Llama 3.1 8B	LoRA	README
PyTorch	Llama 3.1 8B Instruct	LoRA	README
PyTorch	Llama 3.1 70B	LoRA	README
PyTorch	Llama 3.3 70B Instruct	LoRA	README
PyTorch	GPT-OSS 20B	LoRA	README
PyTorch	GPT-OSS 120B	LoRA	README
PyTorch	Qwen 2.5 0.5B	LoRA	README
PyTorch	Qwen 2.5 1.5B	LoRA	README
Pytorch	Qwen 3 4B Instruct 2507	LoRA	README
Pytorch	Qwen 3 8B-Base	LoRA	README
Pytorch	Qwen 3 8B	LoRA	README
PyTorch	Qwen 3 32B	LoRA	README
PyTorch	Gemma 3 1B	LoRA	README
PyTorch	Gemma 1.1 2B	LoRA	README
PyTorch	Gemma 1.1 2B	LoRA, DPO	README
PyTorch	Gemma 4 E2B Instruct	LoRA	README
PyTorch	ALBERT	Adapters	README
PyTorch	Phi-1	LoRA	README
PyTorch	Phi-1.5	LoRA	README
PyTorch	GATv2	Full-model	README
PyTorch	Wan 2.2 5b	LoRA	README
JAX	MLP	Full-model	README
JAX	NeRF	Full-model	README
JAX	Llama 3.2 1B	LoRA	README
JAX	Llama 3.2 1B	DoRA	README
JAX	DistilBERT	Distillation	README
EasyDel	Qwen 3 0.6B	LoRA	README
Lightning	NeRF	Full-model	README

Navigating the Experiment Structure

Within this repository, you’ll find the following structure to help you navigate the experimental setup:

• datasets/: The dataset loaders for specific model training are defined in this directory and organized by the framework they utilize. For example, the loader for the MNIST dataset in PyTorch can be found at datasets/torch/mnist/.

• models/: This directory is organized by framework. Within it, you’ll find subdirectories (e.g., jax/, torch/) containing the model implementations or loader scripts specific to that framework. For instance, the PyTorch implementation of a model for MNIST training would be located in models/torch/mnist/.

• experiments/: Experiments are organized first by the framework they utilize, and then by the specific model or task. For example, the PyTorch-based MNIST experiment can be found under experiments/torch/mnist/. Within each experiment directory, you will typically find the following subdirectories and files:

  • Subdirectories named after fine-tuning methods used in experiments (e.g. lora, dpo, adapters).

  • Subdirectories specifying the compute environment (e.g. single_chip, quietbox, loudbox, galaxy).

  • Within these subdirectories there are YAML files containing the specific configuration parameters, named after the model and dataset used (e.g. gemma11_sst2.yaml - the full path in the experiments directory for this file is torch/gemma11/lora/single_chip/gemma11_sst2.yaml).

  • A Python file defining the configuration structure for the experiment (e.g. configs.py).

  • A Python training script (train.py) responsible for running the experiment using the defined configurations.

  • A README file listing all experiments with the given model and fine-tuning method.