+ 
+
+LLM Finetuning toolkit is a config-based CLI tool for launching a series of LLM finetuning experiments on your data and gathering their results. From one single `yaml` config file, control all elements of a typical experimentation pipeline - **prompts**, **open-source LLMs**, **optimization strategy** and **LLM testing**.
-LLM Finetuning Hub contains code and insights to finetune various large language models for your use-case.
+
+
+
Performance
-- 
Time to Train
-- 
Cost to Train
-- 
Inferencing
+### [Option 1] Docker (recommended)
+```shell
+ docker build -t llm-toolkit
+```
-For each of the above four pillars, we are sharing our codebase and insights to:
-- Assist you to leverage LLMs for your business needs and challenges
-- Decide which LLM suits your needs from a performance and cost perspective
-- Boost reproducibility efforts which are becoming increasingly difficult with LLMs
+#### CPU Only
-We are providing scripts that are ready-to-use for:
-- Finetuning LLMs on your proprietary dataset via PeFT methodologies such as LoRA and Prefix Tuning
-- Performing hyperparameter optimization to get the maximum performance out of these models
+```shell
+ docker run -it llm-toolkit
+```
-## Getting Started
+#### With GPU
-You can start fine-tuning your choice of LLM in 4 easy steps:
+```shell
+ docker run -it --gpus all llm-toolkit
+```
-1. **Setup conda environment**
+### [Option 2] Poetry (recommended)
- ```shell
- wget https://repo.anaconda.com/miniconda/Miniconda3-py38_4.11.0-Linux-x86_64.sh
- bash Miniconda3-py38_4.11.0-Linux-x86_64.sh
- source ~/.bashrc
- conda create --name llm_finetuning python=3.9
- conda activate llm_finetuning
- ```
+See poetry documentation page for poetry [installation instructions](https://python-poetry.org/docs/#installation)
-2. **Install relevant packages**
+```shell
+ poetry install
+```
- ```shell
- git clone https://github.com/georgian-io/LLM-Finetuning-Hub.git
- cd LLM-Finetuning-Hub/
- pip install -r requirements.txt
- ```
+### [Option 3] pip
-3. **Finetune your LLM of choice**
+```shell
+ pip install -r requirements.txt
+```
- For instance, to finetune Llama2-7B or Llama2-13B, do the following:
+### [Option 4] Conda
- ```shell
- cd llama2/ # navigate to Llama2 folder
- python llama2_classification.py --lora_r 8 --epochs 5 --dropout 0.1 --pretrained_ckpt NousResearch/Llama-2-7b-hf # finetune Llama2-7B on newsgroup classification dataset
- python llama2_classification_inference.py --experiment