How to use Intel dGPU to finetune model? e.g. LoRA or Qlora

[JamieVC]

Describe the issue

is that implemented in IPEX to finetune model with Intel hardware? dGPU, or Xeon, Gaudi..etc Thanks!

[unrahul]

Ipex supports XPUs (intel GPUs) and CPUs:

Here you go, both LoRA and QLoRA is supported: https://github.com/intel/intel-extension-for-pytorch/tree/release/xpu/2.6.10/examples/gpu/llm/fine-tuning

On Gaudi, please check out optimum-habana library

[wangkl2]

@JamieVC

• For CPU, while IPEX does support fine-tuning on it (you can refer to https://github.com/intel/intel-extension-for-pytorch/tree/main/examples/cpu/llm/fine-tuning), it is more recommended to use stock PyTorch for training on CPUs directly.
• For Lora fine-tuning on dGPU, refer to the link Rahul provided.
• For QLora fine-tuning on dGPU, refer to https://github.com/intel/intel-extension-for-pytorch/tree/release/xpu/2.6.10/examples/gpu/llm/bitsandbytes.
• For HPU, refer to https://github.com/HabanaAI/Gaudi-tutorials/blob/main/PyTorch/llama2_fine_tuning_inference/llama2_fine_tuning_inference.ipynb as an example with optimum-habana.