Skip to content

Commit

Permalink
New sample notebook: model alignment (#75)
Browse files Browse the repository at this point in the history 
Co-authored-by: Julien Simon <[email protected]>
  • Loading branch information
@juliensimon
juliensimon and Julien Simon authored Jul 24, 2024
1 parent 1cb8369 commit 083ed0f
Showing 1 changed file with 370 additions and 0 deletions.
370 changes: 370 additions & 0 deletions notebooks/model_alignment.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,370 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Aligning a model on Arcee Cloud\n",
"\n",
"In this notebook, you will learn how to align a model with Supervised Fine Tuning on Arcee Cloud.\n",
"\n",
"In order to run this demo, you need a Starter account on Arcee Cloud. Please see our [pricing](https://www.arcee.ai/pricing) page for details.\n",
"\n",
"The Arcee documentation is available at [docs.arcee.ai](https://docs.arcee.ai/deployment/start-deployment)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Prerequisites\n",
"\n",
"Please [sign up](https://app.arcee.ai/account/signup) to Arcee Cloud and create an [API key](https://docs.arcee.ai/getting-arcee-api-key/getting-arcee-api-key).\n",
"\n",
"Then, please update the cell below with your API key. Remember to keep this key safe, and **DON'T COMMIT IT to one of your repositories**."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%env ARCEE_API_KEY=YOUR_API_KEY"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a new Python environment (optional but recommended) and install [arcee-python](https://github.com/arcee-ai/arcee-python)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Uncomment the next three lines to create a virtual environment\n",
"#!pip install -q virtualenv\n",
"#!virtualenv -q arcee-cloud\n",
"#!source arcee-cloud/bin/activate\n",
"\n",
"%pip install -qU arcee-py"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We will also need the Hugging Face [`datasets`](https://huggingface.co/datasets) library to download an alignment model from the Hugging Face hub."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%pip install -q datasets"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import arcee\n",
"import pprint\n",
"import pandas as pd\n",
"from datasets import load_dataset"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Preparing our dataset\n",
"\n",
"Model alignment requires a dataset with questions and answers. At the moment, Arcee Cloud requires this dataset to be in the CSV format, with one column for questions and one for answers.\n",
"\n",
"Let's download the [`reasoning-sharegpt`](https://huggingface.co/datasets/arcee-ai/reasoning-sharegpt) dataset from the Hugging Face hub. \n",
"\n",
"This dataset contains close to 30,000 single-turn question-answers pairs focused on step-by-step reasoning."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"dataset_name = \"arcee-ai/reasoning-sharegpt\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"dataset = load_dataset(dataset_name)\n",
"print(dataset)\n",
"pprint.pprint(dataset['train'][0])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, let's format it in two columns with just the questions and the answers, then save it to the CSV format."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"data = [{\n",
" \"questions\": item['conversations'][0]['value'],\n",
" \"answers\": item['conversations'][1]['value']\n",
"} for item in dataset['train']]\n",
"\n",
"df = pd.DataFrame(data)\n",
"df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The last preparation step is to save the dataset in CSV format."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"csv_name = \"reasoning-sharegpt.csv\"\n",
"\n",
"df.to_csv(csv_name, index=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Uploading our dataset\n",
"\n",
"Now that we've prepared the dataset, we can upload it to Arcee Cloud with the `upload_qa_pairs_from_csv()` API."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"help(arcee.upload_qa_pairs_from_csv)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"qa_set=\"reasoning-share-gpt\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"arcee.upload_qa_pairs_from_csv(\n",
" qa_set=qa_set,\n",
" csv_path=f\"./{csv_name}\",\n",
" question_column=\"questions\",\n",
" answer_column=\"answers\"\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Aligning the model\n",
"\n",
"We can now use the dataset for model alignment. \n",
"\n",
"Here, we will align the [Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) model. We could pick any model available on the Hugging Face hub, or a model we've already worked with on Arcee Cloud.\n",
"\n",
"Let's launch the alignment job with the `start_alignment()` API."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"help(arcee.start_alignment)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"alignment_name = \"llama-3-8B-reasoning-share-gpt\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"response=arcee.start_alignment(alignment_name=alignment_name,\n",
" qa_set=qa_set,\n",
" hf_model=\"meta-llama/Meta-Llama-3-8B\"\n",
")\n",
"print(response)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from time import sleep\n",
"\n",
"while True:\n",
" response = arcee.alignment_status(alignment_name)\n",
" if response[\"processing_state\"] == \"processing\":\n",
" print(\"Alignment is in progress. Waiting 5 minutes before checking again.\")\n",
" sleep(300)\n",
" else:\n",
" print(response)\n",
" break"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deploying our aligned model\n",
"\n",
"Once alignment is complete, we can deploy and test the aligned model. As part of the Arcee Cloud free tier, this is free of charge and the endpoint will be automatically shut down after 2 hours.\n",
"\n",
"Deployment should take 5-7 minutes. Please see the model deployment sample notebook for details."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"deployment_name = \"llama-3-8B-reasoning-share-gpt\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"response = arcee.start_deployment(deployment_name=deployment_name, alignment=alignment_name)\n",
"\n",
"while True:\n",
" response = arcee.deployment_status(deployment_name)\n",
" if response[\"deployment_processing_state\"] == \"pending\":\n",
" print(\"Deployment is in progress. Waiting 30 seconds before checking again.\")\n",
" sleep(30)\n",
" else:\n",
" print(response)\n",
" break"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Once the model endpoint is up and running, we can prompt the model with a domain-specific question."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"query = \"Give me instructions to change the oil on my Harley-Davidson.\"\n",
"\n",
"response = arcee.generate(deployment_name=deployment_name, query=query)\n",
"print(response[\"text\"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Stopping our deployment\n",
"\n",
"When we're done working with our model, we should stop the deployment to save resources and avoid unwanted charges.\n",
"\n",
"The `stop_deployment()` API only requires the deployment name."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"arcee.stop_deployment(deployment_name=deployment_name)\n",
"arcee.deployment_status(deployment_name)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This concludes the model alignment demonstration. Thank you for your time!\n",
"\n",
"If you'd like to know more about using Arcee Cloud in your organization, please visit the [Arcee website](https://www.arcee.ai), or contact [[email protected]](mailto:[email protected]).\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.4"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

0 comments on commit 083ed0f

Please sign in to comment.