0.4.8 - User feedback on sessions

What's Changed

• User provided feedback by @lukemarsden in #151

Full Changelog: 0.4.7...0.4.8

0.4.7 - Remove OpenAI dependency

What's Changed

• Remove the dependency on OpenAI. We now have a dependency on Together.ai instead where we use Mixtral (Mistral 8x7B MoE) - see .env.sample-prod for how to configure this. The next step will be to remove that as well and offer fully self hosted Mixtral for qapair generation

Also, some bugfixes and WIP feature work:

• Fix/pubsub by @rusenask in #147
• only notify the user that the fine tune was completed if there was an… by @binocarlos in #148
• Feature/tools+apis by @rusenask in #149

Full Changelog: 0.4.6...0.4.7

0.4.6 - Even better summarization

Further improvements to summarization prompt

Full Changelog: 0.4.5...0.4.6

0.4.5 - Better summarization

Better prompt for summarization qapairs

Full Changelog: 0.4.4...0.4.5

0.4.4 - More reliable qapairs

What's Changed

• disable one qapair prompt that was unreliable
• fix race in subscriber creation by @rusenask in #141

Full Changelog: 0.4.3...0.4.4

0.4.3 - Disable some of the harder qapair prompts

These were causing timeouts

0.4.2 - Give the finetune bot a name

Tell Helix that its name is Helix.

Full Changelog: 0.4.1...0.4.2

0.4.1 - System prompt, retries & bugfixes

What's Changed

• Fix "ignore errors" button by @binocarlos in #132
• Automatic progression to finetuning after qapairs step by @binocarlos in #133
• Fix cloning fetaure: bug where the top level session lora_dir was not being copied by @binocarlos in #134
• The unstructured downloader will now report back with any upstream errors downloading files with better error messages for the user by @binocarlos in #136
• If there's an error in dataprep, don't automatically progress to finetuning by @binocarlos in #138
• Add system prompt for text finetuning improving fine tuning performance and self-awareness, and add retries by @lukemarsden in #139

Full Changelog: 0.4.0...0.4.1

0.4.0 - Mixtral for qapairs, qapair prompt suite

What's Changed

• Fix finetuning uploads by @rusenask in #125
• Finetuning improvements by @lukemarsden in #129

Finetuning improvements

Helix now uses Mixtral on Together.ai instead of GPT-4 on OpenAI to generate the question-answer pairs in the dataprep stage of text finetuning. It also now uses a suite of qapair generation prompts, defined in qapairs_config.yaml. This is both a step towards fully on-prem qapair generation (we can run Mixtral locally) and results in better quality fine text tuning.

Upgrade notes

Be sure to add TOGETHER_API_KEY to an API key from together.ai instead of the OpenAI key in .env.

Full Changelog: 0.3.8...0.4.0

0.3.8 - GPU selection support on multi-GPU systems

GPU selection

You can now specify which GPU you want the runner to target by specifying CUDA_VISIBLE_DEVICES in the environment for your runner.

Example, to use the second GPU in the system, set CUDA_VISIBLE_DEVICES=1 :

sudo docker run --privileged --gpus all --shm-size=10g \
    --restart=always -d \
    --name helix-runner --ipc=host --ulimit memlock=-1 \
    --ulimit stack=67108864 \
    -e CUDA_VISIBLE_DEVICES=1 \
    -v ${HOME}/.cache/huggingface:/root/.cache/huggingface \
    europe-docker.pkg.dev/helixml/helix/runner:0.3.8 \
    --api-host <http(s)://YOUR_CONTROLPLANE_HOSTNAME> --api-token <RUNNER_TOKEN_FROM_ENV> \
    --runner-id $(hostname) \
    --memory 40GB \
    --allow-multiple-copies --timeout-seconds 120

You can still only run the runner against a single GPU at a time but now you can select which one it is using. If you want to use multiple GPUs in a system, you can run multiple runner containers (just give them different names).

Automatic exit on coredump

We've observed GPU runners occasionally segfault with an "Illegal instruction" message. When this happens, it seems to be sufficient simply to restart the container. We've made the runner automatically exit when it detects this condition. This should make the system more reliable. Make sure to run your container with --restart always or equivalent, depending on your container runtime.