Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Groq AI support. #381

Merged
merged 1 commit into from
Sep 7, 2024
Merged

Add Groq AI support. #381

merged 1 commit into from
Sep 7, 2024

Conversation

ozeliurs
Copy link
Contributor

Following comments in #346, I set up groq support with the OCO_GROQ_API_KEY variable by setting the OCO_AI_PROVIDER to groq.

I am not very familiar with this project, so please feel free to point out mistakes.

@ozeliurs ozeliurs changed the base branch from master to dev August 17, 2024 09:13
@ozeliurs ozeliurs changed the base branch from dev to master August 17, 2024 09:14
@di-sukharev
Copy link
Owner

how do you pay for the groq api key?

@ozeliurs
Copy link
Contributor Author

how do you pay for the groq api key?

it is free brother, no need to pay.

Go to https://console.groq.com/keys, create an account and generate an api key.

@di-sukharev
Copy link
Owner

hmm.. do they then somehow log your requests or they dont explicitly tell how requests privacy is managed?

@ozeliurs
Copy link
Contributor Author

don't know, all I found is this: https://wow.groq.com/privacy-policy/

They say they don't retain our data but they have to keep count on how much request we make so they for sure log something.

@di-sukharev
Copy link
Owner

thank you! @ozeliurs please fix the conflicts, sorry for that, just released 3.0.18 🥵

@ozeliurs
Copy link
Contributor Author

thank you! @ozeliurs please fix the conflicts, sorry for that, just released 3.0.18 🥵

Should I switch to dev before ?

@di-sukharev
Copy link
Owner

dev is best, yes

@ozeliurs ozeliurs changed the base branch from master to dev August 19, 2024 01:41
@ozeliurs ozeliurs force-pushed the master branch 2 times, most recently from 890e1c0 to 7fe7ae2 Compare August 22, 2024 07:32
@ozeliurs
Copy link
Contributor Author

Updated to be able to be merged to dev :)

@di-sukharev
Copy link
Owner

there was a refactoring #391, please fix conflicts

@di-sukharev
Copy link
Owner

you will also need to adjust your code a bit, it will be easier to support

@di-sukharev
Copy link
Owner

@ozeliurs i have resolved the conflicts, but you still need to change the GroqAiEngine a bit, it's simple, please take a look at any other Engine implementation, e.g. OpenAiEngine.

Every Engine is now taking parameters in the constructor, i passed them to your new GroqAi(here) call, but you need to add the constructor anyway

@di-sukharev
Copy link
Owner

@ozeliurs

@di-sukharev
Copy link
Owner

please follow any aiEngine (e.g. OpenAiEngine) implementation in this file, there are few more lines to add to the PR so we can merge it @ozeliurs

@ozeliurs ozeliurs marked this pull request as draft September 7, 2024 09:35
@ozeliurs ozeliurs marked this pull request as ready for review September 7, 2024 10:09
@ozeliurs
Copy link
Contributor Author

ozeliurs commented Sep 7, 2024

Hi,

I made the modifications according to your refactoring. The code works now by setting:

  • OCO_AI_PROVIDER='groq'
  • OCO_API_KEY=<groq_api_key>
  • OCO_MODEL=llama3-70b-8192 (or any available model)

@di-sukharev
Copy link
Owner

awesome @ozeliurs, thanks for the contribution, tests are passing, merging it, gonna be available in the next release

@di-sukharev di-sukharev merged commit 2d7e384 into di-sukharev:dev Sep 7, 2024
3 checks passed
di-sukharev added a commit that referenced this pull request Sep 7, 2024
@di-sukharev @matscube @ozeliurs
3.2.2 (#413)
2769121 
* feat(config): add support for groq AI provider, including config validation and engine implementation (#381)

* fix migrations (#414)

---------

Co-authored-by: Takanori Matsumoto <[email protected]>
Co-authored-by: BILLY Maxime <[email protected]>
@ozeliurs
Copy link
Contributor Author

ozeliurs commented Sep 7, 2024

Thanks a lot, pleasure contributing. :)

@erickedji
Copy link

This will still fail in src/migrations/00_use_single_api_key_and_url.ts#L35 because the switch doesn't handle groq.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@di-sukharev di-sukharev di-sukharev approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ozeliurs @di-sukharev @erickedji