Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] DetectPromptInjection validator asking for pinecone_environment #760

Closed
msintaha opened this issue May 8, 2024 · 7 comments
Closed
Labels
bug Something isn't working Stale

Comments

@msintaha
Copy link

msintaha commented May 8, 2024

Describe the bug
The DetectPromptInjection library is currently unusable because the underlying library (Rebuff) has not released the change for their API initialization. It seems you have reached out to them on their github repo, but I was wondering if the dependency on Rebuff can be changed to an alternative option as they dont seem to be responsive. Perhaps a pre-trained huggingface model?

To Reproduce
Adding pinecone_api_key following the steps defined in DetectPromptInjection validator throws RebuffSdk.__init__() missing 1 required positional argument: 'pinecone_environment' error

Expected behavior
Works without defining the pinecone_environment or using a different underlying library to detect prompt injections

Library version:
Latest

@msintaha msintaha added the bug Something isn't working label May 8, 2024
@zsimjee
Copy link
Collaborator

zsimjee commented May 13, 2024

We're looking at this, and we think that we can get rebuff to work in one of the following ways

  1. add the github release to the pyproject
  2. add a pip installation from github to the post_install script

As far as the idea of using a diff prompt injection model from huggingface goes, I think this is something we should pursue regardless. If you would like to help create that validator, I can show you how and you can share it on the hub. Otherwise, we might take a bit to get to it

@msintaha
Copy link
Author

msintaha commented May 14, 2024

We're looking at this, and we think that we can get rebuff to work in one of the following ways

  1. add the github release to the pyproject
  2. add a pip installation from github to the post_install script

As far as the idea of using a diff prompt injection model from huggingface goes, I think this is something we should pursue regardless. If you would like to help create that validator, I can show you how and you can share it on the hub. Otherwise, we might take a bit to get to it

@zsimjee thanks for getting back on this. For integrating the huggingface models, if there's some guideline on getting it integrated with a validator, I can take a look when I have some time, but for the rebuff workaround, I hope that's something coming up soon! Can you share timelines for this change?

@zsimjee
Copy link
Collaborator

zsimjee commented May 14, 2024

@msintaha sweet yes the rebuff validator fix is merged, but I haven't validated it yet. Should be able to try again, otherwise I'll validate it later tonight.

For huggingface models, you can follow the pattern here -

validator code - https://github.com/guardrails-ai/sensitive_topics/blob/67adbe4f47716b61a61486ad7fc7f87766f86d89/validator/main.py#L111

post-install script - https://github.com/guardrails-ai/sensitive_topics/blob/main/validator/post-install.py

validator - https://hub.guardrailsai.com/validator/guardrails/sensitive_topics

@msintaha
Copy link
Author

@zsimjee I just tested by pulling the DetectPromptInjection validator again, it seems the error for the positional argument being required (pinecone_environment) still exists

Copy link

github-actions bot commented Aug 9, 2024

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 14 days.

@github-actions github-actions bot added the Stale label Aug 9, 2024
Copy link

This issue was closed because it has been stalled for 14 days with no activity.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Aug 23, 2024
@ChristianWeyer
Copy link

ChristianWeyer commented Nov 29, 2024

@zsimjee I just tested by pulling the DetectPromptInjection validator again, it seems the error for the positional argument being required (pinecone_environment) still exists

Did you ever get this solved @msintaha ? I actually cannot even find DetectPromptInjection

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Stale
Projects
None yet
Development

No branches or pull requests

3 participants