Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Extend CustomLogitProcessor to Support input_ids in call Method #3524

Open
1 of 2 tasks
Aykhan-sh opened this issue Feb 12, 2025 · 2 comments
Open
1 of 2 tasks
Assignees
Labels

Comments

@Aykhan-sh
Copy link

Aykhan-sh commented Feb 12, 2025

Checklist

Motivation

Thanks @hongpeng-guo for PR #2396. After reviewing your work, I'd like to propose an enhancement to the CustomLogitProcessor. Specifically, I suggest modifying its __call__ method to accept input_ids as an additional parameter—similar to the implementation in Huggingface (see this doc). This change would allow constraints to be applied conditionally based on the entire history of input tokens, enabling more flexible and context-aware processing.

Thank you for considering this feature request!

Related resources

Huggingface LogitsProcessor.

@Aykhan-sh Aykhan-sh changed the title [Feature] Extend CustomLogitProcessor to Support nput_ids in call Method [Feature] Extend CustomLogitProcessor to Support input_ids in call Method Feb 12, 2025
@jhinpan jhinpan self-assigned this Feb 12, 2025
@jhinpan
Copy link
Collaborator

jhinpan commented Feb 12, 2025

Thank you for your suggestion! You are welcome to raise a new PR to realize the enhancement as you mentioned.

@hongpeng-guo
Copy link
Contributor

@Aykhan-sh Thanks for your suggestions. It makes a lot of sense to include input_ids as an extra args. I am thinking about if it's necessary to also include args and kwargs to make this function more flexible. But for now, let's just align with the HuggingFace API.

Feel free to start a PR and assign me as a reviewer if you have time. Otherwise, I can take this issue later this week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants