Skip to content

Pull requests: pytorch-labs/tokenizers

New pull request New
6 Open 78 Closed
6 Open 78 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Add python sentencepiece tokenizer CLA Signed This label is managed by the Meta Open Source bot.
#88 opened Jun 26, 2025 by jackzhxng Loading…
Fix CQS signal performance-faster-string-find in fbcode/pytorch/tokenizers/src CLA Signed This label is managed by the Meta Open Source bot. fb-exported
#86 opened Jun 25, 2025 by facebook-github-bot Loading…
2
Make pretokenizer optional in hf tokenizer CLA Signed This label is managed by the Meta Open Source bot.
#81 opened Jun 5, 2025 by jackzhxng • Draft
Fix CQS signal. Id] 10393698 -- performance-faster-string-find in fbcode/pytorch/tokenizers/src CLA Signed This label is managed by the Meta Open Source bot. fb-exported
#76 opened May 20, 2025 by facebook-github-bot Loading…
2
Fix CQS signal. Id] 10393698 -- performance-faster-string-find in fbcode/pytorch/tokenizers/src CLA Signed This label is managed by the Meta Open Source bot. fb-exported
#44 opened Apr 14, 2025 by facebook-github-bot Loading…
2
Fix CQS signal. Id] 9250181 -- readability-redundant-string-init in fbcode/pytorch/tokenizers/src CLA Signed This label is managed by the Meta Open Source bot. fb-exported
#43 opened Apr 10, 2025 by facebook-github-bot Loading…
2
ProTip! Find all pull requests that aren't related to any open issues with -linked:issue.