Skip to content

Commit d261f58

Browse files
authored
Add is_loaded() API
Differential Revision: D73165546 Pull Request resolved: #53
1 parent def3a96 commit d261f58

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

include/pytorch/tokenizers/tokenizer.h

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,10 @@ class Tokenizer {
5656
return eos_tok_;
5757
}
5858

59+
virtual bool is_loaded() const {
60+
return initialized_;
61+
}
62+
5963
protected:
6064
bool initialized_ = false;
6165
int32_t vocab_size_ = 0;

0 commit comments

Comments
 (0)