File tree 2 files changed +9
-2
lines changed
2 files changed +9
-2
lines changed Original file line number Diff line number Diff line change @@ -21,8 +21,15 @@ The project follows [semantic versioning 2.0.0](https://semver.org/). The API co
21
21
22
22
### New features
23
23
24
- * Introduce the "Token API": a set of methods that manipulate ` Token ` objects instead of serialized strings
24
+ ### Fixes and improvements
25
+
26
+ ## [ v1.19.0] ( https://github.com/OpenNMT/Tokenizer/releases/tag/v1.19.0 ) (2020-09-02)
27
+
28
+ ### New features
29
+
25
30
* Add BPE dropout ([ Provilkov et al. 2019] ( https://www.aclweb.org/anthology/2020.acl-main.170/ ) )
31
+ * [ Python] Introduce the "Token API": a set of methods that manipulate ` Token ` objects instead of serialized strings
32
+ * [ Python] Add ` unicode_ranges ` argument to the ` detokenize_with_ranges ` method to return ranges over Unicode characters instead of bytes
26
33
27
34
### Fixes and improvements
28
35
Original file line number Diff line number Diff line change @@ -40,7 +40,7 @@ def _maybe_add_library_root(lib_name, header_only=False):
40
40
41
41
setup (
42
42
name = "pyonmttok" ,
43
- version = "1.18.5 " ,
43
+ version = "1.19.0 " ,
44
44
license = "MIT" ,
45
45
description = "OpenNMT tokenization library" ,
46
46
long_description = _get_long_description (),
You can’t perform that action at this time.
0 commit comments