🤗Tokenizers
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
|
How to truncate from the head in AutoTokenizer?
|
|
2 | 16 | August 11, 2020 |
|
Add new tokens for subwords
|
|
10 | 107 | August 11, 2020 |
|
Token alignment for word-level tasks
|
|
2 | 24 | August 5, 2020 |
|
ByteLevelBPETokenizer inconsistent behavior
|
|
1 | 23 | July 23, 2020 |
|
Use a pretrained ByteLevelBPETokenizer on text
|
|
2 | 75 | July 17, 2020 |
|
Mask only specific words
|
|
3 | 104 | July 11, 2020 |
|
Continuation token in pertained tokenizer bert-base-chinese
|
|
1 | 34 | July 11, 2020 |
|
Tokenizers v0.8.0 is out!
|
|
1 | 69 | July 7, 2020 |
|
About the Tokenizers category
|
|
2 | 25 | July 7, 2020 |