Hi everyone.
Is it possible to use a customized algorithm for tokenizing and building the vocab during Language Modeling?
Thanks!
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Using customized algorithm | 0 | 321 | August 17, 2022 | |
| Customizing T5 tokenizer for finetuning | 1 | 541 | May 2, 2024 | |
| Trianing a model using predefined vocab | 0 | 224 | August 16, 2022 | |
| Tokenizer from own vocab | 0 | 439 | July 11, 2022 | |
| Tokenizer effect on the fine-tuning | 0 | 358 | October 6, 2023 |