We would like to update the existing vocabulary to support custom synonyms with sentence transformers. As per our use case we would like to update existing token similarities or add new ones so as to fetch results for semantic search on the basis of new synonyms configured.
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Can we fine-tune right away after add_tokens? | 0 | 829 | January 31, 2023 | |
| Discussing the Pros and Cons of Using add_tokens vs. Byte Pair Encoding (BPE) for Adding New Tokens to an Existing RoBERTa Model | 0 | 761 | July 14, 2023 | |
| How to use a trained tokenizer for semantic search? | 0 | 355 | November 5, 2023 | |
| Identifying most useful domain-specific tokens for adding to the existing tokenizer | 1 | 475 | February 2, 2024 | |
| How give weight to some specific tokens in BERT? | 2 | 1001 | October 29, 2021 |