E5 embedding models

koleckar · March 17, 2025, 12:50pm

Hello everyone,
we are unable to find any existing pre-training/fine-tuning scripts for e5-models family.
Can someone help us?

Thank you very much for any answers.
W/regards
David Koleckar

John6666 · March 17, 2025, 4:54pm

I wonder if the model is for SentenceTransformer rather than Transformers, so it’s different in terms of usability.

github.com/microsoft/unilm

The project of E5: How to fine-tune on the E5 model?

opened 07:26AM - 14 Apr 23 UTC

closed 09:51AM - 08 May 23 UTC

EasyLuck

Dear author: The following informations is about the paper and project of E5: […Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://arxiv.org/pdf/2212.03533.pdf). Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei, arXiv 2022 I want to fine-tune on my dataset. However, I do not know how to do it, there is no more information on the main site of GitHub: https://github.com/microsoft/unilm/tree/master/e5. 1. I don't know how to prepare my data. Same as what is the data structure should I design. 2. What the tools should I use? Can you provide more information or train_script / finetune_script on the GitHub . Thank you very much.

Topic		Replies	Views
How is T5 pretrained? 🤗Transformers	3	495	July 12, 2021
mT5/T5v1.1 Fine-Tuning Results Models	16	7423	March 8, 2022
Example of how to pretrain T5? 🤗Transformers	15	15755	March 16, 2023
Fine tune T5 for text generation in specific domain Beginners	3	1988	August 28, 2023
Continue pretraining on a released model Beginners	0	761	January 1, 2024

E5 embedding models

Related topics