Hello everyone,
I hope this message finds you well. I am currently working on a project that involves generating technical documents, specifically CCTP (Cahier des Clauses Techniques Particulières), from DQE ( a document which contains the summary for the drafting of the cctp) using large language models (LLMs). I have access to several examples of both CCTP and DQE, as well as a powerful GPU setup with an A100 40GB.
I am looking for advice on the following aspects:
- Model Selection: Which open-source LLMs would be best suited for fine-tuning to generate detailed technical documents? I am considering models like BLOOM, T5, BART, Llama, and Mistral.
- Data Preparation: How should I structure and prepare my training data to effectively fine-tune these models? I have extracted text from PDFs and need guidance on annotation and creating input-output pairs.
- Fine-Tuning Process: Any tips or best practices for fine-tuning these models on my specific task? I am particularly interested in ensuring the generated documents are accurate and coherent.
I would greatly appreciate any insights, resources, or experiences shared by the community. Thank you in advance for your help!
Best regards,