How to Configure LLaMA-3:8B on HuggingFace to Generate Responses Similar to Ollama?

[John6666]

The issue of different output when using pipelines and when using them locally or in Spaces is sometimes talked about.
While pipelines are easy to deal with, they seem to be working by supplementing a lot of information on their own, which means it is difficult to determine what is wrong with them.

Completely different results for model in pipeline and by itself

Ah, actually, I think I see the issue, right after posting it. I think it’s about setting up the tokenizer? Could that be it? In preprocess I make sure to have overlap, pad and max length = model.max_length I’m guessing by default when I just pass tokenizer to the pipeline, none of these get set. Can I pass a pre setup tokenizer? Figured it out, there are two parameters, max_seq_len and max_answer_len that can set during the call but not initialization. thanks a lot!

Different outputs when using pipeline

Hi folks! I’m currently working on my master thesis and need to implement a text-classification model. Therefore I’m fine-tuning a base BERT model on my data. So far so good. To evaluate my fine-tuned model, I’m passing my test data to it for predictions. For this purpose I’m using the “pipeline” API provided by huggingface with a sigmoid function (since I’m facing a multi-label classification problem). It does work and I get as an output the probs for each class. Now if instead of using the pi…

I’m not sure about Ollama, but I think it would be better to use something other than pipelines to set chatbot-like parameters.
chat_completion, for example, is often used.
Also, I heard that Llama3 has some kind of bug, though I’m not sure what it has to do with this issue.

Chatting with Transformers

Chat Completion