Ubuntu 24.04, RTX 4060, i5-13400F, 16 GB DDR5 RAM I’m new to WASM and Llama but I’m trying to set up a local RAG API server. What I’d like to eventually do is repeatedly check a text file (that is growing in size) and let me ask it questions about it. My issue right now is that the Instruct model …

Unable to push embeddings onto GPU

John6666 June 5, 2025, 5:37am 13

Probably I found similar unresolved issue.

Topic		Replies	Views
Make Text Embedding Server compatible 🤗Optimum	2	280	August 8, 2024
Host a Model with vllm for RAG Models	6	3701	September 12, 2024
Cannot create new endpoints: WebserverFailed Inference Endpoints on the Hub	1	770	November 30, 2023
Processong speed for text embedding models Models	0	174	April 5, 2024
Text-generation-inference: "You are using a model of type llama to instantiate a model of type ." Models	5	7573	November 3, 2023