Unable to Train Lora with Oobabooga

363ls2gto · June 5, 2025, 9:39pm

I am a beginner with LLMs but I have been able to install Ollama, Oobabooga, sillytavern, anything llm, and convert between GGUF to GPTQ. I use windows 10 and Ubuntu 24.04 and also have some training experience with Flux on my home computer and Massed Compute.

I have been trying to train my own Lora using Oogbooga. I have tried on linux and windows. I have tried GGUF models and GPTQ models. I have tried .txt files and Json files generated from past chats. Nothing seems to work. I have also installed the Training Pro extension.

Every time I try a GGUF model I receive the errpr:

Attribute Error: ‘LlamaServer’ object has no attribute ‘bos_token_id’

I was hoping that Training Pro would fix this error as it has a box to add a bos token to each data set item.

I get even more errors when trying to train a GPTQ model.

I have searched for alternate training.py files if that is the problem and have not found any that work.

I have not found much help on the internet or github.

Any suggestion?

The whole console output for the Lora is:

16:24:07-798561 INFO Loaded “nvidia_Llama-3.1-Nemotron-Nano-4B-v1.1-Q6_K.gguf” in 2.51 seconds.
16:24:07-800568 INFO LOADER: “llama.cpp”
16:24:07-801571 INFO TRUNCATION LENGTH: 8192
16:24:07-802575 INFO INSTRUCTION TEMPLATE: “Custom (obtained from model metadata)”
16:24:23-882099 INFO Loading Text file…
Precise raw text slicer: ON
Sentences: 2967
Text Blocks: 230

Overlapping blocks: 228
16:24:28-939665 WARNING LoRA training has only currently been validated for LLaMA, OPT, GPT-J, and GPT-NeoX models.
(Found model type: LlamaServer)
*** LoRA: 1 ***
16:24:33-942140 INFO Loading text file…
Precise raw text slicer: ON
Sentences: 2967
Text Blocks: 230
Overlapping blocks: 228
Traceback (most recent call last):
File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\queueing.py”, line 580, in process_events
response = await route_utils.call_process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\route_utils.py”, line 276, in call_process_api
output = await app.get_blocks().process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\blocks.py”, line 1928, in process_api
result = await self.call_function(
^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\blocks.py”, line 1526, in call_function
prediction = await utils.async_iteration(iterator)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py”, line 657, in async_iteration
return await iterator.anext()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py”, line 650, in anext
return await anyio.to_thread.run_sync(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\anyio\to_thread.py”, line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\anyio_backends_asyncio.py”, line 2470, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\anyio_backends_asyncio.py”, line 967, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py”, line 633, in run_sync_iterator_async
return next(iterator)
^^^^^^^^^^^^^^
File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py”, line 816, in gen_wrapper
response = next(iterator)
^^^^^^^^^^^^^^
File “C:\Oobabooga\text-generation-webui-main\extensions\Training_PRO\script.py”, line 704, in do_train
train_data = Dataset.from_list([tokenize(x, add_EOS_to_all, add_bos_token) for x in text_chunks])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Oobabooga\text-generation-webui-main\extensions\Training_PRO\script.py”, line 704, in
train_data = Dataset.from_list([tokenize(x, add_EOS_to_all, add_bos_token) for x in text_chunks])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Oobabooga\text-generation-webui-main\extensions\Training_PRO\script.py”, line 623, in tokenize
input_ids = encode(prompt, prepend_bos_token)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Oobabooga\text-generation-webui-main\extensions\Training_PRO\script.py”, line 613, in encode
if len(result) >= 2 and result[:2] == [shared.tokenizer.bos_token_id, shared.tokenizer.bos_token_id]:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: ‘LlamaServer’ object has no attribute ‘bos_token_id’

John6666 · June 6, 2025, 11:24am

From a quick read of the code, I don’t think training a GGUF-quantized model is intended. How about trying it with the Transoformers-format model before GGUF quantization?

github.com/oobabooga/text-generation-webui

extensions/Training_PRO/script.py

main

import os

os.environ["WANDB_MODE"] = "offline"
# os.environ["WANDB_DISABLED"] = "true"

import json
import math
import random
import shutil
import sys
import threading
import time
import traceback
from datetime import datetime
from pathlib import Path

import gradio as gr
import pandas as pd
import torch
import transformers

This file has been truncated. show original

363ls2gto · June 7, 2025, 3:24am

Thank you for the reply. I also tried training using a transformers based GPTQ model. I received several errors attempting to train this format as well. I will try and get them posted. At least I know where not to waste my time now.

363ls2gto · June 7, 2025, 9:49pm

I found the solution. I selected transformers but received errors. I was told to use pip-install XYZ (I can’t remember the exact command).

For Ubuntu, run the cmd_linux.sh in konsole by right clicking and selecting this option. Make sure to select the “run in terminal” option vs “open terminal here” option. The cmd_linux.sh file is located in the same folder as the start.sh and update programs.

Copy the pip install command from oobabooga and paste it into the terminal you just opened. This command should be located in the bottom right portion of the page after all the previous errors listed in the training tab of the gradio.

You have to do this a second time for a new package that also needs to be installed. This time oobabooga gives you an option of two different pip installs. Select the second option as the first does not work.

Copy and paste this new pip-install command that oobabooga gives you into the terminal. (you may have to close and restart the run in cmd_linux.sh terminal for the new pip install.)

If you can load a GPTQ file using transformers, you should be able to train a LORA using either the normal or training pro extension.

system · June 8, 2025, 9:50am

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Trouble training LoRA in Oobabooga Beginners	2	1155	February 27, 2024
Lama 3.23b performs great when I download and use using ollama but when I manually download the model or if I use the gguf model by unsloth, it gives me irrelevant response. Please help me out Beginners	9	1324	October 31, 2024
Cannot download llama 2 models Beginners	1	2860	August 24, 2023
Errors when trying to fine-tune OpenLLaMA using Trainer API 🤗Transformers	1	373	December 26, 2024
Loading Lora models after trainning Beginners	1	3206	June 24, 2024

Unable to Train Lora with Oobabooga

Related topics