Unable to Train Lora with Oobabooga

I am a beginner with LLMs but I have been able to install Ollama, Oobabooga, sillytavern, anything llm, and convert between GGUF to GPTQ. I use windows 10 and Ubuntu 24.04 and also have some training experience with Flux on my home computer and Massed Compute.

I have been trying to train my own Lora using Oogbooga. I have tried on linux and windows. I have tried GGUF models and GPTQ models. I have tried .txt files and Json files generated from past chats. Nothing seems to work. I have also installed the Training Pro extension.

Every time I try a GGUF model I receive the errpr:

Attribute Error: ‘LlamaServer’ object has no attribute ‘bos_token_id’

I was hoping that Training Pro would fix this error as it has a box to add a bos token to each data set item.

I get even more errors when trying to train a GPTQ model.

I have searched for alternate training.py files if that is the problem and have not found any that work.

I have not found much help on the internet or github.

Any suggestion?

The whole console output for the Lora is:

16:24:07-798561 INFO Loaded “nvidia_Llama-3.1-Nemotron-Nano-4B-v1.1-Q6_K.gguf” in 2.51 seconds.
16:24:07-800568 INFO LOADER: “llama.cpp”
16:24:07-801571 INFO TRUNCATION LENGTH: 8192
16:24:07-802575 INFO INSTRUCTION TEMPLATE: “Custom (obtained from model metadata)”
16:24:23-882099 INFO Loading Text file…
Precise raw text slicer: ON
Sentences: 2967
Text Blocks: 230

  • Overlapping blocks: 228
    16:24:28-939665 WARNING LoRA training has only currently been validated for LLaMA, OPT, GPT-J, and GPT-NeoX models.
    (Found model type: LlamaServer)
    *** LoRA: 1 ***
    16:24:33-942140 INFO Loading text file…
    Precise raw text slicer: ON
    Sentences: 2967
    Text Blocks: 230
  • Overlapping blocks: 228
    Traceback (most recent call last):
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\queueing.py”, line 580, in process_events
    response = await route_utils.call_process_api(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\route_utils.py”, line 276, in call_process_api
    output = await app.get_blocks().process_api(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\blocks.py”, line 1928, in process_api
    result = await self.call_function(
    ^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\blocks.py”, line 1526, in call_function
    prediction = await utils.async_iteration(iterator)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py”, line 657, in async_iteration
    return await iterator.anext()
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py”, line 650, in anext
    return await anyio.to_thread.run_sync(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\anyio\to_thread.py”, line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\anyio_backends_asyncio.py”, line 2470, in run_sync_in_worker_thread
    return await future
    ^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\anyio_backends_asyncio.py”, line 967, in run
    result = context.run(func, *args)
    ^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py”, line 633, in run_sync_iterator_async
    return next(iterator)
    ^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py”, line 816, in gen_wrapper
    response = next(iterator)
    ^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\extensions\Training_PRO\script.py”, line 704, in do_train
    train_data = Dataset.from_list([tokenize(x, add_EOS_to_all, add_bos_token) for x in text_chunks])
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\extensions\Training_PRO\script.py”, line 704, in
    train_data = Dataset.from_list([tokenize(x, add_EOS_to_all, add_bos_token) for x in text_chunks])
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\extensions\Training_PRO\script.py”, line 623, in tokenize
    input_ids = encode(prompt, prepend_bos_token)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\extensions\Training_PRO\script.py”, line 613, in encode
    if len(result) >= 2 and result[:2] == [shared.tokenizer.bos_token_id, shared.tokenizer.bos_token_id]:
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    AttributeError: ‘LlamaServer’ object has no attribute ‘bos_token_id’
1 Like

From a quick read of the code, I don’t think training a GGUF-quantized model is intended. How about trying it with the Transoformers-format model before GGUF quantization?

Thank you for the reply. I also tried training using a transformers based GPTQ model. I received several errors attempting to train this format as well. I will try and get them posted. At least I know where not to waste my time now.

1 Like

I found the solution. I selected transformers but received errors. I was told to use pip-install XYZ (I can’t remember the exact command).

For Ubuntu, run the cmd_linux.sh in konsole by right clicking and selecting this option. Make sure to select the “run in terminal” option vs “open terminal here” option. The cmd_linux.sh file is located in the same folder as the start.sh and update programs.

Copy the pip install command from oobabooga and paste it into the terminal you just opened. This command should be located in the bottom right portion of the page after all the previous errors listed in the training tab of the gradio.

You have to do this a second time for a new package that also needs to be installed. This time oobabooga gives you an option of two different pip installs. Select the second option as the first does not work.

Copy and paste this new pip-install command that oobabooga gives you into the terminal. (you may have to close and restart the run in cmd_linux.sh terminal for the new pip install.)

If you can load a GPTQ file using transformers, you should be able to train a LORA using either the normal or training pro extension.

1 Like

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.