Unable to Train Lora with Oobabooga

Beginners
1

I am a beginner with LLMs but I have been able to install Ollama, Oobabooga, sillytavern, anything llm, and convert between GGUF to GPTQ. I use windows 10 and Ubuntu 24.04 and also have some training experience with Flux on my home computer and Massed Compute.

I have been trying to train my own Lora using Oogbooga. I have tried on linux and windows. I have tried GGUF models and GPTQ models. I have tried .txt files and Json files generated from past chats. Nothing seems to work. I have also installed the Training Pro extension.

Every time I try a GGUF model I receive the errpr:

Attribute Error: ‘LlamaServer’ object has no attribute ‘bos_token_id’

I was hoping that Training Pro would fix this error as it has a box to add a bos token to each data set item.

I get even more errors when trying to train a GPTQ model.

I have searched for alternate training.py files if that is the problem and have not found any that work.

I have not found much help on the internet or github.

Any suggestion?

The whole console output for the Lora is:

16:24:07-798561 INFO Loaded “nvidia_Llama-3.1-Nemotron-Nano-4B-v1.1-Q6_K.gguf” in 2.51 seconds.
16:24:07-800568 INFO LOADER: “llama.cpp”
16:24:07-801571 INFO TRUNCATION LENGTH: 8192
16:24:07-802575 INFO INSTRUCTION TEMPLATE: “Custom (obtained from model metadata)”
16:24:23-882099 INFO Loading Text file…
Precise raw text slicer: ON
Sentences: 2967
Text Blocks: 230

  • Overlapping blocks: 228
    16:24:28-939665 WARNING LoRA training has only currently been validated for LLaMA, OPT, GPT-J, and GPT-NeoX models.
    (Found model type: LlamaServer)
    *** LoRA: 1 ***
    16:24:33-942140 INFO Loading text file…
    Precise raw text slicer: ON
    Sentences: 2967
    Text Blocks: 230
  • Overlapping blocks: 228
    Traceback (most recent call last):
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\queueing.py”, line 580, in process_events
    response = await route_utils.call_process_api(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\route_utils.py”, line 276, in call_process_api
    output = await app.get_blocks().process_api(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\blocks.py”, line 1928, in process_api
    result = await self.call_function(
    ^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\blocks.py”, line 1526, in call_function
    prediction = await utils.async_iteration(iterator)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py”, line 657, in async_iteration
    return await iterator.anext()
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py”, line 650, in anext
    return await anyio.to_thread.run_sync(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\anyio\to_thread.py”, line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\anyio_backends_asyncio.py”, line 2470, in run_sync_in_worker_thread
    return await future
    ^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\anyio_backends_asyncio.py”, line 967, in run
    result = context.run(func, *args)
    ^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py”, line 633, in run_sync_iterator_async
    return next(iterator)
    ^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py”, line 816, in gen_wrapper
    response = next(iterator)
    ^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\extensions\Training_PRO\script.py”, line 704, in do_train
    train_data = Dataset.from_list([tokenize(x, add_EOS_to_all, add_bos_token) for x in text_chunks])
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\extensions\Training_PRO\script.py”, line 704, in
    train_data = Dataset.from_list([tokenize(x, add_EOS_to_all, add_bos_token) for x in text_chunks])
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\extensions\Training_PRO\script.py”, line 623, in tokenize
    input_ids = encode(prompt, prepend_bos_token)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File “C:\Oobabooga\text-generation-webui-main\extensions\Training_PRO\script.py”, line 613, in encode
    if len(result) >= 2 and result[:2] == [shared.tokenizer.bos_token_id, shared.tokenizer.bos_token_id]:
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    AttributeError: ‘LlamaServer’ object has no attribute ‘bos_token_id’
1 Like
2

From a quick read of the code, I don’t think training a GGUF-quantized model is intended. How about trying it with the Transoformers-format model before GGUF quantization?

3

Thank you for the reply. I also tried training using a transformers based GPTQ model. I received several errors attempting to train this format as well. I will try and get them posted. At least I know where not to waste my time now.

1 Like