God, prepare_model_for_kbit_training dosen’t support quantized model 
(llm4decompile) root@autodl-container-392d4eb87e-ee6d7970:~/autodl-tmp# python train_pt.py
2025-02-06 15:17:44.694429: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-02-06 15:17:44.711058: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1738826264.729778 1236 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1738826264.735138 1236 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-02-06 15:17:44.754843: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Traceback (most recent call last):
File "/root/miniconda3/envs/llm4decompile/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1817, in _get_module
return importlib.import_module("." + module_name, self.__name__)
File "/root/miniconda3/envs/llm4decompile/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/root/miniconda3/envs/llm4decompile/lib/python3.9/site-packages/transformers/generation/streamers.py", line 231, in <module>
class AsyncTextIteratorStreamer(TextStreamer):
File "/root/miniconda3/envs/llm4decompile/lib/python3.9/site-packages/transformers/generation/streamers.py", line 285, in AsyncTextIteratorStreamer
self, tokenizer: "AutoTokenizer", skip_prompt: bool = False, timeout: float | None = None, **decode_kwargs
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/root/autodl-tmp/train_pt.py", line 1, in <module>
from transformers import *
File "<frozen importlib._bootstrap>", line 1053, in _handle_fromlist
File "<frozen importlib._bootstrap>", line 1055, in _handle_fromlist
File "/root/miniconda3/envs/llm4decompile/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1806, in __getattr__
value = getattr(module, name)
File "/root/miniconda3/envs/llm4decompile/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1805, in __getattr__
module = self._get_module(self._class_to_module[name])
File "/root/miniconda3/envs/llm4decompile/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1819, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.generation.streamers because of the following error (look up to see its traceback):
unsupported operand type(s) for |: 'type' and 'NoneType'
/root/miniconda3/envs/llm4decompile/lib/python3.10/site-packages/transformers/training_args.py:1575: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
PyTorch: setting up devices
The default value for the training argument `--report_to` will change in v5 (from all installed integrations to none). In v5, you will need to use `--report_to all` to get the same behavior as now. You should start updating your code and make this info disappear :-).
Traceback (most recent call last):
File "/root/autodl-tmp/train_pt.py", line 85, in <module>
trainer = Trainer(
File "/root/miniconda3/envs/llm4decompile/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 165, in wrapped_func
return func(*args, **kwargs)
File "/root/miniconda3/envs/llm4decompile/lib/python3.10/site-packages/transformers/trainer.py", line 553, in __init__
raise ValueError(
ValueError: You cannot perform fine-tuning on purely quantized models. Please attach trainable adapters on top of the quantized model to correctly perform fine-tuning. Please see: https://huggingface.co/docs/transformers/peft for more details
from transformers import *
from peft import *
import torch
from datasets import load_dataset
import os
from torch.utils.data import DataLoader
from transformers import default_data_collator, get_linear_schedule_with_warmup
from tqdm import tqdm
from datasets import load_dataset
from tensorboard import *
device = "cuda"
tokenizer_name_or_path = "LLM4Binary/llm4decompile-1.3b-v1.5"
model_name_or_path = "LLM4Binary/llm4decompile-1.3b-v1.5"
dataset_name = "asm2c"
text_column = "asm text"
label_column = "text_label"
max_length = 64
lr = 3e-2
num_epochs = 50
batch_size = 8
from datasets import load_dataset
dataset = load_dataset("json", data_files="./traindata.jsonl")
dataset = dataset["train"].train_test_split(0.2)
tokenizer = AutoTokenizer.from_pretrained("LLM4Binary/llm4decompile-1.3b-v1.5")
def preprocess_function(examples):
inputs = examples["input"]
outputs = examples["output"]
# 合并input和output列
merged_texts = [f"{input} {output_text}" for input, output_text in zip(inputs, outputs)]
model_inputs = tokenizer(merged_texts, truncation=True, padding="max_length", max_length=512)
model_inputs["labels"] = model_inputs["input_ids"].copy() # 设置labels
return model_inputs
processed_datasets = dataset.map(
preprocess_function,
batched=True,
num_proc=1,
remove_columns=dataset["train"].column_names,
load_from_cache_file=False,
desc="Running tokenizer on dataset",
)
train_dataset = processed_datasets["train"]
eval_dataset = processed_datasets["test"]
peft_config = PromptTuningConfig(
task_type=TaskType.CAUSAL_LM,
prompt_tuning_init=PromptTuningInit.TEXT,
num_virtual_tokens=8,
prompt_tuning_init_text="What's the souce code of this asm?",
tokenizer_name_or_path=model_name_or_path,
)
checkpoint_name = f"{dataset_name}_{model_name_or_path}_{peft_config.peft_type}_{peft_config.task_type}_v1.pt".replace(
"/", "_"
)
# creating model
model = AutoModelForCausalLM.from_pretrained("LLM4Binary/llm4decompile-1.3b-v1.5", load_in_8bit=True, torch_dtype=torch.float16, device_map="auto")
model = prepare_model_for_kbit_training(model)
peft_model = get_peft_model(model, peft_config)
training_args = TrainingArguments(
output_dir="./results4", # 保存模型的目录
evaluation_strategy="epoch", # 每个 epoch 进行评估
save_strategy="epoch", # 每个 epoch 结束时保存模型
learning_rate=2e-5,
per_device_train_batch_size=4, # 训练时的batch_size
per_device_eval_batch_size=8, # 验证时的batch_size
logging_steps=10, # log 打印的频率
num_train_epochs=3,
weight_decay=0.01,
load_best_model_at_end=False
)
trainer = Trainer(
model=model,
args=training_args,
train_dataset=train_dataset,
eval_dataset=eval_dataset,
#data_collator=DataCollatorForSeq2Seq(tokenizer=tokenizer, padding=True)
)
trainer.train()
'''
trainer.evaluate(eval_dataset)
# 训练结束后手动保存模型
trainer.save_model(output_dir="./tuned_model") # 保存最终的模型到指定的目录
tokenizer.save_pretrained(save_directory="./tuned_tokenizer") # 保存tokenizer
'''
lora_adapter = "./lora_adapter"
peft_model.save_pretrained(lora_adapter, save_adapter=True, save_config=True)
model_to_merge = PeftModel.from_pretrained(AutoModelForCausalLM.from_pretrained(model_name_or_path).to("cuda"), lora_adapter)
merged_model = model_to_merge.merge_and_unload()
merged_model.save_pretrained(merged_model)
“merge_and_unload” method seems not to be supported.
(llm4decompile) root@autodl-container-392d4eb87e-ee6d7970:~/autodl-tmp# python merge.py
PeftModelForCausalLM(
(base_model): LlamaForCausalLM(
(model): LlamaModel(
(embed_tokens): Embedding(32256, 2048)
(layers): ModuleList(
(0-23): 24 x LlamaDecoderLayer(
(self_attn): LlamaAttention(
(q_proj): Linear(in_features=2048, out_features=2048, bias=False)
(k_proj): Linear(in_features=2048, out_features=2048, bias=False)
(v_proj): Linear(in_features=2048, out_features=2048, bias=False)
(o_proj): Linear(in_features=2048, out_features=2048, bias=False)
)
(mlp): LlamaMLP(
(gate_proj): Linear(in_features=2048, out_features=5504, bias=False)
(up_proj): Linear(in_features=2048, out_features=5504, bias=False)
(down_proj): Linear(in_features=5504, out_features=2048, bias=False)
(act_fn): SiLU()
)
(input_layernorm): LlamaRMSNorm((2048,), eps=1e-06)
(post_attention_layernorm): LlamaRMSNorm((2048,), eps=1e-06)
)
)
(norm): LlamaRMSNorm((2048,), eps=1e-06)
(rotary_emb): LlamaRotaryEmbedding()
)
(lm_head): Linear(in_features=2048, out_features=32256, bias=False)
)
(prompt_encoder): ModuleDict(
(default): PromptEmbedding(
(embedding): Embedding(8, 2048)
)
)
(word_embeddings): Embedding(32256, 2048)
)
Traceback (most recent call last):
File "/root/miniconda3/envs/llm4decompile/lib/python3.10/site-packages/peft/peft_model.py", line 824, in __getattr__
return super().__getattr__(name) # defer to nn.Module's logic
File "/root/miniconda3/envs/llm4decompile/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1695, in __getattr__
raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'PeftModelForCausalLM' object has no attribute 'merge_and_unload'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/autodl-tmp/merge.py", line 15, in <module>
model = model.merge_and_unload()
File "/root/miniconda3/envs/llm4decompile/lib/python3.10/site-packages/peft/peft_model.py", line 828, in __getattr__
return getattr(self.base_model, name)
File "/root/miniconda3/envs/llm4decompile/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1695, in __getattr__
raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'LlamaForCausalLM' object has no attribute 'merge_and_unload'
, so the safetensors file is basically a version that has had as much of the dangerous parts removed as possible.
The reason you are getting quantization errors is either because quantization is being applied when loading, or your LoRA is quantized. The way to solve quantization… I think you can solve simple ones just by specifying torch_dtype.