I don’t think there are any major issues with quantization. The LoRA settings also appear to be correct. If there are any problems, it usually results in an error…
Regarding tokenization, I later noticed that the handling of special tokens and masks may not be complete.
And since the training loop section is manually created, making debugging difficult… For example, I can assist with fixes for cases where it doesn’t work due to syntax errors, but it’s challenging to identify logical oversights.
Given that manually performing all of this correctly is quite tedious (there are numerous model-specific conventions and details that aren’t typically obvious), I think it would be simpler to implement using an existing trainer.