I’m trying to replicate this article using bloom as the model to fine-tune instead of a GPT model:
However, bloom doesn’t seem to have a double head framework available.
So I figured I’d need to create a custom head on top of the BloomForCausalLM, and the closest I’ve found to this is this thread and article:
I understand that blenderbot and dialoGPT are available models, but from what I’ve seen and tested they’re not the most impressive. I’d like to see what I can do on a custom data set of my own design.
I think what I’m doing should be feasible, but I’m entirely self-taught in programming → machine learning → deep learning, and my method is usually to start with a goal, then figure out how to get there. If someone could at minimum tell me if what I’m doing isn’t even possible, that would save me time continuously trying to figure it out.
Any other help is welcome from learning materials to guidance on this project.