Hello,
I would like to combine the outputs from a pretrained model like bert-base-uncased with some numerical data to use in a classification task. What would be the suggested way to do to this? Would I need to convert the pretrained model to pytorch and then write my own pytorch module, for example?
thanks in advance for any help!
If your custom model just needs to call Bert without changing any of this stuff, then you could just create a class on top of Bert, where Bert is just an attribute that you call when you need to. But if you want Bert to output a custom loss, logits, etc, then you should make a subclass of Huggingface’s Bert
Would I need to convert the pretrained model to pytorch and then write my own pytorch module, for example?
You shouldn’t need to explicitly convert a pretrained HF model to PyTorch. Whenever you instantiate the model with from_pretrained, it’ll already be a PyTorch module.
On the other hand, whether you need your custom class to be a PyTorch module too depends on what you need. If you want that to be a trainable module, then you’d likely want to create a PyTorch module. But if you just need a quick and easy class to run inference, it probably wouldn’t be necessary.