Attaching TF models to CNN features

Research
1

This may be not entirely about NLP.
I am working on Image captioning and learning textual representations from CNN features.
Idea is to train CNN using captioning. So, I tried to use GPT-2 tokenizer but I had to create Captioning model from scratch.
Is there any way to attach TF Transformer models to other CV applications for better learning?

My VirTex implementation in Keras

2

Hey @surajp. Sorry, I’m not familiar enough with VirTex to give a concrete response here. But our TF models are compatible with TF2/Keras, so you should be able to include them in your TF graph. If you’re having trouble with this, please post some more specifics and I’ll see if we can be of any more help.

1 Like