When I run:
from transformers import FlaxBertModel
bert_checkpoint = "bert-base-cased"
bert_base = FlaxBertModel.from_pretrained(bert_checkpoint)
I get the warning that weights are missing for some layers:
Some weights of FlaxBertModel were not initialized from the model checkpoint at bert-base-cased and are newly initialized: {(‘pooler’, ‘dense’, ‘kernel’), (‘pooler’, ‘dense’, ‘bias’)} You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Am I doing something wrong? Is this checkpoint actually from the PyTorch model, and that’s why some weights are missing? How do I filter the models in the hub with all the weights for the base model?