I am getting the warning above after finetuning a model for multiclassification ?
I tried to use it on non annotated data ? some comments says it is nothing but I am bit worried since I will send my pipeline to be integrated in an application ?
Some weights of the model checkpoint at /gpfswork/rech/kpf/umg16uw/results_hf/sm/checkpoint-10 were not used when initializing FlaubertForSequenceClassification: ['encoder.ffns.1.lin2.weight', 'encoder.ffns.0.lin1.bias', 'encoder.layer_norm2.2.bias', 'encoder.attentions.2.k_lin.weight', 'encoder.ffns.0.lin2.bias', 'encoder.attentions.0.k_lin.bias', 'encoder.attentions.2.v_lin.bias', 'encoder.layer_norm1.4.weight', 'classifier.0.bias', 'encoder.layer_norm_emb.weight', 'encoder.attentions.2.k_lin.bias', 'encoder.attentions.0.q_lin.weight', 'encoder.ffns.2.lin2.bias', 'encoder.layer_norm1.4.bias', 'encoder.attentions.4.k_lin.bias', 'encoder.layer_norm1.1.bias', 'classifier.3.bias', 'encoder.attentions.3.out_lin.weight', 'encoder.layer_norm2.4.bias', 'encoder.attentions.1.q_lin.weight', 'encoder.attentions.0.k_lin.weight', 'encoder.attentions.4.q_lin.bias', 'encoder.ffns.5.lin2.bias', 'encoder.layer_norm2.1.weight', 'encoder.attentions.0.v_lin.weight', 'encoder.ffns.3.lin1.weight', 'encoder.ffns.1.lin2.bias', 'encoder.layer_norm2.3.bias', 'encoder.ffns.0.lin1.weight', 'encoder.layer_norm1.0.bias', 'encoder.attentions.3.k_lin.weight', 'encoder.ffns.4.lin2.bias', 'encoder.layer_norm1.2.bias', 'encoder.attentions.1.k_lin.weight', 'encoder.ffns.0.lin2.weight', 'encoder.attentions.3.v_lin.weight', 'encoder.attentions.1.v_lin.bias', 'encoder.attentions.2.q_lin.weight', 'encoder.ffns.4.lin1.bias', 'encoder.attentions.1.out_lin.bias', 'encoder.layer_norm2.3.weight', 'encoder.embeddings.weight', 'classifier.0.weight', 'encoder.attentions.4.v_lin.bias', 'encoder.layer_norm2.5.weight', 'encoder.layer_norm1.1.weight', 'encoder.attentions.4.k_lin.weight', 'encoder.attentions.1.q_lin.bias', 'encoder.attentions.3.q_lin.weight', 'encoder.attentions.0.out_lin.bias', 'encoder.attentions.4.q_lin.weight', 'encoder.ffns.5.lin1.weight', 'encoder.ffns.2.lin1.bias', 'encoder.layer_norm1.0.weight', 'encoder.attentions.0.q_lin.bias', 'encoder.attentions.4.out_lin.bias', 'encoder.ffns.3.lin1.bias', 'encoder.layer_norm2.4.weight', 'encoder.attentions.2.out_lin.weight', 'encoder.layer_norm1.3.weight', 'encoder.position_embeddings.weight', 'encoder.ffns.1.lin1.bias', 'encoder.layer_norm_emb.bias', 'encoder.attentions.3.q_lin.bias', 'encoder.attentions.3.v_lin.bias', 'encoder.attentions.4.out_lin.weight', 'encoder.attentions.3.out_lin.bias', 'encoder.ffns.2.lin2.weight', 'encoder.ffns.2.lin1.weight', 'encoder.layer_norm2.0.bias', 'encoder.layer_norm1.3.bias', 'encoder.attentions.5.v_lin.weight', 'encoder.ffns.5.lin2.weight', 'encoder.layer_norm2.2.weight', 'encoder.ffns.3.lin2.bias', 'encoder.attentions.5.q_lin.weight', 'encoder.attentions.0.out_lin.weight', 'encoder.attentions.2.q_lin.bias', 'encoder.attentions.5.v_lin.bias', 'encoder.attentions.2.v_lin.weight', 'encoder.attentions.5.k_lin.bias', 'encoder.layer_norm2.1.bias', 'encoder.attentions.1.k_lin.bias', 'encoder.attentions.1.v_lin.weight', 'encoder.layer_norm2.5.bias', 'encoder.layer_norm2.0.weight', 'encoder.attentions.0.v_lin.bias', 'encoder.ffns.3.lin2.weight', 'classifier.3.weight', 'encoder.attentions.2.out_lin.bias', 'encoder.ffns.5.lin1.bias', 'encoder.layer_norm1.5.weight', 'encoder.ffns.4.lin2.weight', 'encoder.layer_norm1.5.bias', 'encoder.position_ids', 'encoder.attentions.5.q_lin.bias', 'encoder.attentions.3.k_lin.bias', 'encoder.attentions.5.out_lin.bias', 'encoder.ffns.1.lin1.weight', 'encoder.ffns.4.lin1.weight', 'encoder.attentions.4.v_lin.weight', 'encoder.layer_norm1.2.weight', 'encoder.attentions.5.out_lin.weight', 'encoder.attentions.5.k_lin.weight', 'encoder.attentions.1.out_lin.weight']
- This IS expected if you are initializing FlaubertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing FlaubertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of FlaubertForSequenceClassification were not initialized from the model checkpoint at /Flaubert/finetune/Multiclass/checkpoint-10 and are newly initialized: ['sequence_summary.summary.bias', 'sequence_summary.summary.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
***** Running Prediction *****
Num examples = 192
Batch size = 8
This does not appear whan I did the finetune.