Fail to load from custom pretrained model

Beginners
1

Hi,
I have pretrained a model using transformers example.
And I got a custom model like below.

$  CUDA_VISIBLE_DEVICES="0,1,2" accelerate launch run_mlm_no_trainer.py \
    --model_type bert, \
    --tokenizer_name ./tokenizer/wordpiece/ \
    --train_file ./test/ \
    --checkpointing_steps 1000 \
    --validation_split_percentage 10 \
    --max_seq_length 512 \
    --per_device_train_batch_size 64 \
    --per_device_eval_batch_size 64 \
    --output_dir ./result/ \
    --preprocessing_num_workers 32 \
    --seed 42 \
    --num_train_epochs 2

$ ls -l saved_model
-rw-r–r–. 1 ps ps 34 Mar 11 14:35 all_results.json
-rw-r–r–. 1 ps ps 589 Mar 11 14:35 config.json
-rw-r–r–. 1 ps ps 90 Mar 11 14:35 generation_config.json
-rw-r–r–. 1 ps ps 374369196 Mar 11 14:35 model.safetensors
-rw-r–r–. 1 ps ps 695 Mar 11 14:35 special_tokens_map.json
-rw-r–r–. 1 ps ps 714913 Mar 11 14:35 tokenizer.json
-rw-r–r–. 1 ps ps 1264 Mar 11 14:35 tokenizer_config.json
-rw-r–r–. 1 ps ps 211389 Mar 11 14:35 vocab.txt

Now I want to load the model, but RuntimeError occurs.

from transformers import AutoModel, AutoConfig
config = AutoConfig.from_pretrained(‘/opt/patbert/bert/test_result2/’)
model = AutoModel.from_pretrained(‘/opt/patbert/bert/test_result2/’, config)

RuntimeError Traceback (most recent call last)
Cell In[4], line 4
1 from transformers import AutoTokenizer, AutoModel, AutoConfig
3 config = AutoConfig.from_pretrained(‘saved_model’)
----> 4 model = AutoModel.from_pretrained(‘saved_model’, config)

File ~/.conda/envs/pyenv38/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py:566, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
564 elif type(config) in cls._model_mapping.keys():
565 model_class = _get_model_class(config, cls._model_mapping)
→ 566 return model_class.from_pretrained(
567 pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
568 )
569 raise ValueError(
570 f"Unrecognized configuration class {config.class} for this kind of AutoModel: {cls.name}.\n"
571 f"Model type should be one of {', '.join(c.name for c in cls._model_mapping.keys())}."
572 )

File ~/.conda/envs/pyenv38/lib/python3.8/site-packages/transformers/modeling_utils.py:3706, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, *model_args, **kwargs)
3697 if dtype_orig is not None:
3698 torch.set_default_dtype(dtype_orig)
3699 (
3700 model,
3701 missing_keys,
3702 unexpected_keys,
3703 mismatched_keys,
3704 offload_index,
3705 error_msgs,
→ 3706 ) = cls._load_pretrained_model(
3707 model,
3708 state_dict,
3709 loaded_state_dict_keys, # XXX: rename?
3710 resolved_archive_file,
3711 pretrained_model_name_or_path,
3712 ignore_mismatched_sizes=ignore_mismatched_sizes,
3713 sharded_metadata=sharded_metadata,
3714 _fast_init=_fast_init,
3715 low_cpu_mem_usage=low_cpu_mem_usage,
3716 device_map=device_map,
3717 offload_folder=offload_folder,
3718 offload_state_dict=offload_state_dict,
3719 dtype=torch_dtype,
3720 is_quantized=(getattr(model, “quantization_method”, None) == QuantizationMethod.BITS_AND_BYTES),
3721 keep_in_fp32_modules=keep_in_fp32_modules,
3722 )
3724 model.is_loaded_in_4bit = load_in_4bit
3725 model.is_loaded_in_8bit = load_in_8bit

File ~/.conda/envs/pyenv38/lib/python3.8/site-packages/transformers/modeling_utils.py:4166, in PreTrainedModel._load_pretrained_model(cls, model, state_dict, loaded_keys, resolved_archive_file, pretrained_model_name_or_path, ignore_mismatched_sizes, sharded_metadata, _fast_init, low_cpu_mem_usage, device_map, offload_folder, offload_state_dict, dtype, is_quantized, keep_in_fp32_modules)
4162 if “size mismatch” in error_msg:
4163 error_msg += (
4164 “\n\tYou may consider adding ignore_mismatched_sizes=True in the model from_pretrained method.”
4165 )
→ 4166 raise RuntimeError(f"Error(s) in loading state_dict for {model.class.name}:\n\t{error_msg}")
4168 if is_quantized:
4169 unexpected_keys = [elem for elem in unexpected_keys if “SCB” not in elem]

RuntimeError: Error(s) in loading state_dict for BertModel:
size mismatch for bert.embeddings.word_embeddings.weight: copying a param with shape torch.Size([8532139]) from checkpoint, the shape in current model is torch.Size([32000, 768]).
size mismatch for bert.embeddings.position_embeddings.weight: copying a param with shape torch.Size([0]) from checkpoint, the shape in current model is torch.Size([512, 768]).
You may consider adding ignore_mismatched_sizes=True in the model from_pretrained method.

Of course, if i use ignore_mismatched_sizes=True, the model can be loaded.
Is it common to use like this?