A very strange error when saving the checkpoint

Beginners
1

I am training a NER model using the following scripts:

from transformers import TrainingArguments

args = TrainingArguments(
    "saved_models/bert-finetuned-ner-100examples",
    learning_rate=2e-5,
    num_train_epochs=10,
    weight_decay=0.01,
    
    evaluation_strategy="epoch",
    save_strategy="epoch",
    load_best_model_at_end = True,
    metric_for_best_model = 'f1'
)

from transformers import Trainer

trainer = Trainer(
    model=model,
    args=args,
    train_dataset=tokenized_datasets["train"].select(range(100)),
    eval_dataset=tokenized_datasets["validation"].select(range(100)),
    data_collator=data_collator,
    compute_metrics=compute_metrics,
    tokenizer=tokenizer,
)
trainer.train()

and I got:

File /opt/conda/lib/python3.8/site-packages/transformers/configuration_utils.py:805, in PretrainedConfig.to_json_file(self, json_file_path, use_diff)
    794 def to_json_file(self, json_file_path: Union[str, os.PathLike], use_diff: bool = True):
    795     """
    796     Save this instance to a JSON file.
    797 
   (...)
    803             is serialized to JSON file.
    804     """
--> 805     with open(json_file_path, "w", encoding="utf-8") as writer:
    806         writer.write(self.to_json_string(use_diff=use_diff))

FileNotFoundError: [Errno 2] No such file or directory: 'saved_models/bert-finetuned-ner-100examples/checkpoint-5/config.json'

I just don’t know why this error could happen?

2

change to transformers 4.36.1 may solve this stupid damn issue

1 Like