How to save my model to use it later

DeleMike · January 14, 2024, 9:38pm

huggingface/transformers/blob/main/examples/pytorch/translation/README.md

<!---
Copyright 2020 The HuggingFace Team. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

## Translation

This directory contains examples for finetuning and evaluating transformers on translation tasks.
Please tag @patil-suraj with any issues/unexpected behaviors, or send a PR!

This file has been truncated. show original

I am using this repo to run a translation task. Especially I’m using it to build a diacritization model. I need to save the model after the process is done. How can I do that?

CUDA_VISIBLE_DEVICES=0 python run_translation.py --model_name_or_path Davlan/oyo-t5-small --do_train --do_eval --source_lang unyo --target_lang dcyo --source_prefix "<unyo2dcyo>: " --train_file data_prep_eng/output_data/bible_train.json --validation_file data_prep_eng/output_data/dev.json --test_file data_prep_eng/output_data/test.json --output_dir oyot5_small_unyo_dcyo_bible --max_source_length 512 --max_target_length 512 --per_device_train_batch_size=24 --per_device_eval_batch_size=24 --num_train_epochs 3 --overwrite_output_dir --predict_with_generate --save_steps 10000 --num_beams 10 --do_predict

Am I missing a flag like --save-model? I need the saved model to be part of the directory.

See what I have now:

Topic		Replies	Views
How to save and load fine-tune model 🤗Transformers	4	24622	October 25, 2021
Saving local bert/roberta model not working using save_pretrained Beginners	0	1061	November 18, 2022
Save a Bert model with custom forward function and heads on Hugginface Intermediate	1	1963	June 7, 2022
An efficient way of loading a model that was saved with torch.save Beginners	4	9845	May 6, 2022
How can I share a pytorch saved model on huggingFace hub Beginners	0	675	May 4, 2022

How to save my model to use it later

Related topics