"run_lm_finetuning.py" was replaced?

kintaro · September 4, 2020, 2:55pm

Hello to all,

Looking at tutorials and examples I see that many do fine tuning with run_lm_finetuning.py. However when I install the latest version of transformers this file doesn’t exist, I haven’t found it in the github repository either.

So my question is if this file was discontinued, and, if so, what is the actual .py file that should be used for fine tuning of a language model?

thomwolf · September 4, 2020, 5:14pm

Hi,
It’s now here: https://github.com/huggingface/transformers/tree/master/examples/language-modeling

kintaro · September 4, 2020, 10:01pm

Great!!! Thanks a lot for your help

Any reason for the change?

thomwolf · September 7, 2020, 6:52am

I think it’s because the script can also be used for training from scratch so having “fintetuning” in the name made it not very clear for many.

kintaro · September 8, 2020, 6:28pm

Thanks a lot for you guidance

krishnagarg09 · June 1, 2021, 4:21pm

The link is outdated.

This is the new link:

github.com

huggingface/transformers/blob/27c1b656cca75efa0cc414d3bf4e6aacf24829de/examples/run_lm_finetuning.py

# coding=utf-8
# Copyright 2018 The Google AI Language Team Authors and The HuggingFace Inc. team.
# Copyright (c) 2018, NVIDIA CORPORATION.  All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
Fine-tuning the library models for language modeling on a text file (GPT, GPT-2, BERT, RoBERTa).
GPT and GPT-2 are fine-tuned using a causal language modeling (CLM) loss while BERT and RoBERTa are fine-tuned
using a masked language modeling (MLM) loss.
"""

This file has been truncated. show original

Topic		Replies	Views
Missing examples in transformers/examples/language-modeling Beginners	2	448	April 22, 2021
Loading pre-trained BERT model error - Error no file named ['pytorch_model.bin', 'tf_model.h5'] found Beginners	0	3164	December 1, 2020
Difference between language modeling scripts Models	1	433	December 20, 2021
How to train from scratch with run_mlm.py, .txt file? Beginners	19	5886	January 5, 2023
Do the data-load pipeline have the memory limit? I have not test it, Thank you! Beginners	0	243	May 24, 2023

"run_lm_finetuning.py" was replaced?

Related Topics