"run_lm_finetuning.py" was replaced?

kintaro · September 4, 2020, 2:55pm

Hello to all,

Looking at tutorials and examples I see that many do fine tuning with run_lm_finetuning.py. However when I install the latest version of transformers this file doesn’t exist, I haven’t found it in the github repository either.

So my question is if this file was discontinued, and, if so, what is the actual .py file that should be used for fine tuning of a language model?

thomwolf · September 4, 2020, 5:14pm

Hi,
It’s now here: https://github.com/huggingface/transformers/tree/master/examples/language-modeling

kintaro · September 4, 2020, 10:01pm

Great!!! Thanks a lot for your help

Any reason for the change?

thomwolf · September 7, 2020, 6:52am

I think it’s because the script can also be used for training from scratch so having “fintetuning” in the name made it not very clear for many.

kintaro · September 8, 2020, 6:28pm

Thanks a lot for you guidance

krishnagarg09 · June 1, 2021, 4:21pm

The link is outdated.

This is the new link:

github.com

huggingface/transformers/blob/27c1b656cca75efa0cc414d3bf4e6aacf24829de/examples/run_lm_finetuning.py

# coding=utf-8
# Copyright 2018 The Google AI Language Team Authors and The HuggingFace Inc. team.
# Copyright (c) 2018, NVIDIA CORPORATION.  All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
Fine-tuning the library models for language modeling on a text file (GPT, GPT-2, BERT, RoBERTa).
GPT and GPT-2 are fine-tuned using a causal language modeling (CLM) loss while BERT and RoBERTa are fine-tuned
using a masked language modeling (MLM) loss.
"""

This file has been truncated. show original

Topic		Replies	Views
Fine-tune BERT for Masked Language Modeling 🤗Transformers	3	2985	January 25, 2021
Language-modeling script "killed" when fine-tuning gpt2-medium Beginners	3	6779	May 19, 2023
Missing examples in transformers/examples/language-modeling Beginners	2	509	April 22, 2021
Loading pre-trained BERT model error - Error no file named ['pytorch_model.bin', 'tf_model.h5'] found Beginners	0	3721	December 1, 2020
How to fine-tune a pretrained LLM on custom code libraries? Beginners	2	4967	April 29, 2024

"run_lm_finetuning.py" was replaced?

Related topics