Modify HF model for training

IanBestFellow · December 22, 2023, 11:02am

Hello!
As a beginner,I have a question.
I want to modify the forward part of T5ForConditionalGeneration for my research.
However, I don’t know how to modified it and Trainer for training.
Can you please guide me on how to modify Seq2SeqTrainer and T5ForConditionalGeneration for training?
Thank you very much.

msthil2 · December 22, 2023, 6:11pm

So as far as I know, you can change the T5ForConditionalGeneration class attributes if that does the job for you. Otherwise, you might have to bite the bullet and maybe create your own class. I guess it depends on what you want to change in the forward pass. If it’s involved, probably have to create your own class I think though.

For reference, this is the source code of the T5ForConditionalGeneration:

github.com

huggingface/transformers/blob/main/src/transformers/models/t5/modeling_t5.py

# coding=utf-8
# Copyright 2018 Mesh TensorFlow authors, T5 Authors and HuggingFace Inc. team.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
""" PyTorch T5 model."""


import copy
import math
import os

This file has been truncated. show original

Topic		Replies	Views
I could not able to use save_pretrained on my T5 Model 🤗Transformers	3	920	October 25, 2021
How to train TFT5ForConditionalGeneration model? 🤗Transformers	5	3066	November 21, 2020
How to conver Pytorch model to Flax Models	1	1649	November 16, 2021
How to use T5ForConditionalGeneration to train your custom model? Beginners	0	215	September 27, 2022
Custom embedding / prompt tuning Beginners	0	1498	September 20, 2021

Modify HF model for training

Related Topics