How to view the changes in a model after training?

Beginners
1

Hello,

I trained a BART model (facebook-cnn) for summarization and compared summaries with a pretrained model

model_before_tuning_1 = AutoModelForSeq2SeqLM.from_pretrained(model_name)

trainer = Seq2SeqTrainer(
model=model,
args=training_args,
data_collator=data_collator,
train_dataset=train_data,
eval_dataset=validation_data,
tokenizer=tokenizer,
compute_metrics=compute_metrics,
)

trainer.train()

Summaries from model() and model_before_tuning_1() are different but when i compare the model config and/or print(model) it gives exact same things for both.

How to know, what exact parameters have this training changed?

2

When fine-tuning a Transformer-based model, such as BART, all parameters of the model are updated. This means, any tensor that is present in model.parameters() can have updated values after fine-tuning.

The configuration of a model (config.json) before and after fine-tuning can be identical. The configuration just defines basic hyperparameters such as the number of hidden layers, the number of attention heads, etc.

3

so i should check model.parameters() for the difference? thanks

4

Yes, indeed.

5

I checked (model.parameters) and its coming exactly identical