Pretrain model not accepting optimizer

🤗Transformers
22

This is the solution that worked for me. Thank you for sharing!

23

Same issue. Tried everything shared before. Not working on google colab for me with Adam using t5-small

24

For me, in movinet_streaming_model_training_and_inference.ipynb, this problem and others concerning checkpoints were solved by starting with this code;

!pip install --upgrade transformers
!pip install tf-keras
import os
os.environ[‘TF_USE_LEGACY_KERAS’] = ‘1’

Thanks to all of you for this discussion

2 Likes
25

Can confirm this works perfectly as of 16 Apr 2024. I put these lines preceding anything in my Colab notebook, restarted kernel, and worked flawlessly. Cheers

26

Fine-tuning a model with Keras - Hugging Face NLP Course in google colab

try everything, but still broken

update: according the issue ValueError: Could not interpret optimizer identifier: <keras.src.optimizers.adam.Adam object at 0x79d9071160e0> ¡ Issue #19262 ¡ keras-team/keras ¡ GitHub
also need update tf_keras

  1. update the depencies
!pip install --upgrade transformers
!pip install --upgrade tf_keras
  1. restart the colab session
  2. using keras 2
import os
os.environ['TF_USE_LEGACY_KERAS'] = '1'

is work for me, PS: this is my version info

run !pip show tensorflow transformers tf_keras

Name: tensorflow
Version: 2.16.1
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: packages@tensorflow.org
License: Apache 2.0
Location: /usr/local/lib/python3.10/dist-packages
Requires: absl-py, astunparse, flatbuffers, gast, google-pasta, grpcio, h5py, keras, libclang, ml-dtypes, numpy, opt-einsum, packaging, protobuf, requests, setuptools, six, tensorboard, tensorflow-io-gcs-filesystem, termcolor, typing-extensions, wrapt
Required-by: dopamine-rl, tf_keras
---
Name: transformers
Version: 4.39.3
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: transformers@huggingface.co
License: Apache 2.0 License
Location: /usr/local/lib/python3.10/dist-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, safetensors, tokenizers, tqdm
Required-by: 
---
Name: tf_keras
Version: 2.16.0
Summary: Deep learning for humans.
Home-page: https://keras.io/
Author: Keras team
Author-email: keras-users@googlegroups.com
License: Apache 2.0
Location: /usr/local/lib/python3.10/dist-packages
Requires: tensorflow
Required-by: tensorflow-hub

if you guys like me also run in the local machine with IDEA,
so restarting the kernel maybe not enough, also need to restart the IDEA

1 Like
27

This worked to me today on Google Colab.
It’s better to start the session and set this in the begining., before import anything else.

My problem was that I was already working in my project with a pipeline in memory and some libs too, so this update didn’t work in the first time. Later I started over and followed the steps as the first thing in the notebook and it worked!

I think Google Colab was still using Tensorflow 2.15 and tf-keras 2.15, but the update was running in 2.16.

28

Hi @Rocketknight1.
I get this issue while running this on a physical machine with Tensorflow 2.14 and Keras 2.14, so this cannot be related to the upgraded Keras.
Transformers version is 4.41.
tf-keras is also installed and setting TF_LEGACY_USE_KERAS does not work.
Is it possible that the pre-trained HF models are just not compatible with TF 2.15 and older that uses Keras 2?

29

I was facing with the same issue, and followed this, and it worded up for me

The compile() excecuted succesfully