Pretrain model not accepting optimizer

🀗Transformers
22

This is the solution that worked for me. Thank you for sharing!

23

Same issue. Tried everything shared before. Not working on google colab for me with Adam using t5-small

24

For me, in movinet_streaming_model_training_and_inference.ipynb, this problem and others concerning checkpoints were solved by starting with this code;

!pip install --upgrade transformers
!pip install tf-keras
import os
os.environ[‘TF_USE_LEGACY_KERAS’] = ‘1’

Thanks to all of you for this discussion

1 Like
25

Can confirm this works perfectly as of 16 Apr 2024. I put these lines preceding anything in my Colab notebook, restarted kernel, and worked flawlessly. Cheers

26

Fine-tuning a model with Keras - Hugging Face NLP Course in google colab

try everything, but still broken

update: according the issue ValueError: Could not interpret optimizer identifier: <keras.src.optimizers.adam.Adam object at 0x79d9071160e0> · Issue #19262 · keras-team/keras · GitHub
also need update tf_keras

  1. update the depencies
!pip install --upgrade transformers
!pip install --upgrade tf_keras
  1. restart the colab session
  2. using keras 2
import os
os.environ['TF_USE_LEGACY_KERAS'] = '1'

is work for me, PS: this is my version info

run !pip show tensorflow transformers tf_keras

Name: tensorflow
Version: 2.16.1
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: packages@tensorflow.org
License: Apache 2.0
Location: /usr/local/lib/python3.10/dist-packages
Requires: absl-py, astunparse, flatbuffers, gast, google-pasta, grpcio, h5py, keras, libclang, ml-dtypes, numpy, opt-einsum, packaging, protobuf, requests, setuptools, six, tensorboard, tensorflow-io-gcs-filesystem, termcolor, typing-extensions, wrapt
Required-by: dopamine-rl, tf_keras
---
Name: transformers
Version: 4.39.3
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: transformers@huggingface.co
License: Apache 2.0 License
Location: /usr/local/lib/python3.10/dist-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, safetensors, tokenizers, tqdm
Required-by: 
---
Name: tf_keras
Version: 2.16.0
Summary: Deep learning for humans.
Home-page: https://keras.io/
Author: Keras team
Author-email: keras-users@googlegroups.com
License: Apache 2.0
Location: /usr/local/lib/python3.10/dist-packages
Requires: tensorflow
Required-by: tensorflow-hub

if you guys like me also run in the local machine with IDEA,
so restarting the kernel maybe not enough, also need to restart the IDEA