This is the solution that worked for me. Thank you for sharing!
Same issue. Tried everything shared before. Not working on google colab for me with Adam using t5-small
For me, in movinet_streaming_model_training_and_inference.ipynb, this problem and others concerning checkpoints were solved by starting with this code;
!pip install --upgrade transformers
!pip install tf-keras
import os
os.environ[‘TF_USE_LEGACY_KERAS’] = ‘1’
Thanks to all of you for this discussion
2 Likes
Can confirm this works perfectly as of 16 Apr 2024. I put these lines preceding anything in my Colab notebook, restarted kernel, and worked flawlessly. Cheers
Fine-tuning a model with Keras - Hugging Face NLP Course in google colab
try everything, but still broken
update: according the issue ValueError: Could not interpret optimizer identifier: <keras.src.optimizers.adam.Adam object at 0x79d9071160e0> · Issue #19262 · keras-team/keras · GitHub
also need update tf_keras
- update the depencies
!pip install --upgrade transformers
!pip install --upgrade tf_keras
- restart the colab session
- using keras 2
import os
os.environ['TF_USE_LEGACY_KERAS'] = '1'
is work for me, PS: this is my version info
run !pip show tensorflow transformers tf_keras
Name: tensorflow
Version: 2.16.1
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: packages@tensorflow.org
License: Apache 2.0
Location: /usr/local/lib/python3.10/dist-packages
Requires: absl-py, astunparse, flatbuffers, gast, google-pasta, grpcio, h5py, keras, libclang, ml-dtypes, numpy, opt-einsum, packaging, protobuf, requests, setuptools, six, tensorboard, tensorflow-io-gcs-filesystem, termcolor, typing-extensions, wrapt
Required-by: dopamine-rl, tf_keras
---
Name: transformers
Version: 4.39.3
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: transformers@huggingface.co
License: Apache 2.0 License
Location: /usr/local/lib/python3.10/dist-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, safetensors, tokenizers, tqdm
Required-by:
---
Name: tf_keras
Version: 2.16.0
Summary: Deep learning for humans.
Home-page: https://keras.io/
Author: Keras team
Author-email: keras-users@googlegroups.com
License: Apache 2.0
Location: /usr/local/lib/python3.10/dist-packages
Requires: tensorflow
Required-by: tensorflow-hub
if you guys like me also run in the local machine with IDEA,
so restarting the kernel maybe not enough, also need to restart the IDEA
1 Like
This worked to me today on Google Colab.
It’s better to start the session and set this in the begining., before import anything else.
My problem was that I was already working in my project with a pipeline in memory and some libs too, so this update didn’t work in the first time. Later I started over and followed the steps as the first thing in the notebook and it worked!
I think Google Colab was still using Tensorflow 2.15 and tf-keras 2.15, but the update was running in 2.16.
Hi @Rocketknight1.
I get this issue while running this on a physical machine with Tensorflow 2.14 and Keras 2.14, so this cannot be related to the upgraded Keras.
Transformers version is 4.41.
tf-keras is also installed and setting TF_LEGACY_USE_KERAS does not work.
Is it possible that the pre-trained HF models are just not compatible with TF 2.15 and older that uses Keras 2?
I was facing with the same issue, and followed this, and it worded up for me
The compile() excecuted succesfully
I am Facing same problems despite trying steps above its not working!
Thank you @rhuanbarros . This worked for me.
1 Like
@Rocketknight1
I was able to fix the first error, however, I encountered this error: FailedPreconditionError Traceback (most recent call last)
in <cell line: 1>()
----> 1 model.fit(tf_train_dataset, validation_data=tf_validation_dataset, epochs=3)
2 frames
/usr/local/lib/python3.10/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
51 try:
52 ctx.ensure_initialized()
—> 53 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
54 inputs, attrs, num_outputs)
55 except core._NotOkStatusException as e:
FailedPreconditionError: Graph execution error:
Detected at node Adam/StatefulPartitionedCall_199 defined at (most recent call last):
File “/usr/lib/python3.10/runpy.py”, line 196, in _run_module_as_main
File “/usr/lib/python3.10/runpy.py”, line 86, in _run_code
File “/usr/local/lib/python3.10/dist-packages/colab_kernel_launcher.py”, line 37, in
File “/usr/local/lib/python3.10/dist-packages/traitlets/config/application.py”, line 992, in launch_instance
File “/usr/local/lib/python3.10/dist-packages/ipykernel/kernelapp.py”, line 619, in start
File “/usr/local/lib/python3.10/dist-packages/tornado/platform/asyncio.py”, line 195, in start
File “/usr/lib/python3.10/asyncio/base_events.py”, line 603, in run_forever
File “/usr/lib/python3.10/asyncio/base_events.py”, line 1909, in _run_once
File “/usr/lib/python3.10/asyncio/events.py”, line 80, in _run
File “/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py”, line 685, in
File “/usr/local/lib/python3.10/dist-packages/tornado/ioloop.py”, line 738, in _run_callback
File “/usr/local/lib/python3.10/dist-packages/tornado/gen.py”, line 825, in inner
File “/usr/local/lib/python3.10/dist-packages/tornado/gen.py”, line 786, in run
File “/usr/local/lib/python3.10/dist-packages/ipykernel/kernelbase.py”, line 361, in process_one
File “/usr/local/lib/python3.10/dist-packages/tornado/gen.py”, line 234, in wrapper
File “/usr/local/lib/python3.10/dist-packages/ipykernel/kernelbase.py”, line 261, in dispatch_shell
File “/usr/local/lib/python3.10/dist-packages/tornado/gen.py”, line 234, in wrapper
File “/usr/local/lib/python3.10/dist-packages/ipykernel/kernelbase.py”, line 539, in execute_request
File “/usr/local/lib/python3.10/dist-packages/tornado/gen.py”, line 234, in wrapper
File “/usr/local/lib/python3.10/dist-packages/ipykernel/ipkernel.py”, line 302, in do_execute
File “/usr/local/lib/python3.10/dist-packages/ipykernel/zmqshell.py”, line 539, in run_cell
File “/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py”, line 2975, in run_cell
File “/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py”, line 3030, in _run_cell
File “/usr/local/lib/python3.10/dist-packages/IPython/core/async_helpers.py”, line 78, in _pseudo_sync_runner
File “/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py”, line 3257, in run_cell_async
File “/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py”, line 3473, in run_ast_nodes
File “/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py”, line 3553, in run_code
File “”, line 1, in <cell line: 1>
File “/usr/local/lib/python3.10/dist-packages/transformers/modeling_tf_utils.py”, line 1229, in fit
File “/usr/local/lib/python3.10/dist-packages/tf_keras/src/utils/traceback_utils.py”, line 65, in error_handler
File “/usr/local/lib/python3.10/dist-packages/tf_keras/src/engine/training.py”, line 1804, in fit
File “/usr/local/lib/python3.10/dist-packages/tf_keras/src/engine/training.py”, line 1398, in train_function
File “/usr/local/lib/python3.10/dist-packages/tf_keras/src/engine/training.py”, line 1381, in step_function
File “/usr/local/lib/python3.10/dist-packages/tf_keras/src/engine/training.py”, line 1370, in run_step
File “/usr/local/lib/python3.10/dist-packages/transformers/modeling_tf_utils.py”, line 1709, in train_step
File “/usr/local/lib/python3.10/dist-packages/tf_keras/src/optimizers/optimizer.py”, line 623, in minimize
File “/usr/local/lib/python3.10/dist-packages/tf_keras/src/optimizers/optimizer.py”, line 1309, in apply_gradients
File “/usr/local/lib/python3.10/dist-packages/tf_keras/src/optimizers/optimizer.py”, line 731, in apply_gradients
File “/usr/local/lib/python3.10/dist-packages/tf_keras/src/optimizers/optimizer.py”, line 1339, in _internal_apply_gradients
File “/usr/local/lib/python3.10/dist-packages/tf_keras/src/optimizers/optimizer.py”, line 1431, in _distributed_apply_gradients_fn
File “/usr/local/lib/python3.10/dist-packages/tf_keras/src/optimizers/optimizer.py”, line 1426, in apply_grad_to_update_var
DNN library initialization failed. Look at the errors above for more details.
[[{{node Adam/StatefulPartitionedCall_199}}]] [Op:__inference_train_function_66757]