How to export mistralai/Mistral-7B-v0.1 to Tflite to use in TensorFlow Autocomplete?

🤗Optimum
1

I tried exporting mistralai/Mistral-7B-v0.1 to Tflite to use in TensorFlow Autocomplete? (Auto Complete  |  TensorFlow Lite)

But I get this errors:

root@b8db2089dc9c:/workspace# optimum-cli export tflite --model mistralai/Mistral-7B-v0.1  mistral_onnx4

2024-02-09 17:36:37.060810: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-02-09 17:36:37.715109: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-02-09 17:36:41.122922: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/venv/lib/python3.10/site-packages/optimum/exporters/tflite/__main__.py", line 146, in <module>
    main()
  File "/venv/lib/python3.10/site-packages/optimum/exporters/tflite/__main__.py", line 61, in main
    model = TasksManager.get_model_from_task(
  File "/venv/lib/python3.10/site-packages/optimum/exporters/tasks.py", line 1822, in get_model_from_task
    model = model_class.from_pretrained(model_name_or_path, **kwargs)
  File "/venv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 569, in from_pretrained
    raise ValueError(
ValueError: Unrecognized configuration class <class 'transformers.models.mistral.configuration_mistral.MistralConfig'> for this kind of AutoModel: TFAutoModelForCausalLM.
Model type should be one of BertConfig, CamembertConfig, CTRLConfig, GPT2Config, GPT2Config, GPTJConfig, OpenAIGPTConfig, OPTConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoFormerConfig, TransfoXLConfig, XGLMConfig, XLMConfig, XLMRobertaConfig, XLNetConfig.
Traceback (most recent call last):
  File "/venv/bin/optimum-cli", line 8, in <module>
    sys.exit(main())
  File "/venv/lib/python3.10/site-packages/optimum/commands/optimum_cli.py", line 163, in main
    service.run()
  File "/venv/lib/python3.10/site-packages/optimum/commands/export/tflite.py", line 243, in run
    subprocess.run(full_command, shell=True, check=True)
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'python3 -m optimum.exporters.tflite --model mistralai/Mistral-7B-v0.1 mistral_onnx4' returned non-zero exit status 1.

I able to export to ONNX but tflite not working

My sys info:
Linux b8db2089dc9c 5.15.0-84-generic #93~20.04.1-Ubuntu SMP Wed Sep 6 16:15:40 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

please help me

2

I also would like to have a tflite version of mistral… did you find a way?