Model with Genrate method to torchscript

[itsmax]

I’ve been trying to export the MusicGen model to TorchScript with the problem. This model uses generate() method, which could not be converted directly to TorchScript to get results,. I searched for many websites but hadn’t found any solutions to solve this. Does anyone have solutions to cope with? I only have a mind that I have to separate all structures and convert them separately.
I am trying to do this because I will continue to deploy this model to iOS and Andriod.
Maybe I have to found a better model

[John6666]

There are also means such as the ONNX and GGUF formats.

Generate without using the generate method

Posting this here for visibility. What if you want to decode the output of a generative seq2seq model (like T5, BART, etc.) yourself, without using the .generate() method? The code example below illustrates this. Suppose that the model is given a long text, for which it needs to generate a summary. We illustrate here how to manually decode the generated ids autoregressively. In each iteration, we add the predicted token id by the model to the decoder_input_ids, which are then fed as input to th…

How to convert your PyTorch model to TorchScript

Common AI Model Formats

[itsmax]

New updates! Although, I have not found a way to convert Generate, I found a way to self-implement the Generate Function, which means it could be implemented in other forms