Maybe this is a little to much overkill:
- Save PyTorch GPT2 as pretrained.
- Load the TensorFlow GPT2 from the file I have just saved. (Believing the binaries are framework agnostic).
- Convert the TensorFlow model to TensorflowJS.
Would that work?