As an option, here’s some info for you. To convert your input data, represented as a 1D tensor of size 768, into the format expected by the transformer (dimensions (S, E)), where S is the length of the sequence and E is the dimension of the embedding vector, you will need to perform several steps.