TimeSeriesTransformerModel dimensionality issue

🤗Transformers
1

Hi, fairly new to TimeSeriesTransformerModel here but I can’t figure out what’s going on with the model I’m trying to initialise:
It seems like I have to force feature_size to be 73 to match mat1 (understand the math reasoning but I can’t find what feature_size is in the documentation)
Error Code:

image
image1350Ă—972 46.4 KB

However, if I force the feature size to be 73, the output looks empty…
tensor(, size=(1, 0, 64), grad_fn=)

Any help would be appreciated!

Edit:
Does this happen because the prediction len is 0? I’ve noticed the encoder_last_hidden_states is populated… This is all I actually need as I want to take the encoded representation and fit it to a separate fc head… Could someone confirm my thinking that the encoder_last_hidden_states should be the embeddings for the time_series data prior to any prediction?

2

hi @ArgyleR
If you haven’t yet, please see the example Time Series Transformer.

I’m checking your code.

3

cc @kashif