[Error about InformerForPredict][Bug]

When I set the infomerconfig.input_size = 1, I find a bug, but I don’t know how to fix it.

  • Function Name : create_network_inputs
time_feat = (
            torch.cat(
                (
                    past_time_features[:, self._past_length - self.config.context_length :, ...],
                    future_time_features,
                ),
                dim=1,
            )
            if future_values is not None
            else past_time_features[:, self._past_length - self.config.context_length :, ...]
        )

        print(self._past_length)
        # target
        if past_observed_mask is None:
            past_observed_mask = torch.ones_like(past_values)

        context = past_values[:, -self.config.context_length :]
        observed_context = past_observed_mask[:, -self.config.context_length :]
        _, loc, scale = self.scaler(context, observed_context)

        inputs = (
            (torch.cat((past_values, future_values), dim=1) - loc) / scale
            if future_values is not None
            else (past_values - loc) / scale
        )
        print(loc.shape, scale.shape, inputs.shape)

        # static features
        log_abs_loc = loc.abs().log1p() if self.config.input_size == 1 else loc.squeeze(1).abs().log1p()
        log_scale = scale.log() if self.config.input_size == 1 else scale.squeeze(1).log()
        print(f"log_abs_loc: {log_abs_loc.shape}, {log_scale.shape}")
        print(time_feat.shape, self.config.input_size)
        static_feat = torch.cat((log_abs_loc, log_scale), dim=1)
        print(time_feat.shape, static_feat.shape)
        if static_real_features is not None:
            static_feat = torch.cat((static_real_features, static_feat), dim=1)
        if static_categorical_features is not None:
            embedded_cat = self.embedder(static_categorical_features)
            static_feat = torch.cat((embedded_cat, static_feat), dim=1)
        print(time_feat.shape, static_feat.shape)
        expanded_static_feat = static_feat.unsqueeze(1).expand(-1, time_feat.shape[1], -1)

        # all features
        features = torch.cat((expanded_static_feat, time_feat), dim=-1)

        # lagged features
        subsequences_length = (
            self.config.context_length + self.config.prediction_length
            if future_values is not None
            else self.config.context_length
        )
        lagged_sequence = self.get_lagged_subsequences(sequence=inputs, subsequences_length=subsequences_length)
        lags_shape = lagged_sequence.shape
        reshaped_lagged_sequence = lagged_sequence.reshape(lags_shape[0], lags_shape[1], -1)

        if reshaped_lagged_sequence.shape[1] != time_feat.shape[1]:
            raise ValueError(
                f"input length {reshaped_lagged_sequence.shape[1]} and time feature lengths {time_feat.shape[1]} does not match"
            )

        # transformer inputs
        transformer_inputs = torch.cat((reshaped_lagged_sequence, features), dim=-1)

        return transformer_inputs, loc, scale, static_feat

As we can see, I add some print sentence in the library to see the shape, now the bug is:

Traceback (most recent call last):
  File "/home/wjt/luck/FinalWork/alert_models/informer_based_model_3_cpu.py", line 820, in <module>
    pipline.train_model()
  File "/home/wjt/luck/FinalWork/alert_models/informer_based_model_3_cpu.py", line 466, in train_model
    outputs = model(
  File "/home/wjt/.conda/envs/luckluck/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/wjt/.conda/envs/luckluck/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/wjt/.conda/envs/luckluck/lib/python3.9/site-packages/transformers/models/informer/modeling_informer.py", line 1844, in forward
    outputs = self.model(
  File "/home/wjt/.conda/envs/luckluck/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/wjt/.conda/envs/luckluck/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/wjt/.conda/envs/luckluck/lib/python3.9/site-packages/transformers/models/informer/modeling_informer.py", line 1568, in forward
    transformer_inputs, loc, scale, static_feat = self.create_network_inputs(
  File "/home/wjt/.conda/envs/luckluck/lib/python3.9/site-packages/transformers/models/informer/modeling_informer.py", line 1386, in create_network_inputs
    expanded_static_feat = static_feat.unsqueeze(1).expand(-1, time_feat.shape[1], -1)
RuntimeError: expand(torch.cuda.FloatTensor{[32, 1, 2, 1]}, size=[-1, 27, -1]): the number of sizes provided (3) must be greater or equal to the number of dimensions in the tensor (4)
  • First
log_abs_loc = loc.abs().log1p() if self.config.input_size == 1 else loc.squeeze(1).abs().log1p() log_scale = scale.log() if self.config.input_size == 1 else scale.squeeze(1).log()

If the input_size is 1, loc.squeeze will not execute, but now the loc.shape is (bsz, 1, 1), so it will be a hidden issue, and

  • second
static_feat.unsqueeze(1) will execute, then the error will be reported

I don’t know whether it’s clear to clarify the problem, but I really need a help.

  • environment
Name: transformers
Version: 4.52.3
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: transformers@huggingface.co
License: Apache 2.0 License
Location: /home/wjt/.conda/envs/luckluck/lib/python3.9/site-packages
Requires: filelock, huggingface-hub, numpy, pack

Linux

python3.9.21
1 Like

It looks like you have a reply.