Issues replicating time series analysis with Transformers

evasahlholdt · May 10, 2023, 10:38am

I am trying to replicate one-to-one the univariate time series analysis using the vanilla Transformer provided in this notebook:

github.com

huggingface/notebooks/blob/main/examples/time-series-transformers.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9-a3wNZ9mxWT",
   "metadata": {
    "id": "9-a3wNZ9mxWT"
   },
   "source": [
    "# Probabilistic Time Series Forecasting with 🤗 Transformers\n",
    "\n",
    "## Introduction\n",
    "\n",
    "Time series forecasting is an essential scientific and business problem and as such has also seen a lot of innovation recently with the use of [deep learning based](https://dl.acm.org/doi/abs/10.1145/3533382) models in addition to the [classical methods](https://otexts.com/fpp3/). An important difference between classical methods like ARIMA and novel deep learning methods is the following.\n",
    "\n",
    "##  Probabilistic Forecasting\n",
    "\n",
    "Typically, classical methods are fitted on each time series in a dataset individually. These are often referred to as  \"single\" or \"local\" methods. However, when dealing with a large amount of time series for some applications, it is beneficial to train a \"global\" model on all available time series, which enables the model to learn latent representations from many different sources.\n",
    "\n",
    "Some classical methods are point-valued (meaning, they just output a single value per time step) and models are trained by minimizing an L2 or L1 type of loss with respect to the ground truth data. However, since forecasts are often used in some real-world decision making pipeline, even with humans in the loop, it is much more beneficial to provide the uncertainties of predictions. This is also called \"probabilistic forecasting\", as opposed to \"point forecasting\". This entails modeling a probabilistic distribution, from which one can sample.\n",

This file has been truncated. show original

I am using Jupyter notebook in VS Code, Python 3.10.9 and Jupyter 6.5.4 in a virtual environment.

Everything runs smoothly until the forward pass, where I get the following error (full traceback):

Traceback (most recent call last)
/var/folders/vq/0s913kg97lv155_r9j2g9qxm0000gn/T/ipykernel_63681/2795771002.py:2 in

[Errno 2] No such file or directory:
‘/var/folders/vq/0s913kg97lv155_r9j2g9qxm0000gn/T/ipykernel_63681/2795771002.py’

/opt/homebrew/lib/python3.10/site-packages/torch/nn/modules/module.py:1190 in _call_impl

│ 1187 │ │ # this function, and just call forward. │
│ 1188 │ │ if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks o │
│ 1189 │ │ │ │ or _global_forward_hooks or _global_forward_pre_hooks): │
│ ❱ 1190 │ │ │ return forward_call(*input, **kwargs) │
│ 1191 │ │ # Do not call functions when jit is used │
│ 1192 │ │ full_backward_hooks, non_full_backward_hooks = , │
│ 1193 │ │ if self._backward_hooks or global_backward_hooks: │
│ │
│ /opt/homebrew/lib/python3.10/site-packages/transformers/models/time_series_transformer/modeling │
│ time_series_transformer.py:1813 in forward │
│ │
│ 1810 │ │ if future_values is not None: │
│ 1811 │ │ │ use_cache = False │
│ 1812 │ │ │
│ ❱ 1813 │ │ outputs = self.model( │
│ 1814 │ │ │ past_values=past_values, │
│ 1815 │ │ │ past_time_features=past_time_features, │
│ 1816 │ │ │ past_observed_mask=past_observed_mask, │
│ │
│ /opt/homebrew/lib/python3.10/site-packages/torch/nn/modules/module.py:1190 in _call_impl │
│ │
│ 1187 │ │ # this function, and just call forward. │
│ 1188 │ │ if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks o │
│ 1189 │ │ │ │ or _global_forward_hooks or _global_forward_pre_hooks): │
│ ❱ 1190 │ │ │ return forward_call(*input, **kwargs) │
│ 1191 │ │ # Do not call functions when jit is used │
│ 1192 │ │ full_backward_hooks, non_full_backward_hooks = , │
│ 1193 │ │ if self.backward_hooks or global_backward_hooks: │
│ │
│ /opt/homebrew/lib/python3.10/site-packages/transformers/models/time_series_transformer/modeling │
│ time_series_transformer.py:1626 in forward │
│ │
│ 1623 │ │ use_cache = use_cache if use_cache is not None else self.config.use_cache │
│ 1624 │ │ return_dict = return_dict if return_dict is not None else self.config.use_return │
│ 1625 │ │ │
│ ❱ 1626 │ │ transformer_inputs, scale, static_feat = self.create_network_inputs( │
│ 1627 │ │ │ past_values=past_values, │
│ 1628 │ │ │ past_time_features=past_time_features, │
│ 1629 │ │ │ past_observed_mask=past_observed_mask, │
│ │
│ /opt/homebrew/lib/python3.10/site-packages/transformers/models/time_series_transformer/modeling │
│ time_series_transformer.py:1535 in create_network_inputs │
│ │
│ 1532 │ │ embedded_cat = self.embedder(static_categorical_features) │
│ 1533 │ │ # static features │
│ 1534 │ │ log_scale = scale.log() if self.config.input_size == 1 else scale.squeeze(1).log │
│ ❱ 1535 │ │ static_feat = torch.cat((embedded_cat, static_real_features, log_scale), dim=1) │
│ 1536 │ │ expanded_static_feat = static_feat.unsqueeze(1).expand(-1, time_feat.shape[1], - │
│ 1537 │ │ │
│ 1538 │ │ # all features │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: expected Tensor as element 1 in argument 0, but got NoneType

I am fairly new to Transformers (and Python in general) but I suspect the issue lies in relation to the error:

[Errno 2] No such file or directory:
‘/var/folders/vq/0s913kg97lv155_r9j2g9qxm0000gn/T/ipykernel_63681/2795771002.py’

I checked the directory, and indeed no such file exists. I tried upgrading ipykernel without success. There seems to be no issues loading all required libraries in their newest versions.

Any suggestions as to what the issue might be?

evasahlholdt · May 10, 2023, 8:23pm

I found the error for any other user who might run into this issue:
Follow the documentation on Installation and remember to brew install cmake and pkg-config when running on an M1