python 3.9.7
datasets 2.5.2
I am running into an issue whenever I try to add a column to a dataset I am working with. Below is some sample code:
import numpy as np
z_train = np.array(ds['train']['re_part'].numpy(), dtype=complex)
z_train.imag = ds['train']['im_part'].numpy()
ds['train'].add_column(name='complex_col', column=z_train)
ds['train']['re_part'] and ds['train']['im_part'] are initially pytorch tensors which is why I needed to convert them to numpy arrays first. I get the following error:
Exception has occurred: ArrowNotImplementedError
Unsupported numpy type 15
and I’m not entirely sure what that is referring to. I checked the types of z_train and they are all numpy arrays until you get to the individual elements, in which case they are numpy.complex128. Any ideas? Thanks in advance for your help!