ValueError: Unable to avoid copy while creating an array as requested

🤗Datasets
5

Hi @lhoestq,

I actually still get same error with the following versions:
Transformers version: 4.47.1
Datasets version: 3.2.0
Numpy version: 2.2.1
Fasttext version: 0.9.3

Pandas version: 2.2.3
Scikit-learn version: 1.6.0

It wasn’t the case last week. The only package that was updated is Numpy, so I would guess something’s there.

Bellow is my code:

def predict_label(text, model, language_mapping_dict, use_mapping=False):
    # Remove any newline characters and strip whitespace
    text = str(text).strip().replace('\n', ' ')
    
    if text == '':
        return 'Other'
    
    try:
        # Get top prediction
        prediction = model.predict(text, 1)
        
        # Extract label and remove __label__ prefix 
        label = prediction[0][0].replace('__label__', '')


        # Extract confidence score
        confidence = prediction[1][0]
        
        # map label to language using language_mapping_dict
        if use_mapping:
            label = language_mapping_dict.get(label, 'Other')
        return label
    
    except Exception as e:
        print(f"Error processing text: {text}")
        print(f"Exception: {e}")
        return {'prediction_label': 'Error', 'prediction_confidence': 0.0}
 
# download model and get the model path
model_path = hf_hub_download(repo_id=model_path, filename=model_path_bin, cache_dir=None)

# Load the trained model
print(f"[INFO] Loading model from Path: {model_path}, using version {model_path_bin}...")
model = fasttext.load_model(model_path)

# Predict labels using the model
print(f"[INFO] Running predictions...")
df_eval['preds'] = df_eval['text'].apply(lambda text: predict_label(text, model, language_mapping_dict, use_mapping=use_mapping))

Thanks!!

2 Likes