Regression prediction is not between 0 and 1

Beginners
1

Dear all,

I trained a regression model using XLMRobertaForSequenceClassification and it performed well, with predictions that were close to the actual labels. However, when I trained it on a large dataset, it produced scores that were “below zero and above 1”, which is not possible based on the training samples (I have already conducted a thorough exploratory data analysis). I did some research and found a thread on stackoverflow where people suggested using Softmax to restrict the output between zero and one, but I believe this may not be the correct approach as Softmax may disregard middle data points (i.e. between 0 and 1). Currently, I am using logits to calculate metrics like the following:

def compute_metric(eval_pred):
    logits, labels = eval_pred
    logits = np.squeeze(logits)
    labels = np.squeeze(labels)

    pearson_correlation= pearson_corr(logits, labels)
    spearman_correlation= spearman_corr(logits, labels)
    rmse= mean_squared_error(labels, logits, squared=False)
    mse=  mean_squared_error(labels, logits)

    return {
        'pearson_correlation':pearson_correlation,
        'spearman_corr': spearman_correlation,
        'rmse': rmse,
        'mse':  mse
    }

I am wondering if anyone else has experienced the same problem and could provide some insight. Thank you!

[update]: I am thinking about using ReLU to check if it is an effective solution!