Using hyperparameter-search in Trainer

🤗Transformers
84

Hello,

I am using this code to find the best parameters for my model.

from ray.tune.schedulers import PopulationBasedTraining
from ray.tune import uniform
from random import randint

scheduler = PopulationBasedTraining(
    mode = "max",
    metric='exact_match', # mean_accuracy
    perturbation_interval=2,
    hyperparam_mutations={
        "weight_decay": lambda: uniform(0.0, 0.3),
        "learning_rate": lambda: uniform(1e-5, 5e-5),
        "per_gpu_train_batch_size": [3, 4, 5],
        "num_train_epochs": [10,11,12],
        "warmup_steps":lambda: randint(0, 500)
    }
)

best_trial = trainer.hyperparameter_search(
    direction="maximize",
    backend="ray",
    n_trials=4,
    keep_checkpoints_num=2,
    scheduler=scheduler
)

However, I am having this miskate. Do you have an advice?

/usr/local/lib/python3.7/dist-packages/pyarrow/io.pxi in pyarrow.lib.Buffer.__reduce_ex__()

AttributeError: module 'pickle' has no attribute 'PickleBuffer'

Some people recommend to use python 3.8 instead of python 3.7, however, this workaround did not help me to resolve the issue. I am working in Google Colab.

Thanks in advance.

3 Likes