Progress bars shown despite disable_tqdm=True in Trainer

Beginners
1

I’m running HuggingFace Trainer with TrainingArguments(disable_tqdm=True, …) for fine-tuning the EleutherAI/gpt-j-6B model but there are still progress bars displayed (please see picture below). Does somebody know how to remove these progress bars?

training_args = TrainingArguments(
    disable_tqdm=True,
    output_dir='./checkpoints',
    save_total_limit=10,
    logging_dir='/content/logs',
    num_train_epochs=config["N_EPOCHS"],
    evaluation_strategy='epoch'
    save_strategy='steps',
    save_steps=30,
    logging_steps=10,
    overwrite_output_dir=True,
    per_device_train_batch_size=4,
    per_device_eval_batch_size=4,
    gradient_accumulation_steps=4,
    eval_accumulation_steps=4,
    gradient_checkpointing=True,
    max_grad_norm=0.5,
    lr_scheduler_type="cosine",
    learning_rate=1e-4,
    warmup_ratio=0.05,
    weight_decay=0.1,
    fp16_full_eval=True
    fp16=True,
    fp16_opt_level='O1',
    report_to=['tensorboard']
)

186213857-528e089b-1114-4490-a799-e2598604e1c3
186213857-528e089b-1114-4490-a799-e2598604e1c32279×1357 109 KB

2

Hi ! I believe these are progress bars of the dataset processing step before training, in particular the calls to map using the Hugging Face datasets library

You can disable them with

import datasets 
datasets.disable_progress_bar()
2 Likes
3

How do I pass disable_tqdm=True to Trainer’s TrainingArguments, without giving output_dir?
I am only instantiating Trainer so I can run trainer.predict() right after, I don’t want Trainer to save anything in anywhere.