Is it possible to see what batch size is being used in deepspeed training with auto batch size?
48%|██████████████████████████████████▏ | 13/27 [06:48<09:26, 40.49s/it]
To test, I’m training 1 epoch. 1 epoch in happening in 27 steps. There are 1700 training examples, so it is 63 examples per step. Does this mean that the batch size is 1700/27 ~ 63, and the micro batch size is 63/8 ~= 8 (there are 8 GPUs)? I believe “batch” is defined as the number of training examples involved with a gradient accumulation and weight update, while micro batch is number of examples being processed by a single GPU at once.