I’m trying to fine tune facebook/esm2_t33_650M_UR50D · Hugging Face (and eventually a larger one) on SageMaker, using the Huggingface estimator, to make model parallel easier. I ran into the “unused parameter” problem and so added the appropriate argument to my hyper parameters. hyperparameters = {…

Find_unused_parameters parameter to Huggingface SM Estimator not doing anything?

mdc713 July 12, 2023, 7:05pm 3

No, no progress. It turns out the find unused params argument isn’t passed to the model constructor for SageMaker models. I filed an issue for that.

Topic		Replies	Views
XLNet pre-training fails with multiple GPUs on Sagemaker 🤗Transformers	0	254	July 9, 2023
Simple Fairscale Model Parallelization works locally, but using Sagemaker SMP gives me errors Amazon SageMaker	10	2210	June 27, 2022
Distributed training with Sagemaker 🤗Transformers	0	314	June 26, 2023
Package errors running huggingface estimator on sagemaker Beginners	1	951	February 9, 2023
Sagemaker huggingface estimator tries to import tensorflow when pytorch is defined 🤗Transformers	0	454	August 19, 2022