Inference Hyperparameters

philschmid · September 15, 2021, 3:26pm

Yes,
The Parameter would be correct, but the current released transformers only support padding, and truncation as **kwargs if you take a look here:

github.com

huggingface/transformers/blob/a5fc34437dc2cfbd9e91f732820304c017b5525d/src/transformers/pipelines/base.py#L742

    
      
                      if isinstance(model, tuple):
                          supported_models_names.extend([_model.__name__ for _model in model])
                      else:
                          supported_models_names.append(model.__name__)
                  supported_models = supported_models_names
              if self.model.__class__.__name__ not in supported_models:
                  logger.error(
                      f"The model '{self.model.__class__.__name__}' is not supported for {self.task}. Supported models are {supported_models}."
                  )
          
          
def _parse_and_tokenize(
              self, inputs, padding=True, add_special_tokens=True, truncation=TruncationStrategy.DO_NOT_TRUNCATE, **kwargs
          ):
              """
              Parse arguments and tokenize
              """
              # Parse arguments
              if getattr(self.tokenizer, "pad_token", None) is None:
                  padding = False
              inputs = self.tokenizer(
                  inputs,

But the next transformers version will allow all **tokenizers_kwargs to be passed in.

Topic		Replies	Views
Predict function ignore parameters Amazon SageMaker	8	1172	January 28, 2022
How are the inputs tokenized when model deployment? Amazon SageMaker	13	4278	September 3, 2021
How to deploy a T5 model to AWS SageMaker for fast inference? Amazon SageMaker	13	5795	February 28, 2022
Deploying Open AI's whisper on Sagemaker Amazon SageMaker	54	16200	April 12, 2024
About the Amazon SageMaker category Amazon SageMaker	25	4102	August 5, 2021

Inference Hyperparameters

Related topics