ValueError: Unrecognized configuration class <class 'transformers.models.whisper.configuration_whisper.WhisperConfig'>

Whisper model- Audio Classification

ValueError Traceback (most recent call last)
in <cell line: 5>()
3 num_labels = len(id2label)
4
----> 5 model = AutoModelForAudioClassification.from_pretrained(
6 ā€œopenai/whisper-baseā€,
7 num_labels=num_labels,

/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
465 token = hub_kwargs.pop(ā€œtokenā€, None)
466 use_auth_token = hub_kwargs.pop(ā€œuse_auth_tokenā€, None)
ā†’ 467 if use_auth_token is not None:
468 warnings.warn(
469 ā€œThe use_auth_token argument is deprecated and will be removed in v5 of Transformers. Please use token instead.ā€,

ValueError: Unrecognized configuration class <class ā€˜transformers.models.whisper.configuration_whisper.WhisperConfigā€™> for this kind of AutoModel: AutoModelForAudioClassification.
Model type should be one of ASTConfig, Data2VecAudioConfig, HubertConfig, SEWConfig, SEWDConfig, UniSpeechConfig, UniSpeechSatConfig, Wav2Vec2Config, Wav2Vec2ConformerConfig, WavLMConfig.