Finetuned I-Bert for question answering task

🤗Transformers
1

How can we fine-tune an Integer Bert for question answering task?

2

Hi,

As explained in IBERT’s model card: fine-tuning the model consists of 3 stages:

  1. full precision fine-tuning
  2. quantization
  3. quantization-aware training.

So for the first step, you can fine-tune IBERT just as any BERT model. You can take a look at the example scripts as well as the official QA notebook.

Step 2 is simply setting the quantize attribute of the model’s configuration to True.

Step 3 is the same as step 1, but now with your quantized model.