How to fine tune LUKE for NER?

Beginners
1

Hello. I am wondering if I can fine-tune LUKE with my own dataset of NER. I am aware that LUKE has a unique model so the code in the example notebook is off the table. I am aware that Studio Ousia has fine-tuning code with GitHub - studio-ousia/luke: LUKE -- Language Understanding with Knowledge-based Embeddings, but if I do that route, I am wondering if I can convert the new fine-tuned model to be transformers-compatible.

2

Hi Kerenza

Were you able to make any progress on fine-tuning LukeForEntitySpanClassification for custom labels? Actually, I am also looking to fine-tune Luke for a NER task with multi-token entities. Any help is much appreciated.

Thanks

3

Hi Read through Readme GitHub - studio-ousia/luke: LUKE -- Language Understanding with Knowledge-based Embeddings

4

Hi,

We now have an example script that illustrates how to fine-tune LUKE for NER (and other token classification tasks): transformers/examples/research_projects/luke at master · huggingface/transformers · GitHub

5

Hi @nielsr, I have been trying to run the script but it fails on conll2003 dataset during tokenization. Am i doing somthing wrong? Any specific versions I should use?

6

Hi @nielsr, Thanks for sharing the script. I was able to run the training on conll2003 dataset with small modifications. However, the performance is extremely low: {‘precision’: 0.8983516483516484, ‘recall’: 0.40541976620616366, ‘f1’: 0.5587014888943129, ‘accuracy’: 0.8970917690009956}

Dataset should not be an issue as I am using standard conll2033 dataset. I also tried exact hyper-parameters mentioned in the paper but with no luck. Here is the exact command I ran:

python run_luke_ner_no_trainer.py
–model_name_or_path studio-ousia/luke-base
–dataset_name conll2003
–task_name $TASK_NAME
–max_length 128
–per_device_train_batch_size 8
–learning_rate 1e-5
–num_train_epochs 5
–output_dir ./$TASK_NAME/

Any suggestions on how to match the performance reported in the paper would be highly appreciated.

Thanks in advance!

1 Like