I’m trying to use BERT for custom token classification, I have annotated data for this task using doccano, the exported dataset in jsonl format looks like this: {"text": "EU rejects German call to boycott British lamb.", "label": [ [0, 2, "ORG"], [11, 17, "MIS"] ]} {"text": "Peter Blackburn", "labe…

I found the solution, using this doccano/doccano-transformer from the docanno team. There is a catch though, it has an annoying bug solved by this , and since there is next to no documentation, we can use this to write the file. :ok_hand:

Doccano dataset for named entity recognition task using BERT

Beginners

nielsr May 8, 2024, 11:51am 2

Hi,

That’s already great! Citing myself here NER model fine tuning with labeled spans - #3 by nielsr

Topic		Replies	Views
Do I have to only tokens in Bert dataset for token classification 🤗Datasets	0	130	January 18, 2024
Cannot get DataCollator to prepare tf dataset 🤗Transformers	0	476	July 15, 2022
Tutorial: Fine-tuning with custom datasets – sentiment, NER, and question answering 🤗Transformers	19	12620	February 12, 2024
Multi-input tag and ,multi-label output for token classification using Bert pretrained model 🤗Transformers	1	52	January 9, 2025
How to deal with differences between CoNLL 2003 dataset tokenisation and BER tokeniser when fine tuning NER model? Intermediate	6	2648	November 23, 2021

Doccano dataset for named entity recognition task using BERT

Related topics