Using the jpelhaw / t5-word-sense-disambiguation model

Beginners
1

I’m making my first attempt to use a Huggingface model and I’m having a few issues I’m hoping someone can help me out with. I need a word sense disambiguation algorithm and jpelhaw’s model seems like the only one on hf. I’m having a few issues getting the code to run.

First, even though the code presented seems to be python, the input value isn’t formatted in a way I understand. It looks like the key-value pairs of a dictionary, but the keys and values aren’t their own strings. Also, the “list” for the descriptions key has an or before the string in the last list item. (see below)

descriptions:[  " A drink consisting of an infusion of ground coffee beans " , 
                " a platform-independent programming lanugage "
                ,  or " an island in Indonesia to the south of Borneo " ]

This isn’t a format I’ve ever seen before. Is this the author’s own format for accepting inputs or is it some markup language I haven’t seen before?

Next, when I try to execute the line

answer = model.generate(input_ids=example['input_ids'], 
                               attention_mask=example['attention_mask'], 
                               max_length=135)

it tries to call example like a dictionary, but the previous line of code

example = tokenizer.tokenize(input, add_special_tokens=True)

returns example as a list. Looking through the program I don’t see any objects with the keys ‘input_ids’ or ‘attention_mask’, so i don’t know where those keys are coming from.

I feel like I’m at a dead end. Is there a different way I should be approaching this model or if it possible that this model is not ready for deployment yet. Also, and this is a really basic question, but is there a way I can look at the source code for this model so I can get a better idea what the author is trying to do or is that not a thing huggingface allows? Thanks for your help.

Eric

2

Welcome, @Usylesses! For the first question, I can definitely see the confusion! I think there are some issues in the sample code in the model card, but luckily they have a really cool Space (Word Sense Disambiguation - a Hugging Face Space by Belligerent) that demonstrates how we can use the model. I took a peek at the code for the space (app.py · Belligerent/word-sense-disambiguation at main) and it looks like you should be able to run the model with something like

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, Text2TextGenerationPipeline

pipe = Text2TextGenerationPipeline(
    model = AutoModelForSeq2SeqLM.from_pretrained("jpelhaw/t5-word-sense-disambiguation"),
    tokenizer = AutoTokenizer.from_pretrained("jpelhaw/t5-word-sense-disambiguation")
)

input = (
  """
  question: which description describes the word " java " best in the following context? \
  descriptions:[  " A drink consisting of an infusion of ground coffee beans " , 
                  " a platform-independent programming lanugage "
                  "or an island in Indonesia to the south of Borneo " 
  context: I like to drink " java " in the morning .
  """
)

output = pipe(input)[0]['generated_text']
print(output)

This is using an inference pipeline (Pipelines for inference) to do it with as little code as possible, but you could do it manually as well.

As for the format of the input, I think it’s just supposed to be a big string, and the model’s tokenizer might handle splitting it up into questions, descriptions, and context. And the model itself might then be able to ingest that info as it is. (e.g. In this other project, GitHub - patil-suraj/question_generation: Neural question generation using transformers, the T5 model expects inputs to be processed in a similar way)

This is tough to verify though, because I don’t know where the source code is! On the model repo only the weights are stored, and the actual model code might be somewhere else, e.g. on GitHub. Sometimes people will post a link to their repo on the model card, It might be worth reaching out to the model author over Twitter or email to get some more information.

I’m still learning about this stuff, but I’ve found that this section of the Hugging Face Course helped me out a bit (Question answering - Hugging Face Course).

I hope this helps a bit!

1 Like
3

Thank you so much for such an intelligent and thorough answer. This is exactly what I needed.

1 Like