Demo of Open Domain Long Form Question Answering

Beginners
1

Hi,
I just wanted to try the demo https://huggingface.co/qa/, but it seems it is down. Could we have it again please? Thanks.

2

The demo is back on, thanks for letting us know!

3 Likes
3

Hi @yjernite, thanks for your great works and great blog post!

At the moment, the system is down again, so would it be possible to have Colab or Kaggle notebook, so that we can use our own GPU?

EDIT It’s back! So sometimes the server just down or some problem on GPU … But standalone notebook would be great anyway :slight_smile:

4

Hi @Jung, there is a notebook for it: https://github.com/huggingface/notebooks/blob/master/longform-qa/Long_Form_Question_Answering_with_ELI5_and_Wikipedia.ipynb. I followed the instructions on the notebook, and after several hours (donwloading, indexing, etc, and using pre-trained model) the system is ready to get question.

2 Likes
5

Thanks @cahya for the pointer!!
HF team indeed has many great notebooks where I missed :slight_smile:
I will definitely try it out!

6

Hi Yacine, @yjernite

I believe the demo has been down (I have waited for 24 hours),

https://huggingface.co/qa/
return 502 Bad Gateway

7

Hi, I have dockerized the engine, and provide a rest api to use it, if you want to try https://hub.docker.com/r/wirawan/eli5
I will include later a front end in to the docker image

1 Like
8

@cahya amazing job! Thanks for sharing. :smiley:

BTW, did you further finetune the model or use as is ?
and which beam-number did you use in the telegram bot ?

9

Thanks. I use the model as it is without fine tuning. I use beam number 16, with max_len of 256 to reduce the text generation time. I also use cpu for the faiss, so the whole service can fit to my 8GB GPU ram. Actually it needs now only 5GB gpu ram instead of 13GB when I used gpu for the faiss index.

2 Likes
10

Hi @jung @cahya, just catching up now :slight_smile:

Thanks @cahya for the work on the docker image!

@Jung the demo is supposed to restart when it crashes but that doesn’t always work, and then I have to restart it manually :wink: We’ll eventually change it so it uses our inference API to make it more stable.

I’d be super interested to know what you’re using it for in the meantime!

1 Like
11

Hi @yjernite, I am trying to use it with some books instead of wiki. I will write it here if it works :slight_smile: Until now, I used it in a telegram chatbot for fun (https://github.com/cahya-wirawan/AskMeAnything-Chatbot).

1 Like
12

Hi @yjernite, I did some experiments with the demo. It seems that the Bart model trained for this demo doesn’t really take the retrieved passages as source for its answer. It likes to hallucinate :slight_smile: for example if I ask “what is cempedak fruit”, the answer doesn’t contain any information from the retrieved passages. I think it generates text mostly only from the question and its Barts parameter.

1 Like
13

So that’s a really interesting question: the model tends to rely on documents for some questions and hallucinate more on other - if you play around, you can see that it sometimes gets a good answer using supporting documents even for examples that aren’t in the training set, but it also needs to rely on its own knowledge from time to time which can lead to mistakes.

Definitely an open area of investigation!

2 Likes
14

Btw, you have to try the fruit, it’s really delicious. I think @Jung can confirm it :slight_smile:

2 Likes