Train a GPT2 model for contextual common sense reasoning using the COSMOS QA dataset

Flax/JAX Projects
1

Idea:-A unique task wherein we train a GPT-2 model for contextual common sense reasoning using the COSMOS QA dataset.The goal is the test the GPT-2 for the common sense reasoning case.

Model:-Need to add support for ‘FlaxGPT2ForMultipleChoice’ model which can be easily done using the existing ‘FlaxGPT2Model’.

Dataset:-cosmos_qa · Datasets at Hugging Face available at huggingface.

Available training scripts:-Need to prepare a script for multiple choice training for our Flax Model.

Sample
Sample636×782 264 KB

We need some guidance for the project,we would love to hear feedback(@patrickvonplaten and @valhalla please do have a look) and we are very excited to contribute this task to huggingface.

We are currently a team of two:-1.Rohan.V.Kashyap(@Rohan)
2.Vivek.V.Kashyap(@Vivek)

2

I’m in.Really excited for this upcoming project @Rohan

1 Like
3

Awesome, let’s finalize it - 2 should be enough, but let’s hope more people will join :slight_smile:

2 Likes
4

Sure,Let’s make it final and see if more people wants to join and contribute to this project

1 Like
5

Thanks @patrickvonplaten,we are very much looking forward for this.We are confident of getting this done and hope more people can join our project and contribute for it.

6

@patrickvonplaten it would if you could can our project to the google sheets,we would like to make this a part of huggingface,we are be absolutely excited to get started with this.

1 Like
7

Great adding it!

8

I would love to be part of this project! I am a student with slim experience in Machine Learning, but I would love to help in whatever way I can!

2 Likes
9

I would love to be a part of this project, I have previously contributed to HF transformer. Currently, I am currently working as an ML engineer.

3 Likes
10

added you guys

1 Like
11

Just checking here: Are you guys active?

12

Hi @patrickvonplaten ,we are working on it.We have finished writing our training script for Bert multiple choice using flax and also our cosmosq&a dataset.it’s working perfectly fine.we just need access for the tpu and accomodate training the same with the gpt-2 multiple choice model.That should get completed by tomorrow.We have created the discord channel also.