Super excited for this!!! 
training GPT2 in Bengali would be pretty huge for Bengali NLP research community.
Here’s the topic link: PreTrain GPT2 from scratch in Bengali
5 Likes
Unfortunately I will not be able to attend the talks from 30/06 - 02/07. Will they be recorded and made available?
Hey @patrickvonplaten,
Would it be possible to train an mBART model from scratch in JAX/Flax?
Maybe only for a couple of languages, to fit the time frame.
Hey @bhavnicksm
Sure, why not!
mBART will be merged soon in JAX/Flax, but if you want to train from scratch you could also use BART or T5.
And yeah, starting with a few languages makes sense to fit the time frame.
1 Like
Can i train Wav2Vec2 from JAX?
Is there code for BART pre-training using huggingface?
No, we haven’t yet added BART pre-training script. T5 pre-training script should be available in week.
But if someone wants, feel free to take a shot at yet. The most important part is the bart denosing function. Then one could just leverage the run_summarization_script` with the denoising dataset to pre-train BART
Patrick is working on FlaxWav2vec2, but it will take some time since it’s a complex model and pre-training is also a bit complex.
2 Likes
Sure, try to take in as much as possible during the event!
Wow, cool initiative. Is it late to participate? (sent the google form yesterday)
Nope, not at all, you could still join!
1 Like
Sound cool, submitted the form 
1 Like
Awesome, now you could explore the project ideas and leave a comment there if you want to join that project 
Oh, I thought there’s will be a slack channel, isn’t it?
Yes, there’s a slack channel as well, you’ll be invited to it by tomorrow. But we use forum to post the ideas to make them visible to everyone.
2 Likes
Hi, I submitted the form, but haven’t received any slack channel invitation. Is this normal (am I too late?)
You should have been added this morning
I hope this is not too late to sign up! 
1 Like