[Open-to-the-community] Community week using JAX/Flax for NLP & CV :jax:

Flax/JAX Projects
21

Super excited for this!!! :heart_eyes: :heart_eyes: :heart_eyes:
training GPT2 in Bengali would be pretty huge for Bengali NLP research community.

Here’s the topic link: PreTrain GPT2 from scratch in Bengali

5 Likes
22

Unfortunately I will not be able to attend the talks from 30/06 - 02/07. Will they be recorded and made available?

23

I think so! (@Suzana might know better)

3 Likes
24

Hey @patrickvonplaten,

Would it be possible to train an mBART model from scratch in JAX/Flax?

Maybe only for a couple of languages, to fit the time frame.

25

Hey @bhavnicksm

Sure, why not!

mBART will be merged soon in JAX/Flax, but if you want to train from scratch you could also use BART or T5.

And yeah, starting with a few languages makes sense to fit the time frame.

1 Like
26

Can i train Wav2Vec2 from JAX?

27

Is there code for BART pre-training using huggingface?

28

No, we haven’t yet added BART pre-training script. T5 pre-training script should be available in week.

But if someone wants, feel free to take a shot at yet. The most important part is the bart denosing function. Then one could just leverage the run_summarization_script` with the denoising dataset to pre-train BART

29

Patrick is working on FlaxWav2vec2, but it will take some time since it’s a complex model and pre-training is also a bit complex.

2 Likes
31

Sure, try to take in as much as possible during the event!

32

Hi @Suzana, please share here if you have any updates on this. Thank you!

33

Wow, cool initiative. Is it late to participate? (sent the google form yesterday)

34

Nope, not at all, you could still join!

1 Like
35

Sound cool, submitted the form :grin:

1 Like
36

Awesome, now you could explore the project ideas and leave a comment there if you want to join that project :slight_smile:

38

Oh, I thought there’s will be a slack channel, isn’t it?

40

Yes, there’s a slack channel as well, you’ll be invited to it by tomorrow. But we use forum to post the ideas to make them visible to everyone.

2 Likes
41

Hi, I submitted the form, but haven’t received any slack channel invitation. Is this normal (am I too late?)

42

You should have been added this morning :slight_smile:

43

I hope this is not too late to sign up! :flushed:

1 Like