Modeling bart JAX vs Pytorch/Tensorflow implementation

smallsuper · September 11, 2023, 4:15am

I am comparing the implementation of Bart in JAX vs PyTorch/TensorFlow. I realized causal masking is not done in the JAX implementation compared to PyTorch and TensorFlow. Is there a reason for this?

Topic		Replies	Views
Why does Bart decoder's attention mask mark relevant indices with 0 instead of 1? Models	1	1776	May 31, 2021
Cannot reproduce the results Beginners	5	808	October 5, 2020
How is padding masking considered in the Attention Head of a Transformer? 🤗Transformers	0	2132	December 6, 2022
SDPA attention in e.g. Llama does not use fused accelerations 🤗Transformers	0	120	March 5, 2024
New Paper: Masked Autoencoders Are Scalable Vision Learners Research	0	1237	November 14, 2021

Modeling bart JAX vs Pytorch/Tensorflow implementation

Related Topics