Smallest pretrained model?

What is the smallest English pre-trained model (not distilled)?

BERT-tiny is pretty, uh, tiny (around 16MB).

1 Like