6 and 12 layer English text generation models

Hi :slight_smile: I’m looking for decent 6 and 12 layer English text generation models. Anyone personally created any of these? This topic thread could be a ‘wanted’ avenue for folks looking for specific layers, heads etc. as they are not easy to syphon through in hugging search.
If you have any new ones like this that aren’t listed plz message, cheers.
Contact: tomchho@gmail.com