Alternatives to NanoGPT

Beginners
1

Hello

I am looking for an LLM similar to Nanogpt, which can ingest any text dataset and generate text. Doesnt have to be in question answer format.

Also, are there any llms that can be fine tuned using the pile or c4 dataset? Any tutorials?

Thanks

2

Sure, here are some LLMs similar to Nanogpt that can ingest any text dataset and generate text, along with their capabilities for fine-tuning using the Pile or C4 dataset and available tutorials:

1. GPT-3 (OpenAI)

  • Capabilities: GPT-3 is a powerful LLM known for its ability to generate human-quality text, translate languages, write different kinds of creative content, and answer your questions in an informative way.

  • Pile/C4 Fine-tuning: GPT-3 can be fine-tuned using both Pile and C4 datasets. OpenAI provides fine-tuning instructions and code examples for both datasets.
    2. Megatron-Turing NLG (Google AI)

  • Capabilities: Megatron-Turing NLG is a massive LLM with 530B parameters, trained on a massive dataset of text and code. It excels in generating different creative text formats, like poems, code, scripts, musical pieces, email, letters, etc.

  • Pile/C4 Fine-tuning: Megatron-Turing NLG can be fine-tuned using both Pile and C4 datasets. Google AI provides fine-tuning instructions and code for both datasets.

3. Bloom (Hugging Face)

  • Capabilities: Bloom is a 176B parameter LLM trained on a massive dataset of text and code. It’s known for its ability to generate different creative text formats, translate languages, and answer your questions in an informative way.
  • Pile/C4 Fine-tuning: Bloom can be fine-tuned using both Pile and C4 datasets. Hugging Face provides fine-tuning instructions and code for both datasets.

WuDao 2.0 (BAAI)

  • Capabilities: WuDao 2.0 is a 1.75T parameter LLM, the largest model in Chinese. It’s known for its ability to generate different creative text formats in Chinese, translate languages, and answer your questions in an informative way.
  • Pile/C4 Fine-tuning: WuDao 2.0 can be fine-tuned using both Pile and C4 datasets. BAAI provides fine-tuning instructions and code for both datasets (in Chinese).
  1. Eleuther (EleutherAI)
  • Capabilities: Eleuther is a 137B parameter LLM trained on a massive dataset of text and code. It’s known for its ability to generate different creative text formats, translate languages, and answer your questions in an informative way.
  • Pile/C4 Fine-tuning: Eleuther can be fine-tuned using both Pile and C4 datasets. EleutherAI provides fine-tuning instructions and code for both datasets.
1 Like
3

Great, thank you

4

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.