Hi everyone, I’m working on an overview for non-tech people about LLM training and want to illustrate the need for the humongous amount of data that is required for training LLMs. Do you know where I can find training checkpoints of rather popular LLMs? The illustration I have in mind would be: “Without any training at all (at random init) the output of the model for input […] is […]. And this is the output for the same input after seeing 100 B tokens: […], this after 300 B: […], …, and this is the output of the final model for the same input sequence: […]”
Alternatively: Has anyone published such an “evolution” of outputs over training?